Once a release is cut, napari is distributed in two main ways:
Packages: both to PyPI and conda-forge.
Installers: bundles that include napari plus its runtime dependencies in a step-by-step executable.
Despite its numerous dependencies,
napari itself is a simple Python project that can be packaged
in a straight-forward way.
Creating and submitting the packages to PyPI (the repository you query when you do
is handled in the
.github/workflows/make_release.yml workflow. Creation is
make dist (as specified in our
Makefile) and submission is done using the
official PyPA GitHub Action. This workflow will also create a GitHub Release.
Once the Python package makes it to PyPI, it will be picked by the
conda-forge bots, which will
automatically submit a PR to the
napari-feedstock repository within a few hours. This is all
automated by the
conda-forge infrastructure (see previous examples), so we only need to
check that the metadata in the recipe has been adjusted for the new release. Pay special attention
to the runtime dependencies and version strings!
We keep a copy of the feedstock’s recipe in the
napari/packagingrepo, which is updated manually whenever a change to
setup.cfgis detected. Check the file
conda-recipe/meta.yamland make sure its
outputsare synced to the
Once the conda-forge CI is passing and the PR is approved and merged, the final packages will be
built on the default branch and uploaded to the
conda-forge channel. Due to the staging steps and
CDN synchronization delays, the conda packages can take up to 1h to be available after the
We also build nightly packages off
main and publish them to the
These are the same packages that are used in the
constructor installers (see below), so their CI
is specified in
To do it in a
conda-forge compatible way, we actually clone
napari-feedstock and patch the
source instructions so the code is retrieved from the repository branch directly. The version is
also patched to match the
setuptools-scm string. After rerendering the feedstock, we run
conda-build in the same way
conda-forge would do and upload the resulting tarballs to our
Additionally, the tarballs are also passed as artifacts to the next stage in the pipeline: building
constructor installers (more below).
Once the packages have been built and uploaded to their corresponding repositories, we can bundle
them along with their dependencies in a single executable that end users can run to install napari
on their systems, with no prior knowledge of
conda, virtual environments or anything.
A software installer is usually expected to fulfill these requirements:
It will install the application so it can be run immediately after.
It will provide a convenient way of opening the application, like a shortcut or a menu entry.
It will allow the user to uninstall the application, leaving no artifacts behind.
Right now, we are using two ways of generating the installers:
briefcase, which takes PyPI packages.
constructor, which takes
conda packages offer several advantages when it comes to bundling dependencies, since it makes
very few assumptions about the underlying system installation. As a result,
include libraries that might be missing in the target system and hence should provide a more robust
briefcase based installers are marked for deprecation so we will not discuss them here.
If you are curious, you can check
constructor to build the
napari installers through the
.github/workflows/make_bundle_conda.yml workflow, which only specifies the triggers used to call
the actual workflow implementation under the
napari/packaging repository. This repository stores
all the logic and files needed to create the nightly
conda packages and the
constructor allows you to build cross-platform installers out of
conda packages. It
supports the following installer types:
On Linux, a shell-based installer is generated. Users can execute it with
On macOS, you can generate both PKG and shell-based installers. PKG files are graphical installers native to macOS, and that’s the method we use with napari.
On Windows, a graphical installer based on NSIS is generated.
The configuration is done through a
construct.yaml file, documented here. We generate one on
the fly in the
build_installers.py script found in
Roughly, we will build this configuration file:
# os-agnostic configuration name: napari version: "0.0.1" # this is the _internal_ version of the installer infrastructure company: Napari license: EULA.md channels: # - local # only in certain situations, like nightly installers where we build napari locally - napari/label/bundle_tools # temporary location of our forks of the constructor stack - conda-forge specs: # specs for the 'base' environment - python # pinned to the version of the running interpreter, configured in the CI - conda # we add these to manage different napari versions - mamba # we add these to manage different napari versions - pip # we add these to manage different napari versions extra_envs: napari-0.4.16: # this is the environment that will actually contain the napari packages specs: - napari=0.4.16=*pyside* - napari-menu=0.4.16 - python # pinned to a specific version, configured by CI - conda # needed for the plugin manager - mamba # needed for the plugin manager - pip # needed for the plugin manager menu_packages: - napari-menu # don't create shortcuts for anything else in the environment # linux-specific config default_prefix: $HOME/napari-0.0.1 # default installation path # macos-specific config default_location_pkg : Library # first component of the default path under ~/ pkg_name: napari-0.0.1 # second component of the default path installer_type: pkg # otherwise, defaults to sh (Linux-like) welcome_image: resources/napari_1227x600.png # bg image with the napari logo on bottom-left corner welcome_file: resources/osx_pkg_welcome.rtf # rendered text in the first screen conclusion_text: "" # set to an empty string to revert constructor customizations back to system's readme_text: "" # set to an empty string to revert constructor customizations back to system's signing_identity_name: "Apple Developer ID: ..." # Name of our installer signing certicate # windows-specific config welcome_image: resources/napari_164x314.png # logo image for the first screen header_image: resources/napari_150x57.png # logo image (top left) for the rest of the installer icon_image: napari/resources/icon.ico # favicon for the taskbar and title bar default_prefix: '%USERPROFILE%/napari-0.0.1' # default location for user installs default_prefix_domain_user: '%LOCALAPPDATA%/napari-0.0.1' # default location for network installs default_prefix_all_users: '%ALLUSERSPROFILE%/napari-0.0.1' # default location for admin installs signing_certificate: certificate.pfx # path to signing certificate
The main OS-agnostic keys are:
channels: where the packages will be downloaded from. We mainly rely on conda-forge for this, where
napariis published. However, we also have
napari/label/bundle_tools, where we store our
constructorstack forks (more on this later). In nightly installers, we locally build our own development packages for
condawithout resorting to
conda-forge. To make use of those (which are eventually published to
napari/label/nightly), we unpack the GitHub Actions artifact in a specific location that
constructorrecognizes as a local channel once indexed.
extra_envs> napari-$VERSION: the environment that will actually contain the napari installation. In this key, you will find
specs, which lists the conda packages to be installed in that environment. Constructor will perform a conda solve here to retrieve the needed dependencies.
menu_packages: restrict which packages can create shortcuts. We only want the shortcuts provided by
napari-menu, and not any that could come from the (many) dependencies of napari.
Then, depending on the operating systems and the installer format, we customize the configuration a bit more.
Default installation path¶
This depends on each OS. Our general strategy is to put the general installation under
~/<hidden>/napari-<INSTALLER-VERSION>, which will eventually contain the napari installations under
envs/, with environments named as
napari-<VERSION>. However, there are several constrains we
need to take into account to make this happen:
On Windows, users can choose between an “Only me” and “All users” installation. This changes what we understand by “user directory”. This is further complicated by the existence of “domain users”, which are not guaranteed to have a user directory per se.
On macOS, the PKG installer does not offer a lot of flexibility for this configuration. We will put it under
~/Library/napari-<INSTALLER-VERSION>, by default.
This means that if you install
napari=0.4.16 using the installer, the actual
can be found, by default, on the following locations:
Graphical installers can be customized with logos and icons. These images are stored under the
resources/ directory (outside of the source), with the exception of the square logos/icons (which
are stored under
napari/resources/ so the shortcuts can find them after the installation).
Some of the steps are also configured to display a custom text, like the license or the welcome screen on macOS.
In order to avoid security warnings on the target platform, we need to sign the generated installer.
On macOS, once Apple’s Installer Certificate has been installed to a keychain and unlocked
for its use, you can have
constructor handle the signing via
However, this is not enough for a warning-free installation, since its contents need to be
notarized and stapled too. For this reason,
constructor has been modified to also
codesign the bundled
_conda.exe (the binary provided by conda-standalone, see below) with
the Application Certificate. Otherwise, notarization fails. After that, two actions take care
of notarizing and stapling the resulting PKG.
On Windows, any Microsoft-blessed certificate will do. Our
constructor fork allows us to specify
a path to a PFX certificate and then have the Windows SDK
signtool add the signature. Note that
signtool is not installed by default on Windows (but it is on GitHub Actions).
More details about our packaging infrastructure can be found in the NAP-2 document.
Details of our
constructor stack fork¶
Note: All these changes have been sent upstream. See progress in this issue.
Many of the features here listed were not available on
constructor when we started working on it.
We have added them to the relevant parts of the stack as needed, but that has resulted in a lot of
moving pieces being juggled to make this work. Let’s begin by enumerating the stack components:
constructoris the command-line tool that builds the installer. It depends on
condato solve the
specsrequest. It also requires a copy of
conda-standalone(a PyInstaller-frozen version of
conda) to be present at build time so it can be bundled in the installer. This is needed because that
conda-standalonecopy will handle the extraction, linking and shortcut creation when the user runs the installer on their machine.
conda-standaloneis a frozen version of
conda. Among its dependencies, we can find
menuinst, which handles the creation of shortcuts and menu entries.
menuinstwas only used on Windows before our work, so we basically rewrote it to handle cross-platform shortcuts.
menuinstto delegate the shortcut creation. Since this was only enabled on Windows, we needed to unlock the other platforms and rewrite the parts that assumed Windows only behaviour. Surprise, this involved custom solver behaviour too!
menuinst is frozen together with
conda-standalone, every little change in any
of those requires a rebuild of
constructor can find the new version during
the installer creation. As a result, we needed to fork and repackage all four components!
Notice the repackaging needs. It’s not enough to fork and patch the code. We also need to create
the conda packages and put them in a channel so the downstream dependencies can pick them when they
are rebuilt. This repackaging is done through a separate
conda-forge clone that only handles our
forks. It is configured to use GitHub Actions (instead of Azure) and upload to the
For example, if a patch is introduced in
menuinst, the following needs to happen before it makes
it to the final napari installer:
Write and test the patch. Make sure it passes its own CI.
condastill works with the new changes. It needs to call
menuinstpackage and upload it to Anaconda.org.
Rebuild and upload
conda-standaloneso it picks the new
Trigger the napari CI to build the new installer.
Very fun! So where do all these packages live?
Most of the forks live in
jaimergp’s account, under a non-default branch. They are published
jaimergp-forge every time a commit to
master is made. Versions are arbitrary here,
but they are set to be greater than the latest official version, and the
build number is
incremented for every rebuild.
The only exception is
conda-standalone. It doesn’t have its own repository or fork because it’s
basically a repackaged
conda with some patches. Those patches live in the feedstock only. The
other difference is that the feedstock does not live in
jaimergp-forge, but just as draft PR in
conda-forge original feedstock. This is because, for some reason, if
built on GitHub Actions machines, the Windows version will fail with
_ssl errors which do not
appear in Azure. For this reason, the CI is run as normal on
conda-forge, and then the artifacts
are retrieved from the Azure UI and manually uploaded to the
napari channel. Fun again!
Eventually all these complexities will be gone because all of our changes will have been merged upstream. For now, this not the case. Speaking of which, what are our changes? Below you can find a high-level list of the main changes introduced in the stack.
Add API support for menuinst v2
Enable code paths for non-Windows Platforms
Fix shortcut removal logic
--shortcuts-onlyflag to support
menu_packagesconstructor key natively
Unvendor menuinst patches
Do not vendor constructor NSIS scripts
conda constructorentry point for the new menuinst API
Add branding options for macOS PKG installers
_conda.exein the install location
Do not offer options for conda init or PATH manipulations (these should be Anaconda specific)
Add signing for Windows
Add notarization for macOS PKG