Bioimage visualization in Python

Bioimage visualization in Python#

Python has a rich selection of data visualization tools that cover a wide range of applications, for example Matplotlib (Hunter, 2007), Mayavi (Ramachandran & Varoquaux, 2011), ipyvolume, the yt Project (Turk et al., 2010), ITK (Johnson, McCormick, Ibanez 2015), and more recently napari.

For bioimage visualization some major challenges are:

working with large and complex images: image size and dimensionality
manual interactivity: for human in the loop annotation
interactive analysis: for interactive parameter tuning and quality control

This notebook will introduce napari a fast, interactive, multi-dimensional image viewer for Python.

Introducing napari#

napari is a fast, interactive, multi-dimensional image viewer for Python. It’s designed for browsing, annotating, and analyzing large multi-dimensional images. It’s built on top of Qt (for the GUI), vispy (for performant GPU-based rendering), and the scientific Python stack (NumPy, SciPy).

napari includes critical viewer features out-of-the-box, such as support for large multi-dimensional data, layering, and annotation. By integrating closely with the scientific Python ecosystem, napari can be easily coupled to leading machine learning and image analysis tools (e.g. scikit-image, scikit-learn, PyTorch), enabling more user-friendly automated analysis.

napari supports seven different layer types, Image, Labels, Points, Vectors, Shapes, Surface and Tracks. Each layer corresponds to a different data type and has its own set of visualizations and interactive controls. We provide an associated tutorial for each layer type to help you get started!

You can add multiple layers of different types into the viewer and work with them, adjusting their properties and performing analysis.

napari also supports bidirectional communication between the viewer and the Python kernel, which is especially useful when launching from jupyter notebooks or when using our built-in console. Using the console allows you to interactively load and save data from the viewer and control all the features of the viewer programmatically.

You can (and are encouraged to!) extend napari using custom key bindings, mouse functions, and our new plugin-interface.

Learn more about napari at napari.org, including our tutorials, our API documentation and our mission and values.

(Optional) Preparing to run this notebook on mybinder.org #

# this cell is required to run these notebooks on Binder. Make sure that you also have a desktop tab open.
import os
if 'BINDER_SERVICE_HOST' in os.environ:
    os.environ['DISPLAY'] = ':1.0'

Visualizing data with napari#

Let’s start by importing napari and creating an empty napari viewer

import napari
viewer = napari.Viewer()

Hopfully when you ran the above command a new empty napari viewer appeared in a separate window.

Unlike other jupyter widgets, napari is not embedded inside the jupyter notebook. This is because the graphical parts of napari are written in Qt, making it hard to embed on the web.

Instead, we can take a screenshot of the current state of napari viewer and embed that in the notebook. This can be useful for teaching or sharing purposes where you might want to share key steps in an analysis which makes use of interactive components.

To do this, we use the nbscreenshot utility function

from napari.utils import nbscreenshot

nbscreenshot(viewer)

Note

Unfortunately, in contrast with the real napari viewer, these screenshots won’t be interactive!

Seeing our first image#

There are a few different ways to load images to into our viewer.

By dragging and dropping image files onto the viewer
By selecting image files from the Open File(s) menu option
Using the viewer.open command with a file path from within the notebook
Loading the image data into an array and then passing that array using the viewer.add_image command

For the first three options the file path will get passed through our fileIO plugin interface, allowing you to easily leverage highly customized fileIO plugins for your diverse needs. The fourth option allows you complete control over loading and visualization and is most suited for when you have data already loaded into your notebook from other sources.

Here we will explore the fourth option, explicitly loading a 3D image using the tifffile library and the add_image() method of our Viewer object.

from tifffile import imread

# load the image data and inspect its shape
nuclei = imread('data/nuclei.tif')
print(nuclei.shape)

(60, 256, 256)

Now that we have the data array loaded, we can directly add it to the viewer.

## directly adding image data to the napari viewer
viewer.add_image(nuclei)

<Image layer 'nuclei' at 0x7fe13d5abcd0>

Don’t forget to change windows so you can now see the viewer. By default you’ll just be looking at the middle plane of the 3D data, which a z-stack with 60 slices. You should see a single slider at the buttom of the viewer that will allow you to scroll through the rest of the z-stack. If you find the 30th slice then you should see the same as in the screenshot below.

nbscreenshot(viewer)

In the top left hand corner of the viewer we now have a control panel with controls that cover all our layers, and those that are specific to images like contrast limits and colormap.

Color channels and blending#

Right clicking on the contrast limits slider pulls up an elongated version of the slider which you can type specific numbers into. Let’s give that a try to adjust the contrast limits to [0.07, 0.35], and let’s change the colormap to blue using the drop down menu.

nbscreenshot(viewer)

One of the real strengths of napari is that you have full control over all the critical layer properties both programmatically and via the GUI.

Each layer that is added to the viewer can be found in the viewer.layers list.

# let's see what the layers list has in it right now
print(viewer.layers)

[<Image layer 'nuclei' at 0x7fe13d5abcd0>]

The layer list can be indexed either numerically or by the layer name, which is visible in the panel in the bottom left of the viewer. This layer has the name nuclei, which was automatically imputed because we originally named the variable we loaded from disk nuclei. Pretty cool!

nuclei_layer = viewer.layers['nuclei']
first_layer = viewer.layers[0]
nuclei_layer, first_layer

(<Image layer 'nuclei' at 0x7fe13d5abcd0>,
 <Image layer 'nuclei' at 0x7fe13d5abcd0>)

If we go in and get the nuclei layer from our layer list we can now see and edit the values of some of the properties that we can control in the GUI.

# let's look at the values of some of the properties on the `nuclei` layer
print('Colormap: ', viewer.layers['nuclei'].colormap)
print('Contrast limits: ', viewer.layers['nuclei'].contrast_limits)
print('Opacity: ', viewer.layers['nuclei'].opacity)

Colormap:  colors=ColorArray([[0., 0., 0., 1.],
            [0., 0., 1., 1.]], dtype=float32) name='blue' interpolation=<ColormapInterpolationMode.LINEAR: 'linear'> controls=array([0., 1.], dtype=float32)
Contrast limits:  [0.07, 0.35]
Opacity:  1.0

# Now let's change some of them. Note that the viewer GUI will update in real time as you run this code in the notebook
viewer.layers['nuclei'].colormap = 'red'
viewer.layers['nuclei'].contrast_limits = [0.4, 0.6]
viewer.layers['nuclei'].opacity = 0.9

# We can even rename the layer, but note that from now on you'll need to refer to if with its new name
viewer.layers['nuclei'].name = 'division'

nbscreenshot(viewer)

We could have actually passed these parameters as key-word arguments to during the first add_image call. For example we can add another copy of the data as follows:

viewer.add_image(nuclei, contrast_limits=[0.07, 0.35], colormap='blue', blending='additive')

<Image layer 'nuclei' at 0x7fe158d0d6d0>

Setting the blending of the second layer to additive now lets us see both together, which could be useful for understanding how parts of the image relate to each other.

nbscreenshot(viewer)

Let’s now load in an additional channel of data containing a stain for cell membranes and add them to the viewer as a new layer.

from tifffile import imread

# load the image data and inspect its shape
membranes = imread('data/cell_membranes.tif')
print(membranes.shape)

(60, 256, 256)

viewer.add_image(membranes, contrast_limits=[0.02, 0.2], colormap='green', blending='additive');

nbscreenshot(viewer)