iTranslated by AI
Image Analysis: Cell Segmentation from Microscopy Images with Cellpose (1)
Cellpose is a tool that uses machine learning to separate (segment) small units such as cells one by one.
This article introduces the process from Cellpose installation to segmentation using pre-trained models, as well as the method for retraining it to suit your own data.
Note that this validation was performed on Windows 11.
Anaconda must be installed beforehand, and an environment where Anaconda Prompt can be used must be prepared.
【Installation】
Follow the official pages below to proceed with the installation of Cellpose version 2.0.
https://github.com/mouseland/cellpose
https://pypi.org/project/cellpose/
Creating a Virtual Environment for Cellpose
Launch Anaconda Prompt and create a virtual environment named "cellpose" with the following command. *Note: The GitHub page recommends Python version 3.8, so v3.8 is specified here, but it seems to work with v3.8 or later as well.
conda create --name cellpose python=3.8
Installation Screen
You will be asked whether to proceed with the download during the process; type y.

Installation Screen

Installation Screen - Finished
Installing the Cellpose Package
First, enter the cellpose virtual environment.
conda activate cellpose
Then, install cellpose into the cellpose virtual environment.
python -m pip install cellpose
Installation Screen

Start screen: Downloading of required related packages begins.

Finish screen
Installing the Cellpose GUI
To use the GUI version, which allows mouse-operated use of Cellpose, the following additional installation is required.
python -m pip install cellpose[gui]
Additional required related packages will be installed.
Installation Screen

Finish screen
Updating Cellpose
Update to the latest version. As of July 2023, the latest version appears to be v2.2.2.
python -m pip install cellpose --upgrade

【Launching the Cellpose GUI】
To launch the Cellpose GUI, run the following command.
python -m cellpose

Cellpose launched
Logs from the Cellpose GUI operations will be displayed in the Anaconda Prompt, so it is helpful to proceed while checking them.
【Prediction with Pre-trained Models】
Cellpose is often demonstrated with fluorescent cell images and similar data, but here we will verify the usefulness of Cellpose using images of cultured cells taken with a phase-contrast microscope.

Validation data
cell diameter
In Cellpose, you need to specify the approximate diameter of the objects to be separated in pixels. Using an incorrect value will significantly reduce prediction accuracy, so this is a key item to consider.

Clicking the calibrate button will use a pre-trained model called cyto to determine the diameter. For this demo image, 48.2 px seems to be appropriate.

Channel
Specify the channel to be used for separation in chan to segment:. You can choose from "0: gray", "1: red", "2: green", and "3: blue". In chan2 (optional):, you can choose from "0: none", "1: red", "2: green", and "3: blue". If you are not using channel 2, "0: none" is fine.

The choice of channels depends on the machine learning model. It is best to know which channel each model used for training and then use a similar channel during prediction.
- However, it may still work even if it is not exactly the same as at the time of training.
Executing Prediction
Select a pre-trained model from the model zoo in the left menu. (The first time you select a model, it will be downloaded, which takes extra time.)

Running predictions with the preset models yielded the following results. Round cells are detected well, but elongated cells are not detected properly.
| 「cyto」 | 「cyto2」 |
|---|---|
![]() |
![]() |
【Retraining】
Manually create the correct segmentation masks and perform retraining.
Operations, including mask manipulation, can be checked under Help > Help with GUI.

Some key operations are highlighted below.
| Item | Operation |
|---|---|
| Select mask | Click on mask |
| Delete mask | Ctrl+Click on mask |
| Delete all masks | Ctrl+0 |
| Merge masks | With one mask already selected, Alt+Click on a new mask |
| Draw mask | Right-click once -> Move mouse cursor to draw mask outline -> Return to start point or right-click again |
| Save mask data | Ctrl+S. A file named image_name_seg.npy will be exported. |
| Display RGB channels | Press R, G, B on the keyboard. Can also be changed from Views in the left menu |
| Show/Hide masks | X on the keyboard. Can also be changed from Drawing in the left menu |
| Show mask outlines | Z on the keyboard. Can also be changed from Drawing in the left menu |
| Display flows or prediction probabilities | PageUp/PageDown buttons on the keyboard. Can also be changed from Views in the left menu |
| Change brush size |
, on the keyboard to size down, . on the keyboard to size up. Can also be changed from Drawing in the left menu |
Labeling
Since it is difficult to redo everything manually from scratch, first perform a prediction with a suitable model and then correct the parts with poor accuracy.
If there is no prediction mask for the target, create a new one. Right-click to create a starting point and move the cursor to draw the mask outline.

Creating a mask
If the shape of the prediction mask is insufficient, create a mask in the missing area and Alt+click to merge the two.

Merging masks
Of course, you can also delete a mask with Ctrl+click and redo it from the beginning.
This is how the entire image was corrected.

Saving Masks
Save the mask data via the top menu File ➔ Save masks and image (as *_seg.npy) or by pressing Ctrl+S.

A file named "image_name_seg.npy" will be created in the same location as the image file.
Since training is performed on the images within the folder, you can create several training images.
Executing Training
Click Models ➔ Train new model with image+masks in folder in the top menu. The Train setting window will appear.
| Train setting | |
|---|---|
![]() |
![]() |
There are several parameters, but I proceeded with the default settings this time.

Training start screen

Training finish screen
When training is finished, the new model is applied to the currently open image as well. For areas where predictions are still insufficient, adjust the labels again and retrain.

Prediction using Retrained Model
Retrained models can be selected from custom models in the left menu. Click run model to execute the prediction.

I applied this prediction model to another image, and it seems a good model was created.
| Before prediction | After prediction |
|---|---|
![]() |
![]() |
Demo data for this article can be obtained from here.
To use the downloaded model in your own Cellpose GUI, select the model file via the Cellpose GUI menu Models > Add custom torch model to GUI.
【Using GPU】
Retraining a single image for 100 epochs using the CPU took more than 30 minutes. To make it run faster, this section introduces how to enable GPU support in Cellpose. (Cellpose uses PyTorch internally, so it's a matter of ensuring PyTorch can recognize the GPU.)
Installation commands introduced on GitHub
Installation commands from the GitHub page
In my case, this method did not work, but I will introduce it anyway.

- To install the GPU version of torch, first uninstall the CPU version of torch.
- Cellpose will be unavailable until you reinstall torch.
pip uninstall torch
- Reinstalling Pytorch
conda install pytorch pytorch-cuda=11.6 -c pytorch -c nvidia
Error and solution
In my environment, the following error occurred:
CondaSSLError: OpenSSL appears to be unavailable on this machine. OpenSSL is required to
download and install packages.
Based on this post, I solved it by copying and pasting "C:\ProgramData\Anaconda3\Library\bin\libcrypto-1_1-x64.dll" and "C:\ProgramData\Anaconda3\Library\bin\libssl-1_1-x64.dll" into C:\ProgramData\Anaconda3\DLLs.
- Install cellpose from GitHub
pip install git+https://www.github.com/mouseland/cellpose.git
Character encoding error encountered in the middle.
UnicodeDecodeError: 'cp932' codec can't decode byte 0xef in position 16542: illegal multibyte sequence

Installing PyTorch
Reinstall PyTorch using the installation command from the official PyTorch website.

Select your OS, CUDA version, and other details to obtain the installation command.
- Launch Anaconda Prompt and enter the virtual environment.
conda activate cellpose
- Update PyTorch with the
--updateoption.
pip3 install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
-
Check the output of
pip listin the Anaconda Prompt and verify that the torch version has a CUDA version suffix like +cu118.

-
Furthermore, verify whether the GPU is recognized as follows; if it returns True, there is no problem.
python
import torch
torch.cuda.is_available()

Using GPU Mode
Relaunch the Cellpose GUI. If "use GPU" is checked in the "Segmentation:" section of the left panel, the preparation for using the GPU is complete.
python -m cellpose

From here, the operations for prediction and model training remain the same. The model training, which took more than 30 minutes above, was completed in less than 3 minutes using the GPU.

【Moving to Measurement in ImageJ】
Since Cellpose does not have a function to measure segmented ROIs, it is necessary to pass the ROI information to image analysis software. Two types of data for ImageJ can be obtained from Cellpose. You can use either file, but the method for importing them into ImageJ differs for each.
Method 1
Save the outline information as a text file via File ➔ Save Outlines as text for imageJ, and recreate the ROIs in ImageJ using the ImageJ plugin (imagej_roi_converter.py) provided by Cellpose.
The outline information is recorded with the X and Y coordinates of one object per line, alternating.

outlines.txt
Reference
https://qiita.com/Naka24-sun/items/f7d47171657629431a45#良好な-segmentation-結果を得るために
https://satoshithermophilus.hatenablog.com/entry/2017/06/12/195827
Executing Python from the ImageJ text editor
1. Open the image in ImageJ
First, open the image you want to overlay the outlines on.

2. Open a Text Window
File ➔ New ➔ Text Window

3. Set the Language to Python (Jython)

4. Copy and paste the contents of "imagej_roi_converter.py"
If you have already saved the .py file locally, you can also open it via File ➔ Open.
from ij import IJ
from ij.plugin.frame import RoiManager
from ij.gui import PolygonRoi
from ij.gui import Roi
from java.awt import FileDialog
fd = FileDialog(IJ.getInstance(), "Open", FileDialog.LOAD)
fd.show()
file_name = fd.getDirectory() + fd.getFile()
print(file_name)
RM = RoiManager()
rm = RM.getRoiManager()
imp = IJ.getImage()
textfile = open(file_name, "r")
for line in textfile:
lineText = line.rstrip()
if not lineText:
continue
xy = map(int, line.rstrip().split(","))
X = xy[::2]
Y = xy[1::2]
imp.setRoi(PolygonRoi(X, Y, Roi.POLYGON))
# IJ.run(imp, "Convex Hull", "")
roi = imp.getRoi()
print(roi)
rm.addRoi(roi)
textfile.close()
rm.runCommand("Associate", "true")
rm.runCommand("Show All with labels")

5. Run
Click the Run button. The file explorer will open; select the text file containing the outline coordinates.

It will be reflected in ImageJ instantly.

How to use it as a plugin
1. Register as an ImageJ Plugin
Plugins ➔ Install.

The file explorer will open; select the "imagej_roi_converter.py" file. In the next screen for saving to the plugins folder, select Save.

2. Restart ImageJ
If "imagej_roi_converter" exists in the Plugins tab, it's a success.

3. Open the image in ImageJ
Open the image you want to overlay the outlines on.

4. Run the imagej_roi_converter plugin
The file explorer will open; select the text file containing the outline coordinates. It will be reflected in ImageJ instantly.

Method 2
The zip file obtained from File ➔ Save outlines as .zip archive of ROI files for ImageJ can be opened directly from the ImageJ ROI Manager. In this case, no plugin is required.
Specifically, follow these steps:
- Open the original image in ImageJ.
-
Analyze➔Tools➔ROI Manager. - Open the ROI zip file from
More➔Openin the ROI Manager.
Method 3: Creating an instance mask image from the _seg.npy file
If you were unable to obtain ImageJ data from the Cellpose File menu, this method can be used as an alternative.
The "image_name_seg.npy" file already contains the mask information. We will process this file with Python to create an instance mask image.
The following script is for creating an "_label.png" image from the "_seg.npy" files located in the current directory.
import numpy as np
import imageio
from glob import glob
npfiles = glob("*_seg.npy")
for npfile in npfiles:
# Get filename
filename = npfile[:-8]
# Load npy file
mask = np.load(npfile, allow_pickle=True)
# A dict type is obtained
mask = mask.tolist()
# Export using imageio to support grayscale images of 8-bit or higher
imageio.imwrite(f"{filename}_label.png", mask["masks"])
To proceed from the Anaconda Prompt, follow these steps:
- Save the above script (npy2mask.py) in the folder where the "_seg.npy" files are located.
- Use the
cdcommand to move to the directory containing the "_seg.npy" files. - Execute the Python file:
python npy2mask.py
When you open the exported instance mask image in ImageJ, you can see an image where each ROI has a different intensity value.

Applying instance masks to images using the ImageJ LabelsToRois plugin
Next, use the exported mask image to create ROIs in ImageJ. By using the LabelsToRois plugin, you can attach ROIs calculated from the instance mask image to the original image.
Downloading the plugin
Click the Download.zip link on the top page of the link above.

Extract the downloaded zip file and copy the "Labels_To_Rois.py" file found inside into the ImageJ plugins folder.
Launch ImageJ; if "Labels_To_Rois" appears under Plugins, it's a success.
Single image version
-
Launch
Labels_To_Roisand clickSingle image.

-
Enter the paths for the original image and the instance mask image, then click Next.

-
The image with ROIs and the ROI Manager will launch.

-
Measuring and saving ROIs
ROI Erosion can be additionally performed from theLabelToRoi - ROI erotionwindow.
To save the created ROIs, clickSave ROIs. They will be automatically exported locally. You can call up the ROIs by opening "image_name_Erosion_○px_RoiSet.zip" from the ROI Manager's OPEN.
Select measurement items viaSet measurements, and the measurement results for the ROIs will be saved locally viaSave CSV Table.
Batch processing version for multiple images
When you specify a folder, it searches for image files containing "image_name_labels" and processes them in batch, including the original images.
- Launch
Labels_To_Roisand clickMultiple images. - Specify the folder path.
If you have settings for Erosion or measurement items, set them here.
- Pressing Run will perform everything in batch up to exporting the results.






Discussion