Micro-SAM Repository Summary | Segment Anything for Microscopy

📊 Quick Stats

697

Stars

105

Forks

Sibling Repos

Authors

🏢 About the Project

Authors: Anwai Archit, Paul Hilt, Genevieve Buckley, Constantin Pape

Published in: Nature Methods (2024)

Organization: Computational Cell Analytics Lab @ Uni Göttingen

License: MIT — you can fork, modify, or build on top

What It Does

🔬 Segment Anything for Microscopy

micro-sam is a Python package and napari GUI plugin that adapts Meta's Segment Anything Model (SAM) to microscopy images. Instead of segmenting natural photos, it segments cells, mitochondria, synapses, and other biological structures — with just a few clicks.

2D Interactive Segmentation — click on cells / mitochondria and SAM produces a mask
3D Interactive Segmentation — slice-by-slice segmentation across volumes (e.g. EM data)
Tracking — segment cells across time-lapse movie frames and link them into tracks (via trackastra)
Finetuning — scripts to adapt SAM to your own microscopy domain
Automatic segmentation — generate masks with a grid of points without manual clicks
Pretrained checkpoints — "generalist" and domain-specific finetuned SAM models

Installation

📦 How to Install

# Via pip
pip install micro_sam

# Via conda (recommended for napari)
conda install -c conda-forge micro_sam

# Then launch the napari GUI
micro_sam.annotator_2d

Requires Python ≥ 3.10, PyTorch ≥ 2.5, and a CUDA-capable GPU for best performance.

What You Can Implement

🛠️ Feasibility Breakdown

The full repo is a multi-year, multi-author research product. But many individual pieces are very doable. Here's a realistic breakdown:

Easy SAM inference pipeline — call SAM with image encoder + prompt encoder + mask decoder (~200 lines)

Easy Finetuning SAM on your own microscopy data (well-documented in sibling peft-sam repo)

Easy 2D interactive segmentation logic — run SAM per click, keep mask state, accept/reject

Easy Automatic segmentation with a grid of points/boxes (built into SAM)

Medium 3D interactive segmentation — slice-by-slice + 3D aggregation logic

Medium Tracking pipeline — SAM segmentation over frames + trackastra pairing

Medium Embedding precomputation / caching for performance

Hard Full napari plugin — PyQt/Qt widget project with image layers, 3D/4D tooling, HDF5/Zarr I/O

Hard Model zoo + compatibility layer across SAM backbone variants (vit-huge, vit-base, vit-tiny)

No Reproducing their specific finetuned checkpoint weights (requires their exact training data + GPU time)

Dependencies & Sibling Repos

🐍 Core Stack

Package	Purpose
`segment-anything`	SAM model (Meta)
`torch` / `torchvision`	Deep learning framework
`napari`	Interactive image viewer GUI
`torch-em`	Electron microscopy utilities
`trackastra`	Cell tracking
`scikit-image`	Image processing
`zarr` / `h5py`	Large image I/O

🔗 Sibling Repos (Same Org)

Repo	Stars	Purpose
patho-sam	67 ★	SAM for histopathology
peft-sam	35 ★	LoRA/finetuning recipes for SAM
medico-sam	31 ★	SAM for medical imaging
synapse-net	12 ★	Synaptic structure reconstruction
dl-for-micro	24 ★	Deep learning for microscopy course

⚖️ Verdict: Can You Implement This?

Implement the whole repo? — NO — it's a multi-year, multi-author research product (697 stars, 105 forks, 36 sibling repos).

Implement an interactive SAM segmentation tool for your own microscopy images? — YES — very doable, possibly a weekend-to-weeks project depending on GUI polish.

Implement finetuning + automatic segmentation for a specific domain? — YES — straightforward and well-documented in their sibling repos.

💡 Better path: Since it's MIT-licensed, consider using or extending micro-sam itself rather than reimplementing. The core value is in the finetuned checkpoints and domain-specific training recipes — not the GUI scaffolding.

Citation

📄 How to Cite

If you use this repo in research, cite:

The Nature Methods paper (2024)
The original Segment Anything publication (Kirillov et al., 2023)
If using vit-tiny models: Mobile SAM
If using automatic tracking: Trackastra

Micro-SAM: GitHub Repository Inspection