MouseLand/cellpose

[BUG] model.eval with CPU returns None if insufficient memory

Opened this issue · 4 comments

nick-youngblut commented

Describe the bug

model = models.Cellpose(gpu=False, model_type="nuclei")
masks, _, _, _ = model.eval(image, diameter=diameter)

If < ~80 Gb of memory provided for the job, then masks is None. If sufficient memory is provided then masks is not None.

model.eval is not throwing an insufficient memory error if there is not enough memory.

To Reproduce

See above

Run log

cellpose version: 	3.0.9
platform:       	linux
python version: 	3.11.9
torch version:  	2.1.2.post100
2024-07-30 14:15:36,303 - Masking image...
2024-07-30 14:15:36,304 - >>>> using CPU
2024-07-30 14:15:36,304 - >> nuclei << model set to be used
2024-07-30 14:15:36,344 - >>>> loading model models/nucleitorch_0
2024-07-30 14:15:36,438 - >>>> model diam_mean =  17.000 (ROIs rescaled to this size during training)
2024-07-30 14:15:36,480 - channels set to [0, 0]
2024-07-30 14:15:36,481 - ~~~ FINDING MASKS ~~~
2024-07-30 14:15:38,842 - >>>> TOTAL TIME 2.36 sec

None was returned for the masks variable for this logged run. I provided 8 Gb of memory. If I instead provide 80 Gb of memory, a mask object is returned instead of None.

Conda env

name: cellpose
channels:
  - conda-forge
  - bioconda
dependencies:
  - python=3.11.9
  - pip=24.0
  - bokeh=3.4.2
  - opencv=4.8.1
  - holoviews=1.19.0
  - ipython=8.25.0
  - matplotlib=3.8.4
  - numpy=1.26.4
  - psutil=6.0.0
  - scikit-image=0.24.0
  - tifffile=2024.6.18
  - xmltodict=0.13.0
  - tqdm=4.66.4
  - scipy=1.10.1
  - pims=0.7
  - cellpose=3.0.9
  - pip:
    - aicspylibczi==3.1.2
    - glob2
    - pprintpp
carsen-stringer commented

sorry I think you must be using another wrapper on top of cellpose (our code doesn't say "Masking image...") please try again without the wrapper and let us know if it's still not working

nick-youngblut commented

@carsen-stringer 2024-07-30 14:15:36,303 - Masking image... is just a loggin.info call prior to running:

model = models.Cellpose(gpu=False, model_type="nuclei")
masks, _, _, _ = model.eval(image, diameter=diameter)

There is no wrapper.

carsen-stringer commented

Thanks hmm let me see if I can replicate this in order to debug it. How big is the image?

nick-youngblut commented

Thanks for reopening the issue. I believe the image was ~10 Gb.