What are types of microcopes MMCore does not support or does not support well currently?
Opened this issue · 7 comments
We encountered two main limitations in Micro-Manager:
- High-speed acquisition with sCMOS, I believe it has improved a bit last year (less data copying). I can't develop too much unfortunately as I was not involved in that part in the lab. Maybe light-sheet people around will have better input. Since light-sheet is such an important field in bioimaging, this should be one of the focus for a future version.
- Multi-camera acquisitions. We had both an EMCCD and an sCMOS camera on the same set-up, with different sensor sizes (due to the hardware) and probably image pixel size as well (actually don't remember). Our hacky work-around was to use MM1.4 for the EMCCD and MM2 for the sCMOS in parallel. This is again relevant for light-sheet (multiview microscopes with multiple sCMOS cameras).
We did wide-field superresolution microscopy, although I am mentionning light-sheet microscopy a lot here. ^^
I second the request to have multiple simultaneous cameras supported without the limitation of them sharing the same sequence buffer. Also the use case of rapid acquisitions with massive camera data throughput.
Analog input devices are poorly supported and I've been involved in email conversations with Nico and Mark about that previously, and it's noted in the whitepaper.
Our main issues usually end up being with storing 'unexpected' data types, usually FLIM or OPT datasets, where one of the axes isn't XYZCT, and will either end up replacing one of the 'standard' ones (e.g. OPT theta to 'standard' z) or being modulo along it e.g. for FLIM, time gates along the T axis. Having some sort of common suggested metadata names for these kind of things from the get-go might not be a bad idea, as otherwise there'll probably be about 10 different conventions to merge later?
Multiple cameras are also an issue, for a variety of applications.
I second acquisition of multiple sCMOS cameras at high framerates could use some improvement. I second to have separate sequence buffers for separate cameras or be able to assign cameras to buffers. As far as acquisition speed, I did see some improvement at high framerates after the fix in threading and memory leak issues at the beginning of the year. Perhaps, one thing that could be useful and some DeviceAdapters lack but it's implemented by the vendor's APIs is the control of bandwidth throughput and limit/adjust.
Metadata handling seems to slow down in the current pipeline even if tweaking the JVM garbage collectors at high framerate. (i.e TaggedImage vs Image) . However, without metadata such as timestamps, image counter, camera ID, etc. is trickier to sort data for saving with multiple cameras.
We should have smarter handling of metadata listed somewhere. I imagine starting out with the full set of metadata, and from then on only attaching the metadata that actually changed. This system could perpetuate all the way into storage. This would allow for reconstruction of the full set of metadata for any given image, but with much less storage and cpu cycles involved to do so.
I work on the OpenFlexure Microscope, which is controlled by an integrated Raspberry Pi; that means all the hardware control happens on a remote computer, via HTTP commands. I think this design pattern (integrated HTTP server and network control) is one that I've seen adoped in a bunch of places, and it would be super to make it easier to write lightweight device adapters for HTTP instruments, in the same way that serial is already nicely supported.
I suspect a first step to figuring this out, though, is me writing a set of device adapters for the OFM using the existing MMCore, which I'm yet to find time to do...
A belated thank you to all of you for all this helpful feedback!
We're finally moving forward on a new features that address many of these limitations. Please feel free to chime in and get involved at: https://github.com/micro-manager/mmCoreAndDevices/issues