Common API: Coordinate transformations
Closed this issue · 4 comments
I'm a pretty firm believer that a good hardware abstraction system should be able to model the physical relationships between devices (and in many cases, the optical relationships). For example, let's say I have a camera mounted on a scope with a motorized objective turret, focus, and xy stage. Every image I collect from the camera should have metadata describing its position, orientation, and scale relative to some global / sample coordinate system, such that I can easily ask how the position of a pixel in one frame relates to the pixels in another frame. I can also use this for control, such as "move the stage/focus such that the center of the camera sensor is at this specific 3D position", or "move this pipette tip to the position imaged by a specific pixel".
This is made possible by modeling physical relatioships -- the camera is mounted to the scope, which is mounted to a focusable stage, which is connected to an objective changer (where we know the optical power and physical offset of each objective). The microscope and xy stage are also rigidly connected (via the table), so we know the position of the sample relative to the objective as well.
This seems obvious to me, but I want to hear what you-all think..
I agree, depending on to what level. For cameras I stop at pixel size (as measured in the image) and orientation, I didn't include anything more sophisticated like a first order transform between the coordinate systems of two or more cameras that are mounted on the same microscope. Also, I reference everything to the XY stage position, and orientation is defined by the direction the stage moves.
I'm a convert to @campagnola 's solution to this and love it in ACQ4, but I'm definitely familiar with systems where the cross-device coordinate modeling isn't needed. I can also imagine scenarios where adding this functionality into a driver could impact performance.
correct me if I'm wrong here Luke, but I think this information/code can stay well away from the driver-level code. If it needs to be in the driver-level ABCs, maybe we can make the global coordinate updates and transformations switchable (i.e. stop computing or requesting them) if the driver performance is compromised by them.
Yeah, acq4's transforms are implemented in the hardware abstraction layer, not in the lower-level device wrappers. It's a good point, though--handling these transforms isn't free, and they would probably get in the way if you didn't need them.
More generally, I think it's going to be a challenge to come up with abstract device APIs that everyone likes unless they are as minimal as possible. I'm leaning towards saying that these should not be a part of the abstract device classes, so long as it's functionality that could be added on top elsewhere (either in another application-specific layer, or as attachable components).
Closing this as out-of-scope; would love to revisit it later in another context.