Improve `ImageConverter` performance
Opened this issue · 1 comments
Converting a 2208x1242 RGB image to BGR using the airo_camera_toolkit is much slower (~100x) than the equivalent OpenCV cvtColor method.
The reason is that we store the image as 0-1 float images as an intermediate step. This requires 4x the memory and also a division by 255.0 for all pixels. For this reason I've started to use cvtColor in my performance critical code.
I'm not really sure what the best solution is. We want to keep the ImageConverter for easy torch integration, but maybe we should take a more functional approach like OpenCV and remove the intermediate storage as a float-image?
@tlpss thoughts?
yeah reason for this 'unified' intermediate is to keep implementation complexity low. My first thought is to mention this in the code? This is a simple implementation for convenience, if you need high performance, consider using optimized methods such as opencv? Because if you work on GPU, even opencv will be slow.. so would be hard to support the best option for each case
If you want to provide better performance by dropping the intermediate layer by adding more logic to the class, that is also fine! we could store the image in the format we get it, and then write optimized conversions for all types? But still I would add the warning to the image class!