sciai-lab/covid-if-annotations

[User Feedback] General feedback and observations

Opened this issue · 0 comments

Collecting some of the user feedback and my first impression after going through the initial 22 uploads.

Feedback:

Viewer / Tool:

  • Overall tool seems to be easy enough to use.
  • Fast mouse clicks are not registered, see #29.
  • Points are too large and hide part of the cell. (Comment from me: yes, but making them smaller makes it hard to click them).
  • Cell outlines are too thick for the small cells (Comment from me: ideally we would use inter-pixel contours and even resize them dynamically depending on the zoom level)
  • Correcting segmentation errors can be very time consuming
  • Yellow color for "uncertain" is hard to see in display night mode

Segmentation:

  • Cell segmentation overall looks good.
  • Some systematic issues with the cell segmentation:
    • Cells in mitosis are over segmented into 2 or even more parts (this is caused by the nucleus segmentation used for seeds; getting rid of this is part of the reason for the whole exercise.)
    • Cells sometimes "bleed out" into the background

Observations:

  • Overall annotations look good
  • Instead of correcting the segmentations, most annotators flag objects with segmentation errors as uncertain (given that large segmentation errors are not so common and take some time to correct, this should be fine)
  • Some annotators label a lot of cells as uncertain, including cells that are clearly not infected. In these cases, I have proofread the annotations, because the training data is already skewed towards infected cells and we might have insufficient numbers of labeled non-infected cells if half of them are labeled as uncertain.