HOA-2/SlicerNeuroSegmentation

Add timer for segmentation

Closed this issue · 9 comments

Sunderlandkyl commented

Spun off from: #2

If possible, it would be great to get a timer so we can see how long it takes for each segment. Is that possible? We've been using phones to time.

Likely we would track the amount of time that a segmentation spends in the "In Progress" state during segmentation. Only time when the segment is selected in the editor.

lassoan commented

We could add a segmentation monitoring module (probably Python scripted) that would observe all segment editor nodes in the scene and collect timing statistics. Results could be continuously written to a csv file (either a table node that is saved with the scene, or into a filename specified in application settings).

For a shared computer, it may be important to store name of the operator. Information about the current user can be stored/retrieved by using slicer.app.applicationLogic().GetUserInformation(). User information can be viewed/edited in Application settings / User section, but if we rely on this information then the segmentation monitoring module could show a popup on application startup asking the operator to confirm that the information is correct.

It would make things simpler if the csv files collected in many computers could be easily merged, so columns could be something like this:

  • computer
  • date&time
  • scene file
  • segmentation node name
  • selected segment name (maybe terminology fields as well)
  • operation (selected effect? but probably we should record the time that is spent with just browsing slices, too)
  • duration in seconds
  • 👍1
Sunderlandkyl commented

Any particular repository for this module?

lassoan commented

Probably start this as a module specific to this project, so we can put it in this repository. Later we will see if it can be generalized.

Sunderlandkyl commented

Made an initial version of the Segmentation Timer module. (https://github.com/PerkLab/BwhNeuroImagingCollab/tree/master/SegmentationTimer)

CSV looks like this:

"computer","starttime","scene","segmentation","segment","terminology","operation","duration"
"vein","08/06/2019, 16:50:40","","","","","",8.46263
"vein","08/06/2019, 16:50:48","","vtkMRMLSegmentationNode1","","","",2.92506
"vein","08/06/2019, 16:50:50","","vtkMRMLSegmentationNode1","Segment_1","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","",1.66843
"vein","08/06/2019, 16:50:51","","vtkMRMLSegmentationNode1","Segment_1","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Paint",7.86438
"vein","08/06/2019, 16:50:58","","vtkMRMLSegmentationNode1","Segment_1","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Erase",4.95033
"vein","08/06/2019, 16:51:02","","vtkMRMLSegmentationNode1","Segment_2","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Erase",2.06433
"vein","08/06/2019, 16:51:04","","vtkMRMLSegmentationNode1","Segment_2","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Draw",10.5452
"vein","08/06/2019, 16:51:14","","vtkMRMLSegmentationNode1","Segment_2","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Paint",2.29067
"vein","08/06/2019, 16:51:16","","vtkMRMLSegmentationNode1","Segment_3","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Paint",1.30444
"vein","08/06/2019, 16:51:17","","vtkMRMLSegmentationNode1","Segment_3","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Threshold",3.38272
"vein","08/06/2019, 16:51:20","","vtkMRMLSegmentationNode1","Segment_3","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","",2.68179
"vein","08/06/2019, 16:51:22","","vtkMRMLSegmentationNode1","Segment_3","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Paint",3.73138
"vein","08/06/2019, 16:51:25","","vtkMRMLSegmentationNode1","Segment_3","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","Erase",5.198
"vein","08/06/2019, 16:51:30","","vtkMRMLSegmentationNode1","Segment_3","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","",0.201004

Looks like this within the module:
image

Some things that still need to be implemented:

  • Additonal columns:
    • User
    • Scene
  • Start recording when Slicer is started. Don't wait for module to be opened.
  • Update Segmentation node + segment identifier to use human readable names rather than node/segment id. (Or perhaps include both)
  • Screenshot time-lapse options.
  • Create new row in table when terminology is changed
Sunderlandkyl commented

Updated format and example:

"startTime","operation","durationSec","segmentName","masterVolumeName","segmentationName","segmentTerminology","moduleName","userActivity","computer","userName","scene"
"20190808-160028","",1.24,"","","","","Welcome","wait","vein","Kyle",""
"20190808-160028","",5.6028,"","","","","Welcome","active","vein","Kyle",""
"20190808-160033","",5.51084,"","","","","UserStatistics","active","vein","Kyle",""
"20190808-160038","",0.543849,"","","","","SegmentEditor","active","vein","Kyle",""
"20190808-160038","",4.01293,"","","Segmentation","","SegmentEditor","active","vein","Kyle",""
"20190808-160042","",1.433,"","","Segmentation","","SampleData","wait","vein","Kyle",""
"20190808-160043","",2.84828,"","","Segmentation","","SampleData","active","vein","Kyle",""
"20190808-160045","",2.65686,"","","Segmentation","","SegmentEditor","active","vein","Kyle",""
"20190808-160047","",0.978472,"","MRHead","Segmentation","","SegmentEditor","active","vein","Kyle",""
"20190808-160047","",1.61988,"Segment_1","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160048","Paint",2.85839,"Segment_1","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160050","Paint",2.7232,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160052","Draw",2.41981,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160054","Erase",10.0161,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160104","Erase",10.42,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","wait","vein","Kyle",""
"20190808-160114","Erase",4.08481,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160118","",15.5201,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","UserStatistics","active","vein","Kyle",""
"20190808-160133","",2.9517,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160135","",162.364,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","idle","vein","Kyle",""
"20190808-160417","",1.31958,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160418","Paint",1.02341,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160419","Paint",3.117,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","wait","vein","Kyle",""
"20190808-160422","Paint",3.5303,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160425","Erase",8.9328,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","SegmentEditor","active","vein","Kyle",""
"20190808-160433","",0.935801,"Segment_2","MRHead","Segmentation","Segmentation category and type - 3D Slicer General Anatomy list~SRT^T-D0050^Tissue~SRT^T-D0050^Tissue~^^~Anatomic codes - DICOM master list~^^~^^","UserStatistics","active","vein","Kyle",""

The module now detects 3 different states of user activity:

  • Active: The default state when the user is interacting with Slicer
  • Wait: The state when the main thread has been occupied for (> 1.1 seconds)
  • Idle: The state when the user has not created a mouse or keyboard event in 30 seconds
Sunderlandkyl commented

Added the option to merge existing user statistic tables, and to record screenshots as new rows are created. Currently, if no rows have been created in the last 20 seconds, then a new row is automatically created, even if no parameters have changed.

Current appearance of the module:
image

Sunderlandkyl commented

When the application starts, the user is provided with a dialog to change the current user information:
image

Requires Slicer/Slicer#1193

Sunderlandkyl commented

@lassoan I think that covers all of the features that we discussed.
Are there any other features/improvements that you would like to see before we start actively using this?

lassoan commented

Thank you, these all look great to me! It should be started to be used for segmentations.

@sbouix Let us know if you need help in setting this up.