NVIDIA/ai-assisted-annotation-client

3D slicer client always use the first image when close scene and reload new image

eri820503 opened this issue · 7 comments

eri820503 commented

Hi,

As title, when I load the first image and run an auto segmentation model, the segmentation is as expected.

Then, I close my first image by Close Scene (Not close 3D slicer) and reload the second image to run the same auto segmentation model.
But the segmentation result is as same as the first one, it seems that 3D slicer post the first image's temporary file to the remote server not the second image.

Is this an issue or my operation mistake?

YuanTingHsieh commented

Hi,

I just try 3D slicer on both ubuntu and windows machine.
My server is using Clara-Train 3.0 container and it works as expected.
My 3D slicer is not using the old image.
What is your system?
What are the steps you are using?

What I did is
first load an image (let's say spleen_2)
then click NVIDIA segmentation button
then I click left top Close Scene
then I click load another image (spleen_6)
then above the segments window I select "Create new segmentation"
and right under that I choose master volume to be "spleen_6"
then I click NVIDIA segmentation button
Both results match their own image

eri820503 commented

Hi YuanTing,

My OS is macOS Catalina 10.15.4, server is using Clara-Train 2.0 container and my image is dicom format.

I use the same steps as yours:

  1. First load a dicom (dicom_1) and click Add new empty segment bottom
  2. Click NVIDIA segmentation button and the result is as expected
  3. Then click Close Scene button and reload another dicom (dicom_2)
  4. Click Create new Segmentation bottom, select Master volume as dicom_2 and click Add new empty segment
  5. Click NVIDIA segmentation button but the result is as same as dicom_1's result
YuanTingHsieh commented

Can you try with 3.0 container?
It is available at: https://ngc.nvidia.com/catalog/containers/nvidia:clara-train-sdk
Let me know if you still encounter this problem

eri820503 commented

Yes, I have the same issue in 3.0 container.
The result is correct from py-client but not in slicer-client.
So I think the issue is in slicer-client, not in clara container.

YuanTingHsieh commented

I am trying to reproduce the problem.
Are you using one of our released pre-trained models?
If so, can you provide the model name you are using in that case?
Can you also provide some public available DICOM images that you try (that have this problem)?

Thank you

eri820503 commented

I found that the issue is due to my plugin is older version.
After I used latest plugin with 3D slicer 4.11, the result is correct.
Thanks a lot.

Note: here is my log in 3D slicer 4.10 with older plugin version:

This plugin dir: /<dir>/ai-assisted-annotation-client-master/slicer-plugin/NvidiaAIAA
Using AIAA: http://<ip>:5000
DOC ID inside client: None
Active Label: Segment_1
Fetching List of Models for label: None
CT_Seg_Liver_Tumor = segmentation
CT_Seg_Spleen = segmentation
CT_Annotation_Spleen = annotation
test = segmentation
Segmentation = segmentation
-----------------------------------------------------	
Total Models Loaded: 	5	
-----------------------------------------------------	
Segmentation Models: 	4	
Annotation Models: 	1	
-----------------------------------------------------	

++ Time consumed by onClickModels: 0.230782032013
Run Segmentation for model: CT_Seg_Spleen for label: Segment_1
Using AIAA: http://<ip>:5000
DOC ID inside client: None
Preparing for Segmentation Action
Node Id: vtkMRMLScalarVolumeNode1 => None
Saved Input Node into: /var/folders/yl/61f6t6mj17q74fpllmkz4mmw0000gn/T/slicer-aiaa/tmpTofxA2.nii.gz
Saving Doc-ID: ae11f036-a0a5-11ea-864c-0242ac1c0003
Extreme Points: [[83.0, 231.0, 73.0], [224.0, 218.0, 81.0], [166.0, 141.0, 78.0], [97.0, 263.0, 77.0], [109.0, 190.0, 67.0], [132.0, 229.0, 87.0]]
Update Segmentation Mask from: /var/folders/yl/61f6t6mj17q74fpllmkz4mmw0000gn/T/tmpF_3ezI.nii.gz
Removing temp segmentation with id: Segment_1 with color: (0.5019607843137255, 0.6823529411764706, 0.5019607843137255)
Setting new segmentation with id: Segment_1 => Segment_1
Extreme Points: [[83.0, 231.0, 73.0], [224.0, 218.0, 81.0], [166.0, 141.0, 78.0], [97.0, 263.0, 77.0], [109.0, 190.0, 67.0], [132.0, 229.0, 87.0]]
Total Added Segments for Segment_1: 1
++ Time consumed by updateSegmentationMask: 0.328016996384
++ Time consumed by onClickSegmentation: 86.6156830788
Run segmentation for (CT_Seg_Spleen): SUCCESS	
Time Consumed: 86.6163420677 (sec)
Segment_1 => Segment_1 Extreme points are: [[83.0, 231.0, 73.0], [224.0, 218.0, 81.0], [166.0, 141.0, 78.0], [97.0, 263.0, 77.0], [109.0, 190.0, 67.0], [132.0, 229.0, 87.0]]
Add Fiducial: [83.0, 231.0, 73.0, 1.0] => (-340.2266136407852, -222.57815849781036, 364.99999999999994, 1.0)
Add Fiducial: [224.0, 218.0, 81.0, 1.0] => (-228.1426124572754, -232.91214442253113, 404.99999999999994, 1.0)
Add Fiducial: [166.0, 141.0, 78.0, 1.0] => (-274.2480881214142, -294.1211379766464, 389.99999999999994, 1.0)
Add Fiducial: [97.0, 263.0, 77.0, 1.0] => (-329.09770572185516, -197.1406546831131, 384.99999999999994, 1.0)
Add Fiducial: [109.0, 190.0, 67.0, 1.0] => (-319.5586417913437, -255.16996026039124, 334.99999999999994, 1.0)
Add Fiducial: [132.0, 229.0, 87.0, 1.0] => (-301.27543592453003, -224.16800248622894, 434.99999999999994, 1.0)
Run Segmentation for model: CT_Seg_Spleen for label: Segment_1
Using AIAA: http://<ip>:5000
DOC ID inside client: ae11f036-a0a5-11ea-864c-0242ac1c0003
Preparing for Segmentation Action
Node Id: vtkMRMLScalarVolumeNode1 => /var/folders/yl/61f6t6mj17q74fpllmkz4mmw0000gn/T/slicer-aiaa/tmpTofxA2.nii.gz
Using Saved Node from: /var/folders/yl/61f6t6mj17q74fpllmkz4mmw0000gn/T/slicer-aiaa/tmpTofxA2.nii.gz
Extreme Points: [[83.0, 233.0, 74.0], [224.0, 218.0, 81.0], [156.0, 141.0, 75.0], [97.0, 263.0, 77.0], [108.0, 200.0, 67.0], [176.0, 215.0, 87.0]]
Update Segmentation Mask from: /var/folders/yl/61f6t6mj17q74fpllmkz4mmw0000gn/T/tmp1aaERs.nii.gz
Removing temp segmentation with id: Segment_1 with color: (0.5019607843137255, 0.6823529411764706, 0.5019607843137255)
Setting new segmentation with id: Segment_1 => Segment_1
Extreme Points: [[83.0, 233.0, 74.0], [224.0, 218.0, 81.0], [156.0, 141.0, 75.0], [97.0, 263.0, 77.0], [108.0, 200.0, 67.0], [176.0, 215.0, 87.0]]
Total Added Segments for Segment_1: 1
++ Time consumed by updateSegmentationMask: 0.293701171875
++ Time consumed by onClickSegmentation: 38.7946181297
Run segmentation for (CT_Seg_Spleen): SUCCESS	
Time Consumed: 38.7950739861 (sec)
Segment_1 => Segment_1 Extreme points are: [[83.0, 233.0, 74.0], [224.0, 218.0, 81.0], [156.0, 141.0, 75.0], [97.0, 263.0, 77.0], [108.0, 200.0, 67.0], [176.0, 215.0, 87.0]]
Add Fiducial: [83.0, 233.0, 74.0, 1.0] => (-318.49234092235565, -206.87119352817535, 369.99999999999994, 1.0)
Add Fiducial: [224.0, 218.0, 81.0, 1.0] => (-213.56846237182617, -218.03330826759338, 404.99999999999994, 1.0)
Add Fiducial: [156.0, 141.0, 75.0, 1.0] => (-264.17004919052124, -275.33216392993927, 374.99999999999994, 1.0)
Add Fiducial: [97.0, 263.0, 77.0, 1.0] => (-308.0743671655655, -184.5469640493393, 384.99999999999994, 1.0)
Add Fiducial: [108.0, 200.0, 67.0, 1.0] => (-299.88881635665894, -231.42784595489502, 334.99999999999994, 1.0)
Add Fiducial: [176.0, 215.0, 87.0, 1.0] => (-249.28722953796387, -220.265731215477, 434.99999999999994, 1.0)
  1. For the first dicom:
    I think the plugin transforms .dcm to nii.gz and saves it in /var/folders/yl/61f6t6mj17q74fpllmkz4mmw0000gn/T/slicer-aiaa/tmpTofxA2.nii.gz
    After that plugin posts tmpTofxA2.nii.gz to AIAA server and updates mask image and extreme points.
  2. Then I Close Scene
  3. For the second dicom:
    Plugin used the same DOC ID inside client as before and did not transform .dcm instead of using the temporary file tmpTofxA2.nii.gz and posts it to the AIAA server.

I check the files in /var/folders/yl/61f6t6mj17q74fpllmkz4mmw0000gn/T/slicer-aiaa/.
The temporary file is removed only when 3D slicer is closed.

Here is my plugin function segmentation in SegmentEditorEffect.py 

    def segmentation(self, model, inputVolume):
        self.reportProgress(0)
        logging.info('Preparing for Segmentation Action')

        node_id = inputVolume.GetID()
        in_file = volumeToImageFiles.get(node_id)
        logging.info('Node Id: {} => {}'.format(node_id, in_file))

        if in_file is None:
            in_file = tempfile.NamedTemporaryFile(suffix='.nii.gz', dir=aiaa_tmpdir).name

            self.reportProgress(5)
            slicer.util.saveNode(inputVolume, in_file)

            volumeToImageFiles[node_id] = in_file
            logging.info('Saved Input Node into: {}'.format(in_file))
        else:
            logging.info('Using Saved Node from: {}'.format(in_file))

        self.reportProgress(30)

        result_file = tempfile.NamedTemporaryFile(suffix='.nii.gz').name
        params = self.client.segmentation(model, in_file, result_file, save_doc=True)

        extreme_points = params.get('points', params.get('extreme_points'))
        logging.info('Extreme Points: {}'.format(extreme_points))

        self.reportProgress(100)
        return extreme_points, result_file

The latest function seems to always use the newest dicom :

ai-assisted-annotation-client/slicer-plugin/NvidiaAIAA/SegmentEditorNvidiaAIAALib/SegmentEditorEffect.py

Line 1032 in 3198224

def segmentation(self, image_in, session_id, model):

  • 👍1
YuanTingHsieh commented

Got it.
Thank you for the feedback and finding the root cause of older version.