mask and dicom both？

Question

mask and dicom both？

MOMOANNIE opened this issue 4 years ago · 37 comments

Hello, developer, this project is very meaningful. I am also studying the segmentation of bronchopulmonary. I want to know if your project can be used for segmentation of bronchopulmonary?
If it can be used for bronchopulmonary segmentation, is it necessary to have both the mask and the original dicom data?

Answer 1 · 2021-04-20T08:50:58.000Z

Hello Anne! Yes, it segments the bronchopulmonary segments, but you need the mask for this. The original DICOM data set is not necessary. The mask is essentially a 3D numpy array where a 1 is written where there is bronchus, and a 0 elsewhere. More details in the README. Hope this helps!

Answer 2 · 2021-04-22T01:50:23.000Z

Hello Anne! Yes, it segments the bronchopulmonary segments, but you need the mask for this. The original DICOM data set is not necessary. The mask is essentially a 3D numpy array where a 1 is written where there is bronchus, and a 0 elsewhere. More details in the README. Hope this helps!

Brutenis, thank you very much for answering my questions in such a timely manner. I have a doubt. If I need to segment a patient's bronchopulmonary segment, do I need to correspond to the patient's template image? Still like deep learning, after training this model, I only need to input the original image of the patient to get the segmentation of bronchopulmonary.

Answer 3 · 2021-04-22T06:32:36.000Z

In the process of reproduction, I want to use stage-01 as the input. When I use the file airway/image_processing/save_images_as_npz.py to convert the original dicom data into a numpy array, the following problem appears. How can I solve it?

ERROR: Insufficient count of input/output paths supplied!
Expected 1 output paths and 1 input paths!
Got sys.argv: []

Answer 4 · 2021-04-22T09:45:06.000Z

You can use the script airway-pipeline.py for all the stages, like this: python3 airway-pipeline.py model. This will create stage 1 for you. But you first have to create the defaults, for this copy example_defaults.yaml to defaults.yaml in the root folder of the project and change the line local: /path/to/local_data/ to your data path. Although I highly recommend looking at the diagram in the README which shows the data structure.

The script you used expects two parameters, the input path and the output path, so you could call it with both as well.

You could also create the model with a simple script, but this of course depends on your bronchus mask format. See below for an example how this could be done. In this case all IMG* files are layers in the bronchus mask in the same folder as the script.

from pathlib import Path

import numpy as np
import pydicom

image_files = Path().cwd().glob('IMG*')  # Our data was called IMG1 to (around) IMG700
sort_ascending = sorted(image_files, key=lambda f: int(f.name[3:]))

layers = []
for image_name in sort_ascending:
    # Our DICOM image contained -10000 for empty voxels, and some value around 100 for bronchus
    im = pydicom.dcmread(image_name).pixel_array  # Read the DICOM image
    im = np.add(im, 10000)  # Shift all values to between 0 and around 11000
    im = np.clip(im, 0, 1)  # Clip all values to between 0 and 1
    layers.append(im)  # Append current layer to
model = np.stack(layers)  # Stack all layers into a single array

print(model.shape)  # Should be around 700 x 512 x 512
np.savez_compressed("model", model)  # Save it as a compressed numpy array

I agree that currently this is a bit convoluted, I'll try to simplify the process and add a tutorial for this within the next two weeks or so.

Answer 5 · 2021-04-23T03:17:54.000Z

You can use the script airway-pipeline.py for all the stages, like this: python3 airway-pipeline.py model. This will create stage 1 for you. But you first have to create the defaults, for this copy example_defaults.yaml to defaults.yaml in the root folder of the project and change the line local: /path/to/local_data/ to your data path. Although I highly recommend looking at the diagram in the README which shows the data structure.

The script you used expects two parameters, the input path and the output path, so you could call it with both as well.

You could also create the model with a simple script, but this of course depends on your bronchus mask format. See below for an example how this could be done. In this case all IMG* files are layers in the bronchus mask in the same folder as the script.
from pathlib import Path

import numpy as np
import pydicom

image_files = Path().cwd().glob('IMG*')  # Our data was called IMG1 to (around) IMG700
sort_ascending = sorted(image_files, key=lambda f: int(f.name[3:]))

layers = []
for image_name in sort_ascending:
    # Our DICOM image contained -10000 for empty voxels, and some value around 100 for bronchus
    im = pydicom.dcmread(image_name).pixel_array  # Read the DICOM image
    im = np.add(im, 10000)  # Shift all values to between 0 and around 11000
    im = np.clip(im, 0, 1)  # Clip all values to between 0 and 1
    layers.append(im)  # Append current layer to
model = np.stack(layers)  # Stack all layers into a single array

print(model.shape)  # Should be around 700 x 512 x 512
np.savez_compressed("model", model)  # Save it as a compressed numpy array
I agree that currently this is a bit convoluted, I'll try to simplify the process and add a tutorial for this within the next two weeks or so.

Thank you very much for your patient answers, my stage-01 has run through, and I am currently debugging stage-02.

I read the readme of the project again, and also read other people’s questions. I need to confirm two points with you:

First: If my mask does not segment the lung lobes, bronchus, arteries, and veins like the mask in your project, but only extracts the bronchus (that is, the bronchus is 1 and other places are 0), Can I use my mask to divide the bronchus into 18 segments?

Second: local: /path/to/local_data/ is the path where my mask is stored, not my original dicom path, isn’t it? (You said that the original dicom data is not necessary, the mask is necessary)

Answer 6 · 2021-04-23T08:23:36.000Z

Hello, developer, I put the bronchial mask (bronchial area is 1, and the rest is 0) into the stage-01 folder. When stage-02 is run, it passes successfully, but when stage-04 is run, an error message appears. Is the error reported because my slice thickness is not uniform?

You mentioned in the readme that the slicer thickness in all directions must be the same. I am confused about the slice thickness in all directions. Is there any difference between the slice thickness and the spacing? I don’t know how to unify the slice thickness in all directions

Answer 7 · 2021-04-23T09:23:11.000Z

First: If my mask does not segment the lung lobes, bronchus, arteries, and veins like the mask in your project, but only extracts the bronchus (that is, the bronchus is 1 and other places are 0), Can I use my mask to divide the bronchus into 18 segments?

Yes, I just tested it with only the bronchus mask and it worked well (encountered a bug in stage 60, but this shouldn't be a problem for now. I'll fix it soon though).

Second: local: /path/to/local_data/ is the path where my mask is stored, not my original dicom path, isn’t it? (You said that the original dicom data is not necessary, the mask is necessary)

Yes, this is the directory where your compressed mask is stored, and looks like this:

    ├── stage-01                    🠔 Each stage now has the same basic format
    │   ├── 3123156
    │   │   └── model.npz
    │   ├── 3123193
    │   │   └── model.npz
    │   └── ...
    ├── stage-02                    🠔 Each stage from here on will be created by the pipeline itself
    │   ├── 3123156                    so you do not need to handle this, each of them have different
    │   │   └── reduced_model.npz
    │   └── ...

And no DICOM data is needed at this point, so it should work.

Hello, developer, I put the bronchial mask (bronchial area is 1, and the rest is 0) into the stage-01 folder. When stage-02 is run, it passes successfully, but when stage-04 is run, an error message appears. Is the error reported because my slice thickness is not uniform?

That's unfortunate, could you run it with the -v flag and post the both STDERR and STDOUT output here? Like this python airway-pipeline.py 4 -v. The slice thickness should have no impact for now, as once you convert it to a numpy array it doesn't know that it's not uniform anymore. It might only impact the final results in the classification (stage-11), but the previous stages should still work.

Hope it works for you! Excited to see it work on someone else's data!

Answer 8 · 2021-04-23T09:56:47.000Z

That's unfortunate, could you run it with the -v flag and post the both STDERR and STDOUT output here? Like this python airway-pipeline.py 4 -v. The slice thickness should have no impact for now, as once you convert it to a numpy array it doesn't know that it's not uniform anymore. It might only impact the final results in the classification (stage-11), but the previous stages should still work.

I just re-run the program after unifying the slice thicknesses, and I still get an error at stage-04. I tried to use python airway-pipeline.py 4 -v, but it didn’t print STDERR and STDOUT. I don’t know which stage cause the error and how to debug

Answer 9 · 2021-04-23T10:06:31.000Z

I printed out my compressed mask array, and the result showed that the value of most positions is "false", a few positions are "true", not "0" and "1". Is it the data type in the array that caused stage-04 to report an error?

Answer 10 · 2021-04-23T11:15:41.000Z

I printed out my compressed mask array, and the result showed that the value of most positions is "false", a few positions are "true", not "0" and "1". Is it the data type in the array that caused stage-04 to report an error?

True and False should work, I just tested it. I think numpy doesn't really differentiate between True/False and 1/0.

Are you sure there is a mistake if it doesn't print an error? Because otherwise some things will definitely be printed.

If your data isn't private, I could take a look at it if you can send me the model.npz (bronchus mask only would be enough).

Also, I've pushed some updates for the later stages.

Answer 11 · 2021-04-23T11:32:51.000Z

I printed out my compressed mask array, and the result showed that the value of most positions is "false", a few positions are "true", not "0" and "1". Is it the data type in the array that caused stage-04 to report an error?

True and False should work, I just tested it. I think numpy doesn't really differentiate between True/False and 1/0.

Are you sure there is a mistake if it doesn't print an error? Because otherwise some things will definitely be printed.

If your data isn't private, I could take a look at it if you can send me the model.npz (bronchus mask only would be enough).

Also, I've pushed some updates for the later stages.

Can you see the pictures I uploaded? The picture shows that an error occurred when processing stage-04.
How do I send the data to you? It seems that the data format of npz cannot be uploaded on github

Answer 12 · 2021-04-23T11:37:26.000Z

I printed out my compressed mask array, and the result showed that the value of most positions is "false", a few positions are "true", not "0" and "1". Is it the data type in the array that caused stage-04 to report an error?

True and False should work, I just tested it. I think numpy doesn't really differentiate between True/False and 1/0.

Are you sure there is a mistake if it doesn't print an error? Because otherwise some things will definitely be printed.

If your data isn't private, I could take a look at it if you can send me the model.npz (bronchus mask only would be enough).

Also, I've pushed some updates for the later stages.
model.zip

I put the model.npz file into a folder, compressed it into a zip format and uploaded it, you can see if it can be decompressed normally, and use

Answer 13 · 2021-04-23T11:39:37.000Z

I put the model.npz file into a folder, compressed it into a zip format and uploaded it, you can see if it can be decompressed normally, and use

The zip seems to be empty, it only contains a folder. You could send me an E-Mail to brutenis [at] gmail.com with the model as well.

Answer 14 · 2021-04-23T11:58:23.000Z

I put the model.npz file into a folder, compressed it into a zip format and uploaded it, you can see if it can be decompressed normally, and use

The zip seems to be empty, it only contains a folder. You could send me an E-Mail to brutenis [at] gmail.com with the model as well.

Ok, I have just sent the model.npz data to you via gdutchu@outlook.com, please check it, thank you very much

Answer 15 · 2021-04-23T12:34:30.000Z

It works for me without errors (even stage-04). I've included a rendered version of your model:

Although the 0th axis was flipped (i.e. it was upside-down), I flipped it like this:

import numpy as np

model = np.load("original_model.npz")['arr_0']
model = model.astype(np.uint8)
model = np.flip(model, axis=0)
np.savez_compressed("model.npz", model)

No segments were detected because the bronchus mask currently doesn't have enough detail, it does contain a few splits though, so that works.

For the error, maybe you have more folders than necessary? Make sure that each stage folder contains patient folders which contain the models, if there are other folders delete them. And can you try again with the -v, for me it does show what the error message is exactly.

Answer 16 · 2021-04-23T12:49:16.000Z

You could also try setting force: True in the defaults.yaml.

Answer 17 · 2021-04-25T02:17:06.000Z

For the error, maybe you have more folders than necessary? Make sure that each stage folder contains patient folders which contain the models, if there are other folders delete them. And can you try again with the -v, for me it does show what the error message is exactly.

I deleted all the previous generation folders, and then re-run stages 1-4 through python airway-pipeline.py ×× -f -v, and modified the defaults.yaml file at the same time, set force: True，then found that it can output stdout and stderr. But when I run to stage 4, the error it prints is:

Is this error because I didn't flip the data?

My folder structure is like this:

Is this all right?

Answer 18 · 2021-04-25T21:01:29.000Z

Folder structure looks exactly how it should. But I can't seem to replicate the error with your model on my part, it works even without flipping and without converting it to to np.uint8. Nonetheless, have you tried flipping the model and converting it?

I've pushed a few updates, can you try stage-04 again after pulling?

I've also sent you stage-04 via email, perhaps you can try whether the latter stages work if the other stuff does not.

Answer 19 · 2021-04-27T02:18:36.000Z

Folder structure looks exactly how it should. But I can't seem to replicate the error with your model on my part, it works even without flipping and without converting it to to np.uint8. Nonetheless, have you tried flipping the model and converting it?

I've pushed a few updates, can you try stage-04 again after pulling?

I've also sent you stage-04 via email, perhaps you can try whether the latter stages work if the other stuff does not.

I pulled down your updated project and ran it again. Stage-04 can run through without error, hahaha, thank you very much, I will try the following steps

Answer 20 · 2021-04-27T03:22:15.000Z

I run the next stages, when I run to stage-11, the error is reported as follows:

I tried the following three methods to solve:

Use the data that is not upside down, re-run the updated project, and report an error when it runs to stage-11;
Use the flipped data to run the updated project, and report an error when it runs to stage-11;
Use the package of stage-04 that you sent to me by your mailbox, I copy it to the folder of stage-04, and then run the subsequent stages, still reporting an error in stage-11;

The error of these three methods is the same, as shown above:

Answer 21 · 2021-04-27T09:44:34.000Z

Yes, the problem is that the data does not have enough detail. I had previously attached an image of your 3D model, it lacks the actual segments which should be classified. Compare it with the one shown in the README. So stage-11 fails because there are no segments. Although it should not crash, that is correct.

To create the 3d model you can run this command: python airway-pipeline.py color_mask 3d and then to visualize python airway-vis.py -o. But you have to have blender installed.

Answer 22 · 2021-04-27T09:54:43.000Z

Yes, the problem is that the data does not have enough detail. I had previously attached an image of your 3D model, it lacks the actual segments which should be classified. Compare it with the one shown in the README. So stage-11 fails because there are no segments. Although it should not crash, that is correct.

To create the 3d model you can run this command: python airway-pipeline.py color_mask 3d and then to visualize python airway-vis.py -o. But you have to have blender installed.

Okay, Do you have a complete bronchial mask image? If the data is not private, can you send me a copy? I want to see the segmentation effect

In addition, I tried to use the python airway-pipeline.py color_mask 3d -v to visualize, but I got an error, the error is as follows:

Answer 23 · 2021-04-28T09:26:57.000Z

I've emailed the doctor who gave us the data whether I'm allowed to send to it to anyone, but he told me I should not, at least for now. Maybe there are some public bronchus masks available meanwhile?

Sorry about all the errors you have been experiencing, but this one should be harmless. If you have blender installed you can still try to visualize it by running python airway-vis.py -o

Answer 24 · 2021-04-28T09:58:38.000Z

I've emailed the doctor who gave us the data whether I'm allowed to send to it to anyone, but he told me I should not, at least for now. Maybe there are some public bronchus masks available meanwhile?

Sorry about all the errors you have been experiencing, but this one should be harmless. If you have blender installed you can still try to visualize it by running python airway-vis.py -o

It’s okay, I will find the bronchial mask data, and then run this project to see the segmentation effect

I have installed blender, but when I use python airway-vis.py -o to visualize , the error is reported as follows:

Answer 25 · 2021-04-28T10:55:41.000Z

Ah, maybe blender is not accessible in the current environment? You could try changing blender: blender to blender: /usr/bin/blender in the defaults.yaml.

Answer 26 · 2021-04-29T03:54:00.000Z

Ah, maybe blender is not accessible in the current environment? You could try changing blender: blender to blender: /usr/bin/blender in the defaults.yaml.

I ran this project in a conda virtual environment, and installed version 1.4 of blender in this environment, as follows:

I tried to change blender: blender to blender: /usr/bin/blender in defaults.yaml, and visualized by running python airway-vis.py -o, the result was still an error:

Then I changed blender: blender to blender: /snap/bin/blender in defaults.yaml, and still got an error when visualizing:

Answer 27 · 2021-04-29T08:38:20.000Z

I got a complete bronchial mask image, but when I use this data to run this project, It reports an error in stage-02, and the error as follows:

I want to use blender to visualize to see if there is a problem with the data that caused the program to report an error, but my blender cannot be visualized yet. Can you help me visualize this bronchial mask image?

Answer 28 · 2021-04-29T08:55:42.000Z

Blender is a 3D visualisation program which has to be installed via the package manager: sudo apt install blender. Sorry for the confusion, I did not know there was a blender pip package.

I'll look into the error in stage 02.

Edit: I've fixed it I think. You can git pull and try again.

Answer 29 · 2021-04-29T09:38:10.000Z

Blender is a 3D visualisation program which has to be installed via the package manager: sudo apt install blender. Sorry for the confusion, I did not know there was a blender pip package.

Ok, I have pulled down the latest project, and now I am running with an incomplete bronchial mask image to test the visualization, at the same time, I used sudo apt install blender to install blender, but the following error occurred during visualization:

Edit:Sorry, my version is useless. I reported this error because I used python2, and It don't report an error with python3. I am currently installing the dependency package for visualization.

I'll look into the error in stage 02.

The complete bronchial mask data is only 0 and 1. When using this data to run the project, an error will be reported in stage-02. I am wondering whether the error is caused by a data problem, so I want to use blender to visualize the data, but after I installed blender, The above error occurs when visualizing.

Answer 30 · 2021-04-29T10:06:50.000Z

I use python3 airway-vis.py -o on the remote server for visualization. The program does not report an error, but there is no visual interface. What do I need to do to display the visual interface of the remote server on the local computer?

Answer 31 · 2021-04-29T11:45:02.000Z

The optimal way would be to mount the folder structure in your local computer and to add the path in the local defaults.yaml.

For example, mounting it via sshfs: sshfs anne@remote_server:/home/anne/data/test/ /home/anne/data/test/. If you already have it mounted locally, then you can skip this.

Then install airway locally as well, including blender and its dependencies. Now change the path in the local defaults.yaml to the mountpoint.

Now you should be able to run the same python3 airway-vis.py -o command locally.

Answer 32 · 2021-05-06T06:44:23.000Z

The optimal way would be to mount the folder structure in your local computer and to add the path in the local defaults.yaml.

For example, mounting it via sshfs: sshfs anne@remote_server:/home/anne/data/test/ /home/anne/data/test/. If you already have it mounted locally, then you can skip this.

Then install airway locally as well, including blender and its dependencies. Now change the path in the local defaults.yaml to the mountpoint.

Now you should be able to run the same python3 airway-vis.py -o command locally.

I'm very sorry, because something at home has delayed the process of reproducing the project. At present, I can visualize it on my local computer.

When I try to use the complete bronchial mask image to reproduce, an error will be reported in stage-02, and the error as follows:

Answer 33 · 2021-05-06T08:24:13.000Z

I tried two complete bronchial mask images to reproduce this project. When the first data runs to stage- 11, the error is reported as follows:

Then I use the python airway-pipeline.py color_mask 3d, I still get an error, unable to generate stage34, 60 and 61 files, so that when I used python3 airway-vis.py -o to visualize, it is empty

When I used the second data, it reported an error when it reached stage- 02. The error was as follows:

Can I send you these two pieces of complete bronchial mask data? Please help me to see if it can run normally on your side

Answer 34 · 2021-05-06T15:57:33.000Z

Yes, I get those errors too, but this is because both models are either empty or nearly empty.

In [18]: def show_uniques(name):
    ...:     import numpy as np
    ...:     model = np.load(name)['arr_0']
    ...:     print(*zip(*np.unique(model, return_counts=True)))

In [19]: show_uniques("model1.npz")
(False, 40640282) (True, 230)

In [20]: show_uniques("model2.npz")
(False, 34422500)

model1.npz only contains 230 True==1 (Bronchus) voxels. It looks like this (the bronchus structure can clearly be seen, but its not connected):

And model2.npz is completely empty so it crashes on stage-02. I will try to make the error messages clearer, thanks for noticing!

Can you try creating a connected voxel model? Meanwhile I'll see if I can generate some fake data for testing purposes.

Answer 35 · 2021-05-07T09:42:21.000Z

Yes, I get those errors too, but this is because both models are either empty or nearly empty.
In [18]: def show_uniques(name):
    ...:     import numpy as np
    ...:     model = np.load(name)['arr_0']
    ...:     print(*zip(*np.unique(model, return_counts=True)))

In [19]: show_uniques("model1.npz")
(False, 40640282) (True, 230)

In [20]: show_uniques("model2.npz")
(False, 34422500)
model1.npz only contains 230 True==1 (Bronchus) voxels. It looks like this (the bronchus structure can clearly be seen, but its not connected):

And model2.npz is completely empty so it crashes on stage-02. I will try to make the error messages clearer, thanks for noticing!

Can you try creating a connected voxel model? Meanwhile I'll see if I can generate some fake data for testing purposes.

I use software to display the two raw data. From the software, they seem to be connected together, and the display is as follows:

I guess it might be that I had a problem when preprocessing the original data, so I modified my preprocessing process to regenerate model.npz data. I sent the regenerated .npz data to airway for calculation. The first data has a problem on stage-10. The problems are as follows:

There is a problem with the second data at stage-11. The problems are as follows:

I currently don’t have better bronchial mask test data, so I can only test on these few data.If the error is caused by incorrect preprocessing, I will modify the preprocessing process. If the original data is not good, I can only continue to find new test data

The statistics of true and flase pixels of the re-preprocessed data are as follows:

Answer 36 · 2021-05-07T13:00:31.000Z

Model1 looks great! You just need to flip it, that's why it crashes. (Flipping with the same script as I had mentioned above) Here is what it looks like:

Model2 doesn't look as good (I also flipped it):

After flipping stage-11 works fine as well. But you don't need it for visualization. To visualize only these are required: ./airway-pipeline.py 2-10 color_mask 3d. Even if some show errors visualization will likely still work, so just try it with vis -o.

I also now added a fake bronchus model based on the data I have. It looks a bit funny but all stages should work with it. You can either use it manually in example_data/model.npz or run pytest in the root folder of airway. Pytest will create a folder in /tmp/airway-tests-dummy-(randomletters) and run most stages with it (it will take a while). After that you can cd into that folder and run vis -o -P $PWD. Hope this works for you so you can test it with this data.

Answer 37 · 2021-05-08T09:39:02.000Z

Model1 looks great! You just need to flip it, that's why it crashes. (Flipping with the same script as I had mentioned above) Here is what it looks like:

Model2 doesn't look as good (I also flipped it):

After flipping stage-11 works fine as well. But you don't need it for visualization. To visualize only these are required: ./airway-pipeline.py 2-10 color_mask 3d. Even if some show errors visualization will likely still work, so just try it with vis -o.

I also now added a fake bronchus model based on the data I have. It looks a bit funny but all stages should work with it. You can either use it manually in example_data/model.npz or run pytest in the root folder of airway. Pytest will create a folder in /tmp/airway-tests-dummy-(randomletters) and run most stages with it (it will take a while). After that you can cd into that folder and run vis -o -P $PWD. Hope this works for you so you can test it with this data.

In fact, the two model data（model1.npz, moedl2.npz） I gave have been flipped, and the original bronchial mask data is shown as follows:

In order to pass the test, I canceled the flip of model1.npz, it can run successfully, and the following results appear:

Similarly, I also canceled the flip of model2.npz, but it still reported an error at stage-10, even if it was flipped, it was reported in the code on my side.

At the same time, I used the example data to test the project, and the test results are very good, as follows:

Is your bronchus mask data manually marked by the doctor? In addition, did you write a paper to introduce this project? I want to learn more about this method through your paper. If you have a related paper, can you send it to me to read it? thank you very much