/foreign-object-pipeline

Primary LanguageJupyter Notebook

Experimental plan and literature review

Research question

How many representative objects need to be scanned with (industrial setting) x-ray scanning in order to allow good deep learning based segmentation of foreign objects embedded in products.

Experimental plan

In short: the experimental plan is to follow the pipeline shown in the figure below for differing amounts of objects, in order to create differing amounts of training data. This training data will then be used to train a deep network for image segmentation. Performance of this network will be evaluated on a test set.

The complete workflow of data acquisition (a, b) and the generation of training data (c, h) for deep learning driven foreign object detection, through 3D reconstruction from the CT scan (d, e), segmentation (f), and virtual projections (g). The reconstruction reveals the hidden foreign objects inside the main object. Note that the projection data (d) is usually just the set of fast-acquisition radiographs (d).

In more detail:

  1. Create a specific amount of phantoms (in the range 1 to 100)
  2. Do virtual fast-acquisition scanning by using the Astra toolbox
    • The used scanning settings will be directly taken from the paper
    • They are meant to represent fast-acquisition x-ray scanning in an industrial setting
  3. Use the generated projection data to create ground truth 3D locations of the foreign objects
    • Done by reconstructing the projection data
    • Then segmenting the foreign object
    • And doing fast-acquisition scanning using the Astra toolbox of just the foreign object
  4. Use the created training set (consisting of the full object radiographs and the true foreign object location radiographs) to train a deep learning segmentation network
    • This will probably be an altered version of U-Net as described in the paper
  5. Test the network accuracy for a predefined test set

The plan is to try as many amounts of training phantoms (the phantoms that will be used to generate training data) as possible within the timeframe for this assignment. This will be heavily dependent on the computational cost of the experiment described above.

Literature review

Note: the headings are links to the mentioned papers

Implementation paper

The implementation paper upon which the final paper will mostly be based. It describes important settings for phantom generation, radiograph generation, foreign object segmentation and deep learning based segmentation.

ASTRA toolbox

Paper introducing the ASTRA toolbox. We will use the ASTRA toolbox's rotating cone-beam geometry to create radiographs from phantoms. We will use the ASTRA toolbox's SIRT implementation with 100 iterations for the reconstruction.

Foreign object segmentation

In order to get ground truth for the foreign object locations, we need to segment the foreign objects from the reconstructed objects. This paper describes automated segmentation for computed tomography. The paper mainly focusses on medical imaging implementations where segmentation is mostly based on tissue, but since our phantoms simulate foreign objects by using a different density (ex: flesh object with a bone object embedded in it) we can use the approaches described in this paper.

U-Net

This paper describes U-Net: a deep learning based segmentation method that aims to be as accurate as possible while using as little training data as possible. We will be using a modified implementation of U-Net as our deep learning based segmentation method to test segmentation performance based on the amount of training data.