DynamicSyntesisNetwork

This is the implementation of the dynamic synthesis network (DSN) for large-scale 3D descattering, as presented in our publication:

W. Tahir, H. Wang, L. Tian, "Adaptive 3D descattering with a dynamic synthesis network", Light: Science & Applications volume 11, Article number: 42 (2022).

Figure: DSN Schematic

Training new model

The script Train_new_model/main_train.py contains the code for training the network.
In order to train the model, the data can be downloaded from the following google drive link: https://drive.google.com/drive/folders/1q5hliaW8NVGyA74BP49uqaWpJdVPOmiA
Download the 'data.zip' from the above link and decompress it to the 'Train_new_model' folder, such that it's path is .../Train_new_model/data/
Download the 'DSN_init_chkpt' folder from the above link and put the contents to the 'Train_new_model' folder, such that the paths are .../Train_new_model/trained_chkpts_enc/ and .../Train_new_model/trained_chkpts_dec/. The contained folders 'trained_chkpts_enc' and 'trained_chkpts_dec' are used for the initialization of the encoder and decoder of DSN. They are the trained encoder and decoder from the expert network.
To begin training, execute the script main_train.py as follows: python main_train.py

We provide a pretrained model and a script which can use the pretrained model to descatter a test 3D backpropagation volume.
In order to perform descattering on a test-3D-backpropagation (not used in the training process), download the folder 'test_data' and 'chkpt' from the google drive link provided above.
Copy the downloaded 'test_data' and 'chkpt' folders to the 'Test_trained_model' folder, such that their path is '.../Test_trained_model/test_data' and '.../Test_trained_model/chkpt'.
In the folder Test_trained_model, execute the script main_test.py as follows:

$ python main_test.py

The model will perform descattering on all test data and save results in the folder 'test_results'

The multiple-scattering simulator can be found in the following repository:

environment.yml lists dependencies used to run this code on a machine with 128GB RAM and an Nvidia RTX-8000 GPU.