This is the official implmentation of ICCV 2021 paper: Embedding-Novel-Views-in-a-Single-JPEG-Image
please refer to the enviroment.yml
We provide the test set we use in the Google drive. We randomly select 1500 sequences from RealEstate10K to generate MPIs for testing. The IDs of the sequences for training and testing are in train_data_ids.txt and test_data_ids.txt respectively.
And the stereo_mpi represents the MPIs generated using Stereo Magnification. And the pb_mpi_32 represents the MPIs generated using the Pushing the Boundaries of View Extrapolation with Multiplane Images. The original output of this method generate 128-layer MPIs, we merge the 128-layer MPIs to 32-layer due to network capacity limitation.
We use their public code for the MPIs generations. For the details of how the MPIs are generated, please refer to their public repo.
ICCV2021-Embedding-Novel-Views-in-a-Single-JPEG-Image/
│
└─── datasets/
|
└─── stereo_mpi/
│
└─── test_final/ # the test dataset of stereo-magnification
|
└─── pb_mpi_32/ # the test dataset of pb_mpi, we merge the original 128 mpi to 32 layers
|
└─── test_final /
└─── checkpoints/
|
└─── stereo_final
│
└─── models
│
└─── decoder_003_00015000.pth
│
└─── discriminator_003_00015000.pth
│
└─── encoder_003_00015000.pth
....
test.py
test.sh
....
The pretrained checkpoints are shared in the Google drive link as follows:
| Methods | Links |
|---|---|
| Stereo magnification | Google drive |
| PB-MPI | Google drive |
| LLFF | Google drive |
After put the test data and pretrained checkpoints in the corresponding folder,
Step1:
./test.sh
Run the test.sh to generate the embedding image and predicted MPIs.
The results will be stored in './checkpoints/[checkpoint_name]/test_result/
Step2:
./render.sh
Run the render.sh to render novel views using predicted MPIs. The results will be stored in './checkpoints/[checkpoint_name]/render/
- release training
- Write document and instructions