Overview • Install • Usage • Related • References
Pandora stereo matching framework is designed to provide some state of the art stereo algorithms and to add others one as plugins.
This Pandora plugin aims to compute the cost volume using the similarity measure produced by MC-CNN neural network [MCCNN], with the MCCNN library .
pandora_plugin_mccnn is available on Pypi and can be installed by:
pip install pandora_plugin_mccnnThis command will installed required dependencies as Pandora and MCCNN.
Let's refer to Pandora's readme or online documentation for further information about Pandora general functionalities.
More specifically, you can find :
Pretrained weights for mc-cnn fast and mc-cnn accurate neural networks are available in the weights directory :
- mc_cnn_fast_mb_weights.pt and mc_cnn_accurate_mb_weights.pt are the weights of the pretrained networks on the Middlebury dataset [Middlebury]
- mc_cnn_fast_data_fusion_contest.pt and mc_cnn_accurate_data_fusion_contest.pt are the weights of the pretrained networks on the Data Fusion Contest dataset [DFC]
To download the pretrained weights:
wget https://raw.githubusercontent.com/CNES/Pandora_plugin_mccnn/master/weights/mc_cnn_fast_mb_weights.pt
wget https://raw.githubusercontent.com/CNES/Pandora_plugin_mccnn/master/weights/mc_cnn_fast_data_fusion_contest.pt
wget https://raw.githubusercontent.com/CNES/Pandora_plugin_mccnn/master/weights/mc_cnn_accurate_mb_weights.pt
wget https://raw.githubusercontent.com/CNES/Pandora_plugin_mccnn/master/weights/mc_cnn_accurate_data_fusion_contest.ptThe figures below show disparity maps produced on mountain, and desert areas generated with the Census and MCCNN similarity measures :
| Left image | Left disparity map using Census measure | Left disparity map using mc-cnn fast pretrained on Middlebury | Left disparity map using mc-cnn fast pretrained DFC |
|---|---|---|---|
 |
 |
 |
 |
 |
 |
 |
 |
Pandora - A stereo matching framework
MCCNN - Pytorch/python implementation of mc-cnn neural network
Please cite the following paper when using Pandora and pandora_plugin_mccnn:
Cournet, M., Sarrazin, E., Dumas, L., Michel, J., Guinet, J., Youssefi, D., Defonte, V., Fardet, Q., 2020. Ground-truth generation and disparity estimation for optical satellite imagery. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.
[MCCNN] Zbontar, J., & LeCun, Y. (2016). Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res., 17(1), 2287-2318.
[Middlebury] Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nešić, N., Wang, X., & Westling, P. (2014, September). High-resolution stereo datasets with subpixel-accurate ground truth. In German conference on pattern recognition (pp. 31-42). Springer, Cham.
[DFC] Bosch, M., Foster, K., Christie, G., Wang, S., Hager, G. D., & Brown, M. (2019, January). Semantic stereo for incidental satellite images. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) (pp. 1524-1532). IEEE.