If you find this work interesting please consider reading our paper
This research is done as part of a research lab at TU Darmstadt by the authors
Jan Coccejus, Jan Helge Dörsam, Korbinian Kunst and Felix Wirth
under the supervision of Jun Hur and Prof. Stefan Roth
This paper proposes a modified approach for dense depth estimation from monocular images. We model a complex 3D scene via over-segmentation via superpixels as a piecewise planar and rigid approximation. Based on this assumption we represent every planar by surface normals/plane coefficients. In this way we solve the homogeneous depth estimation problem that our baseline architecture Monodepth2 from Godard et al. 2019 suffered. In particular we propose (i) a normal-2-block inside the architecture that estimates surface normal coefficients, (ii) a superpixel-loss that incorporates superpixel information and exploits sharper edges and (iii) a normal loss that ensure homogeneous depth for planar surfaces. We demonstrate the effectiveness of the proposed improvements in an detailed depth-map analysis and show comparable scoring metric with state-of-the-art results on the KITTI Eigen-Zhou split.
Results from a single image. On the left: Our depth prediction results on a randomly picked image from KITTI on the right: To the chosen image the according over-segmentated image with surface normals as produced by our network are plotted beginning inthe center of mass for each planar structure. The magnitude hasbeen scaled due to visibility reasons On the left the depth map is displayed.
In depth display of modification that have been made in this work, including serveral loss functions and two superpixel methods for oversegmentation.
Decoder | Inp. Channels | Sup. Method | Loss Function | Abs Rel | Sq Rel | RSME | RSME log | δ<1.25 | δ<1.25² | δ<1.25³ | |
---|---|---|---|---|---|---|---|---|---|---|---|
Baseline | standard | 3 | standard | 0.115 | 0.903 | 4.863 | 0.193 | 0.877 | 0.959 | 0.981 | |
N2D | normals | 3 | standard | 0.123 | 0.984 | 5.042 | 0.2 | 0.859 | 0.955 | 0.98 | |
4Ch | standard | 4 | felzenwalb | standard | 0.141 | 1.313 | 5.545 | 0.22 | 0.834 | 0.942 | 0.974 |
4Ch | standard | 4 | slic | standard | 0.255 | 2.237 | 7.892 | 0.342 | 0.594 | 9.832 | 0.927 |
6Ch | standard | 6 | felzenwalb | standard | 0.122 | 0.978 | 5.026 | 0.2 | 0.862 | 0.955 | 0.979 |
4Ch + N2D | normals | 4 | felzenwalb | standard | 0.142 | 1.262 | 5.551 | 0.22 | 0.83 | 0.942 | 0.975 |
4Ch + N2D + bin | normals | 4 | felzenwalb | binary | 0.443 | 4.757 | 12.083 | 0.588 | 0.303 | 0.561 | 0.766 |
3Ch + N2D + bin | normals | 3 | binary | 0.443 | 4.757 | 12.083 | 0.588 | 0.303 | 0.561 | 0.766 | |
4Ch + N2D + cont | normals | 4 | felzenwalb | continous | 0.138 | 1.185 | 5.484 | 0.218 | 0.832 | 0.944 | 0.975 |
4Ch + N2D + cont | normals | 4 | slic | continous | 0.443 | 4.757 | 12.083 | 0.588 | 0.303 | 0.561 | 0.766 |
3Ch + N2D + cont | normals | 3 | continous | 0.443 | 4.757 | 12.083 | 0.588 | 0.303 | 0.561 | 0.766 | |
4Ch + N2D + 0.001 norm | normals | 4 | felzenwalb | 0.001 * normal | 0.139 | 1.193 | 5.525 | 0.22 | 0.831 | 0.941 | 0.974 |
4Ch + N2D + 0.01 norm | normals | 4 | felzenwalb | 0.01 * normal | 0.139 | 1.172 | 5.513 | 0.218 | 0.831 | 0.942 | 0.975 |
4Ch + N2D + 0.1 norm | normals | 4 | felzenwalb | 0.1 * normal | 0.443 | 4.757 | 12.083 | 0.588 | 0.303 | 0.561 | 0.766 |
4Ch + N2D + bin + norm | normals | 4 | felzenwalb | binary + normal | 0.443 | 4.757 | 12.083 | 0.588 | 0.303 | 0.561 | 0.766 |
4Ch + N2D + cont + norm | normals | 4 | felzenwalb | continous + normal loss | 0.141 | 1.276 | 5.549 | 0.221 | 0.832 | 0.941 | 0.974 |
4Ch + N2D + cont + norm | normals | 4 | slic | continous & normal | 0.443 | 4.757 | 12.083 | 0.588 | 0.303 | 0.561 | 0.766 |
3Ch+ N2D + cont + norm | normals | 3 | continous + normal | 0.443 | 4.757 | 12.083 | 0.588 | 0.303 | 0.561 | 0.766 |