Structural analysis of microscopic objects has been a longstanding topic in several scientific disciplines, such as biological, medical, mechanical, and materials sciences. The scanning electron microscope (SEM), as a very promising imaging equipment has been around for decades to analyze the surface properties (e.g., compositions or geometries) of microscopic samples by achieving increased magnification, contrast, and resolution greater than one nanometer [1]. While SEM micrographs still remain two-dimensional (2D), many research and educational questions truly require information about their three-dimensional (3D) structures. 3D surface reconstruction from SEM images leads to remarkable understanding of microscopic surfaces, allowing informative and qualitative visualization of the samples being investigated. The objective of this GitHub repository is to provide a source for 3D SEM surface reconstruction study, presenting recent 3D SEM advances and progresses to the research community.
Making a 3D shape model of a microscopic sample is still difficult to solve since its 3D shape model in the real world is only projected into and available as only 2D digital images. Over the past few years, there has been an expansion in designing and developing 3D surface reconstruction algorithms for images obtained by a SEM. All these algorithms are categorized into three main classes: 1) Single-View, 2) Multi-View and 3) Hybrid [1, 2, 3], all with use of either sparse or dense reconstruction strategies.
The state-of-the-art of 3D SEM surface reconstruction is fully explained in the following works:
[1] Tafti, A.P., Kirkpatrick, A.B., Alavi, Z., Owen, H.A. and Yu, Z., 2015. Recent advances in 3D SEM surface reconstruction Micron, 78, pp.54-66.
[2] Kremer, A., Lippens, S., Bartunkova, S., Asselbergh, B., Blanpain, C., Fendrych, M., Goossens, A., Holt, M., Janssens, S., Krols, M. and LARSIMONT, J.C., 2015. Developing 3D SEM in a broad biological context Journal of microscopy, 259(2), pp.80-96.
[3] Baghaie A, Tafti AP, Owen HA, D’Souza RM, Yu Z., 2017. Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation PloS one. 2017;12(4).
There is a considerable amount of research activities that tried to design and develop 3D SEM surface reconstruction algorithms. Examples include:
[1] Tafti, A.P., Kirkpatrick, A.B., Alavi, Z., Owen, H.A. and Yu, Z., 2015. Recent advances in 3D SEM surface reconstruction. Micron, 78, pp.54-66.
[2] Eulitz, M. and Reiss, G., 2015. 3D reconstruction of SEM images by use of optical photogrammetry software. Journal of structural biology, 191(2), pp.190-196.
[3] Tafti, A.P., 2016. 3D SEM Surface Reconstruction: An Optimized, Adaptive, and Intelligent Approach. (Doctoral dissertation, The University of Wisconsin-Milwaukee).
[4] Kudryavtsev, A.V., Dembélé, S. and Piat, N., 2017. Stereo-image rectification for dense 3D reconstruction in scanning electron microscope. In Manipulation, Automation and Robotics at Small Scales (MARSS), 2017 International Conference on (pp. 1-6). IEEE.
[5] Yan, S., Adegbule, A. and Kibbey, T.C., 2017. A hybrid 3D SEM reconstruction method optimized for complex geologic material surfaces Micron, 99, pp.26-31.
[6] Tafti, A.P., Holz, J.D., Baghaie, A., Owen, H.A., He, M.M. and Yu, Z., 2016. 3DSEM++: Adaptive and intelligent 3D SEM surface reconstruction. Micron, 87, pp.33-45.
[7] Baghaie, A., Tafti, A.P., Owen, H.A., D'Souza, R.M. and Yu, Z., 2017. SD-SEM: sparse-dense correspondence for 3D reconstruction of microscopic samples. Micron, 97, pp.41-55.
[8] Baghaie A, Tafti AP, Owen HA, D’Souza RM, Yu Z., 2017. Three-dimensional reconstruction of highly complex microscopic samples using scanning electron microscopy and optical flow estimation PloS one. 2017;12(4).
Click on the pictures below to start YouTube video demonstrations.
3D Surface Modeling of Microscopic Objects: a Computer Vision Adventure
Multiple-View Geometry
Application of 3D SEM surface reconstruction broadly lies in different scietific domains. The following works are good resource to imvestigate the application area of 3D SEM vision:
[1] Limandri, S., Josa, V.G., Valentinuzzi, M.C., Chena, M.E. and Castellano, G., 2016. 3D scanning electron microscopy applied to surface characterization of fluorosed dental enamel. Micron, 84, pp.54-60.
[2] Eulitz, M. and Reiss, G., 2015. 3D reconstruction of SEM images by use of optical photogrammetry software. Journal of structural biology, 191(2), pp.190-196.
[3] Omrani, E., Tafti, A.P., Fathi, M.F., Moghadam, A.D., Rohatgi, P., D'Souza, R.M. and Yu, Z., 2016. Tribological study in microscale using 3D SEM surface reconstruction. Tribology International, 103, pp.309-315.
[4] Kim, K.W., 2016. Biomedical Applications of Stereoscopy for Three-Dimensional Surface Reconstruction in Scanning Electron Microscopes. Applied Microscopy, 46(2), pp.71-75.
3DSEM Dataset: Harvard Dataverse
Paper: Tafti AP, Kirkpatrick AB, Holz JD, Owen HA, Yu Z, 2016. 3DSEM: A 3D microscopy dataset. Data in Brief. 2016 Mar 1;6:112-6.
MeX (Alicona): Mex
Avizo Software: Avizo
Ahmad P. Tafti, PhD
Ahmadreza Baghaie, PhD
Mona Eulitz, PhD
Zahrasadat Alavi, PhD
Emad Omrani, PhD
Mojtaba Fathi, PhD
Afsaneh Dorri Moghadam, PhD
Andrew B. Kirkpatrick, MS
Heather Owen, PhD
Zeyun Yu, PhD
[1] Tafti, A.P., Kirkpatrick, A.B., Alavi, Z., Owen, H.A. and Yu, Z., 2015. Recent advances in 3D SEM surface reconstruction Micron, 78, pp.54-66.
[2] Tafti, A.P., 2016. 3D SEM Surface Reconstruction: An Optimized, Adaptive, and Intelligent Approach (Doctoral dissertation, The University of Wisconsin-Milwaukee).
[3] Tafti, A.P., Holz, J.D., Baghaie, A., Owen, H.A., He, M.M. and Yu, Z., 2016. 3DSEM++: Adaptive and intelligent 3D SEM surface reconstruction. Micron, 87, pp.33-45.