ORB_SLAM3_AR-for-Android

Introduction

This is a Android Augmented Reality APP based on ORB-SLAM3 and OpenGL. (TODO: The demo video can be found in the links below.)

Notice

We thank the great contribution of @muziyongshixin.

This project is simple and basic, and I am focus on SLAM algorithm, so you can disscuss with @Abonaventure or bonaventure@163.com if you have any issues or questions.

Demo Videos

(TODO)

Dependencies & Installation & Usage

(1)Due to the project is based on ORB-SLAM3, OpenCV4Android is needed. Other third part dependence like DBow2, g2o, Sophus, Eigen,boost, openssl and opencv, are all included in the project(I have uploaded them all except Eigen source files, boost head files and OpenCV4Android). As for the IDE, android-studio-2021.1.1.22 or higher version is recommended. To compile the project you may need to edit some configurations in the CMakeLists.txt file, which is in the path /app/CMakeLists.txt

## download Eigen, boost and OpenCV4Android

1)download eigen-3.4.0 and crack its source files into the following directory
 */app/src/main/cpp/Eigen/eigen-3.4.0/
 
2)download boost-1_72_0 and crack its head files(boost directory) into the following directory
 */app/src/main/cpp/boost/boost-1_72_0/
 
3)download opencv-4.5.5-android-sdk and crack its source files into the following directory
 */opencvLibrary/
 
4)download opencv-4.5.5-android-sdk and crack its include files and libs files into the following directory
4.1)from opencv-4.5.5-android-sdk/sdk/native/jni/include/ to
 */app/src/main/cpp/opencv/opencv-4.5.5/include/
4.2)from opencv-4.5.5-android-sdk/sdk/native/libs/ to
 */app/src/main/jniLibs/

(2)To run this APP you need to copy the ./SLAM folder into your mobile phones, which contains the vocabulary file(ORBvoc.bin) and camera Calibration file(PARAconfig.yaml). Then you need to change the row " SLAM = new ORB_SLAM3::System("/storage/emulated/0/SLAM/VOC/ORBvoc.bin","/storage/emulated/0/SLAM/Calibration/PARAconfig.yaml",ORB_SLAM3::System::MONOCULAR,false); "in the ./app/src/main/cpp/native-lib.cpp to configure the correct path in your phone. You may also need to calibrate your phone's camera for better performance. More details of camera calibration can be found on google.

 SLAM = new ORB_SLAM3::System("/storage/emulated/0/SLAM/VOC/ORBvoc.bin","/storage/emulated/0/SLAM/Calibration/PARAconfig-mono-orb3.yaml",ORB_SLAM3::System::MONOCULAR,false);

(3)Due to the diversity of Android system version, I am not sure weather my configuration can work well on other Android devices. So you may also need to change some configurarions in the AndroidManifest.xml to make sure that the app have the authority to use the camera and file system.

Framework & Results

The system is consisted of two parts(https://github.com/Abonaventure/ORB_SLAM3_AR-for-Android), the ORB-SLAM3 part which is used to get the camera's pose matrix. The other part is the OpenGL Rendering module, which use the pose matrix to render the 3D object(the earth in this project).