AlvaAR

AlvaAR is a realtime visual SLAM algorithm running as WebAssembly, in the browser. It is a heavily modified version of the OV²SLAM and ORB-SLAM2 projects. SLAM is the core building block of Augmented Reality applications focusing on world tracking.

Examples

The examples use ThreeJS to apply and render the estimated camera pose to a 3d environment.

Video Demo: A desktop browser version using a video file as input.
Camera Demo: The mobile version will access the device camera as input.

Run with http server

To run the examples on your local machine, start a simple http server in the examples/ folder:

$: python 2: python -m SimpleHTTPServer 8080 or
$: python 3: python -m http.server 8080 or
$: emrun --browser chrome ./

Then open http://localhost:8080/public/video.html in your browser.

Run with https server

To run the examples on another device in your local network, they must be served via https. For convenience, a simple https server was added to this project – do not use for production.

1) Install server dependencies

    $: cd ./AlvaAR/examples/
    $: npm install

2) Generate self-signed certificate

    $: cd ./AlvaAR/examples
    $: mkdir ssl/
    $: cd ssl/
    $: openssl req -nodes -new -x509 -keyout key.pem -out cert.pem

3) Run

    $: cd ./AlvaAR/examples/
    $: nvm use 13.2
    $: npm start

Then open https://YOUR_IP:443/video.html in your browser. If met with a ERR_CERT_INVALID error in Chrome, try typing badidea or thisisunsafe directly in Chrome on the same page. Don’t do this unless the site is one you trust or develop.

Usage

This code shows how to send image data to AlvaAR to compute the camera pose.

import { AlvaAR } from 'alva_ar.js';

const videoOrWebcam = /*...*/;

const width = videoOrWebcam.width;
const height = videoOrWebcam.height;

const canvas = document.getElementById( 'canvas' );
const ctx = canvas.getContext( '2d' );

canvas.width = width;
canvas.height = height;

const alva = await AlvaAR.Initialize( width, height );

function loop()
{
    ctx.clearRect( 0, 0, width, height );
    ctx.drawImage( videoOrWebcam, 0, 0, width, height );
    
    const frame = ctx.getImageData( 0, 0, width, height );
    
    // cameraPose holds the rotation/translation information where the camera is estimated to be
    const cameraPose = alva.findCameraPose( frame );
    
    // planePose holds the rotation/translation information of a detected plane
    const planePose = alva.findPlane();
    
    // The tracked points in the frame
    const points = alva.getFramePoints();

    for( const p of points )
    {
        ctx.fillRect( p.x, p.y, 2, 2 );
    }
};

Build

Prerequisites

Emscripten

Ensure Emscripten is installed and activated in your session.

    $: source [PATH]/emsdk/emsdk_env.sh 
    $: emcc -v

C++11 or Higher

Alva makes use of C++11 features and should thus be compiled with a C++11 or higher flag.

Dependencies

Dependency	Description
Eigen3	Download Eigen 3.4. Find all releases here.This project has been tested with 3.4.0
OpenCV	Download OpenCV 4.5. Find all releases here.This project has been tested with 4.5.5.
iBoW-LCD	A modified version of iBoW-LCD is included in the libs folder. It has been turned into a static shared lib. Same goes for OBIndex2, the required dependency for iBoW-LCD. Check the lcdetector.h and lcdetector.cc files to see the modifications w.r.t. to the original code. Both CMakeList have been adjusted to work with Emscripten.
Sophus	Sophus is used for SE(3), SO(3) elements representation.
Ceres Solver	Ceres is used for optimization related operations such as PnP, Bundle Adjustment or PoseGraph Optimization. Note that Ceres dependencies are still required.
OpenGV	OpenGV is used for Multi-View-Geometry (MVG) operations.

Build Dependencies

For convenience, a copy of all required libraries has been included in the libs/ folder. Run the following script to compile all libraries to wasm modules which can be linked into the main project.

    $: cd ./AlvaAR/src/libs/
    $: ./build.sh

Build Project

Run the following in your shell before invoking emcmake or emmake:

    $: [PATH]/emsdk/emsdk_env.sh

Then, run the following:

    $: cd ./AlvaAR/src/slam
    $: mkdir build/
    $: cd build/
    $: emcmake cmake .. 
    $: emmake make install

Roadmap

Improve the initialisation phase to be more stable and predictable.
Move feature extraction and tracking to GPU.
Blend visual SLAM with IMU data to increase robustness.

License

AlvaAR is released under the GPLv3 license.

OV²SLAM and ORB-SLAM2 are both released under the GPLv3 license. Please see 3rd party dependency licenses in libs/.

Contact

Alan Ross: @alan_ross or me@aross.io
Project: https://github.com/alanross/AlvaAR

HEOJUNFO/CCSLAM