/vision

Object detection in the browser

Primary LanguageHTML

Vision app

Use your device's camera to run object detection on the world around you.

Bounding box example

Online demo

You can see a demo here

Building the app

You need to install binaryen in order to have access to wasm-opt, which will help to shirkn the final size of the wasm binary.

Through a combination of building in release mode, compiling with Link Time Optimizations (LTO), telling LLVM to optimize for size instead of speed, and running wasm-opt, we can reduce the size of the final wasm binary by as much as 80%.

We recommend reading https://rustwasm.github.io/book/reference/code-size.html#optimizing-builds-for-code-size for additional information.

See our build script for the exact commands that we use to build the application.

See the [profile.release] section of Cargo.toml to take a look at the LLVM optimizations used during the build.

Development Mode

The following command will keep a copy of the app running locally, if you want to play around with the code.

cargo web start --target=wasm32-unknown-unknown

Attribution

Much of the HTML and javascript basis were made possible through openly licensed work from the Tensorflow.js COCO-SSD model. Thank you.

The drawing of bounding boxes was almost entirely made possible by Paul Irish's demo of multi-touch. Thank you.