It is a real-time face detection and face tracking SDK. It analyses image data (eg. a camera stream, video stream or a static image) and returns facial landmarks and data to place 3d objects on a face.
Read the EULA (eula.txt) carefully before using the trial SDK. You can try and test the trial SDK free of charge. Before you buy a license, please test the SDK thoroughly to see if it meets the requirements of your project. Once you decide to use BRFv5 commercially, please contact us by email. You will receive a separate license agreement, which you must agree to.
- GitHub (all repos)
- BRFv5 Demo (platform: browser)
- ARTOv5 - Augmented Reality Try-On based on BRFv5
- TPPTv5 - ThreeJS Post Processing Tool for ARTOv5
- Docs / API
- Website
To test BRFv5 simply visit the JavaScript demo site:
Or download this repository from GitHub and run the index.html on a local server.
If you are looking to implement an Augmented Reality Try-On, that places 3d objects on to of the face, we got you covered. Try the demo here:
It's a Vue.js based web component, can be plugged into any website and is easily configurable.
See artov5/artov5_static/js/artov5_api.js for what's configurable.
ARTOv5 is available for paying customers.
Also available is TPPTv5 - ThreeJS Post Processing Tool for ARTOv5.
It allows you to load 3D models into the ThreeJS editor, place them correctly on a 3D head and export those models for ARTOv5. You can also conveniently try on your model within the editor.
Run the index.html on a local server.
Open the Xcode project. Attach your actual iOS device and run the example app on your device (examples need the device camera).
Open the Android Studio project. Attach your Android device and run the example app on your device (examples need the device camera).
Have OpenCV brewed (opencv3) on your system. Open the Xcode project and just run it on your Mac.
Good luck in trying to compile OpenCV for your Windows. Update the Visual Studio (2017) project properties that mention OpenCV. Then run the Release x64 target. Fingers crossed!
Use your preferred IDE. Add the src folder and the ANE itself to your class path and run the example class on your desired device (not in a simulator).
BRFv5 comes with the following components:
- Face Detection - finds faces (rectangles) in image data (camera stream, video or still image)
- Face Tracking - finds 68 facial landmarks/features
All available platform-specific packages have approximately the same content and come with a number of examples to demonstrate the use of the SDK.
You can input any image size.
Internally BRFv5 uses a XYZx480 (landscape) or 480xXYZ (portrait) image for the analysis. 480px is the base size that every other input size gets scaled to, eg.
landscape:
- 640 x 480 -> 640 x 480 // fastest, no scaling
- 1280 x 720 -> 854 x 480
- 1920 x 1080 -> 854 x 480
portrait:
- 480 x 640 -> 480 x 640 // fastest, no scaling
- 720 x 1280 -> 480 x 854
- 1080 x 1920 -> 480 x 854
BRFv5 scales the results up again, so you don't have to do that yourself. All parameters named *size or *width are pixel values based on the actual image size. eg. telling BRF what face sizes to initially detect:
If you work with a 640x480 camera stream, it would be something like this:
brfv5Config.faceDetectionConfig.minFaceSize = 144Where as if you work with a 1280x720 camera stream, you will need something like this:
brfv5Config.faceDetectionConfig.minFaceSize = 144 * (720 / 480)A common factor for the image base size might come in handy:
const inputSize = Math.min(imageWidth, imageHeight)
const sizeFactor = inputSize / 480.0
brfv5Config.faceDetectionConfig.minFaceSize = 144 * sizeFactorThis is implemented in the examples, just take a look at brfv5__configure.js.
Can I track other objects, eg. hands or neck or full body?
- No, the library is tracking faces only.
Can you improve the performance?
- BRFv5 has various configuration values that you can use to influence performance.
brfv5Config.faceTrackingConfig.numFacesToTrack
brfv5Config.faceTrackingConfig.numTrackingPasses
brfv5Config.faceTrackingConfig.enableStabilizer
brfv5Config.faceTrackingConfig.enableFreeRotation- BRFv5 also comes with different models, 42 landmarks for 3d object placement and 68 landmarks for detailed analysis, both come in a min, medium and max version. Please try those out and find the most fitting for your project.
Can you make the library smaller?
- We put a lot of effort into making the smallest possible library. The min versions of the models are the smallest that don't sacrifice too much accuracy. This might not be true for your use case. We might be able to provide even less accurate thus smaller models, but only for commercial customers.
When will the other platforms be available.
- We plan to work on the other example projects until the end of 2019.
v5.1.5 - 17th March 2020
- Added an ASM.js export. We do now support a manual ASM.js fallback. Since WebAssembly is available almost everywhere, there is no automatic checking/switching.
- Added a nodejs example. There is an issue with loading more than 6 model chunks for 68l, which leads to wrong results, not sure why yet.
v5.1.3 - 29th January 2020
- Version bump for emscripten to v1.39.6 - upstream
v5.1.0 - 14th November 2019
- BRFv5 is now 40% faster.
- v5.1.0 are not compatible with v5.0.x.
- Model files are not structured in chunks instead of _min, _medium, _max. Load any number of chunks from 4 to 8 chunks (lower is possible, but accuracy won't be great).
v5.0.2 - 10th November 2019
- Version bump for emscripten to v1.39.2 - fastcomp
v5.0.1 - 11th September 2019
- Internal restructuring of code, small fixes.
v5.0.0 - 26th August 2019
It's done! After over a year of development Tastenkunst is proud to announce the release of BRFv5.
Features:
- Face Tracking
- Face Detection
JavaScript Examples:
- Single Face Tracking
- Multi Face Tracking
- ThreeJS 3D Object Placement
- Image Overlay Placement
- Texture Overlay Placement
- Face Swap
- Extended Face Shape Estimation (Forehead)
- Smile Detection
- Blink Detection
- Yawn Detection
- Coloring Libs
Changes from BRFv4:
- Smaller models: The largest model is now 5MB (instead of 9MB for BRFv4). The smallest for 3D placement is 2,5MB.
- Rewrote the whole API to streamline configuration.
- Removed: Point Tracking. It was rarely requested and might come as a separate library if there is demand for it.
- Removed ASM package: WebAssembly is now widely supported.