This is a camera app that continuously detects the objects (bounding boxes and classes) in the frames seen by your device's back camera, using a quantized MobileNet SSD model trained on the COCO dataset. These instructions walk you through building and running the demo on an Android device.
The model files are downloaded via Gradle scripts when you build and run. You don't need to do any steps to download TFLite models into the project explicitly.
Application can run either on device or emulator.
-
If you don't have already, install Android Studio, following the instructions on the website.
-
You need an Android device and Android development environment with minimum API 21.
-
Android Studio 3.2 or later.
-
Open Android Studio, and from the Welcome screen, select Open an existing Android Studio project.
-
From the Open File or Project window that appears, navigate to and select the tensorflow-lite/examples/object_detection/android directory from wherever you cloned the TensorFlow Lite sample GitHub repo. Click OK.
-
If it asks you to do a Gradle Sync, click OK.
-
You may also need to install various platforms and tools, if you get errors like "Failed to find target with hash string 'android-21'" and similar. Click the
Run
button (the green arrow) or selectRun > Run 'android'
from the top menu. You may need to rebuild the project usingBuild > Rebuild
Project. -
This object detection Android reference app demonstrates two implementation solutions, lib_task_api that leverages the out-of-box API from the TensorFlow Lite Task Library, and lib_interpreter that creates the custom inference pipleline using the TensorFlow Lite Interpreter Java API. You can change the build variant to whichever one you want to build and run—just go to
Build > Select Build Variant
and select one from the drop-down menu. See configure product flavors in Android Studio for more details. -
If it asks you to use Instant Run, click Proceed Without Instant Run.
-
Also, you need to have an Android device plugged in with developer options enabled at this point. See here for more details on setting up developer devices.
Downloading, extraction and placing it in assets folder has been managed automatically by download.gradle.
If you explicitly want to download the model, you can download from here. Extract the zip to get the .tflite and label file.
This example shows you how to perform TensorFlow Lite object detection using a custom model.
- Clone the TensorFlow models GitHub repository to your computer.
git clone https://github.com/tensorflow/models/
- Build and install this repository.
cd models/research
python3 setup.py build && python3 setup.py install
- Download the MobileNet SSD trained on Open Images v4 here. Extract the pretrained TensorFlow model files.
- Go to
models/research
directory and execute this code to get the frozen TensorFlow Lite graph.
python3 object_detection/export_tflite_ssd_graph.py \
--pipeline_config_path object_detection/samples/configs/ssd_mobilenet_v2_oid_v4.config \
--trained_checkpoint_prefix <directory with ssd_mobilenet_v2_oid_v4_2018_12_12>/model.ckpt \
--output_directory exported_model
- Convert the frozen graph to the TFLite model.
tflite_convert \
--input_shape=1,300,300,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 \
--allow_custom_ops \
--graph_def_file=exported_model/tflite_graph.pb \
--output_file=<directory with the TensorFlow examples repository>/lite/examples/object_detection/android/app/src/main/assets/detect.tflite
input_shape=1,300,300,3
because the pretrained model works only with that input shape.
allow_custom_ops
is necessary to allow TFLite_Detection_PostProcess operation.
input_arrays
and output_arrays
can be drawn from the visualized graph of the example detection model.
bazel run //tensorflow/lite/tools:visualize \
"<directory with the TensorFlow examples repository>/lite/examples/object_detection/android/app/src/main/assets/detect.tflite" \
detect.html
- Get
labelmap.txt
from the second column of class-descriptions-boxable. - In
DetectorActivity.java
setTF_OD_API_IS_QUANTIZED
tofalse
.
Please do not delete the assets folder content. If you explicitly deleted the files, then please choose Build->Rebuild from menu to re-download the deleted model files into assets folder.