TensorFlow Lite Micro for Espressif Chipsets

As per TFLite Micro guidelines for vendor support, this repository has the esp-tflite-micro component and the examples needed to use Tensorflow Lite Micro on Espressif Chipsets (e.g., ESP32-P4) using ESP-IDF platform.
The base repo on which this is based can be found here.

Build Status

Build Type	Status
Examples Build

How to Install

ESP-IDF Support Policy

We keep track with the ESP-IDF's support period policy mentioned here.

Currently ESP-IDF versions release/v4.4 and above are supported by this project.

Install the ESP IDF

Follow the instructions of the ESP-IDF get started guide to setup the toolchain and the ESP-IDF itself.

The next steps assume that this installation is successful and the IDF environment variables are set. Specifically,

the IDF_PATH environment variable is set
the idf.py and Xtensa-esp32 tools (e.g., xtensa-esp32-elf-gcc) are in $PATH

Using the component

Run the following command in your ESP-IDF project to install this component:

idf.py add-dependency "esp-tflite-micro"

Building the example

To get the example, run the following command:

idf.py create-project-from-example "esp-tflite-micro:<example_name>"

Note:

If you have cloned the repo, the examples come as the part of the clone. Simply go to the example directory (examples/<example_name>) and build the example.

Available examples are:

hello_world
micro_speech
person_detection

Set the IDF_TARGET

idf.py set-target esp32p4

To build the example, run:

idf.py build

Load and run the example

To flash (replace /dev/ttyUSB0 with the device serial port):

idf.py --port /dev/ttyUSB0 flash

Monitor the serial output:

idf.py --port /dev/ttyUSB0 monitor

Use Ctrl+] to exit.

The previous two commands can be combined:

idf.py --port /dev/ttyUSB0 flash monitor

Please follow example READMEs for more details.

ESP-NN Integration

ESP-NN contains optimized kernel implementations for kernels used in TFLite Micro. The library is integrated with this repo and gets compiled as a part of every example. Additional information along with performance numbers can be found here.

Performance Comparison

A quick summary of ESP-NN optimisations, measured on various chipsets:

Target	TFLite Micro Example	without ESP-NN	with ESP-NN	CPU Freq
ESP32-P4	Person Detection	1395ms	73ms	360MHz
ESP32-S3	Person Detection	2300ms	54ms	240MHz
ESP32	Person Detection	4084ms	380ms	240MHz
ESP32-C3	Person Detection	3355ms	426ms	160MHz

Note:

The above is time taken for execution of the invoke() call
Internal memory used
ESP32-P4 optimisation is work in progress

Without ESP-NN case is when esp-nn is completely disabled by removing below flag from CMakeLists.txt:

  # enable ESP-NN optimizations by Espressif
  target_compile_options(${COMPONENT_LIB} PRIVATE -DESP_NN)

Detailed kernelwise performance can be found here.

Sync to latest TFLite Micro upstream

As per the upstream repository policy, the tflite-lib is copied into the components directory in this repository. We keep updating this to the latest upstream version from time to time. Should you, in any case, wish to update it locally, you may run the scripts/sync_from_tflite_micro.sh script.

Contributing

If you find an issue in these examples, or wish to submit an enhancement request, please use the Issues section on Github.
For ESP-IDF related issues please use esp-idf repo.
For TensorFlow related information use tflite-micro repo.

License

This component and the examples are provided under Apache 2.0 license, see LICENSE file for details.

TensorFlow library code and third_party code contains their own license specified under respective repos.

kyvaith/esp-tflite-micro