nxp-appcodehub/dm-ml-ir-sensor-people-counting

This shows how to create a deep learning model to count the number of people in a room using a low-resolution 8x8 infrared array sensor.

NXP Application Code Hub

Leveraging deep learning to count the number of people in a room using a low-resolution 8x8 infrared array sensor

This demo shows how to create a deep learning model to count the number of people in a room using a low-resolution 8x8 infrared array sensor.

Boards: LPCXpresso55S69, LPCXpresso55S28

Categories: AI/ML, Vision

Peripherals: I2C, SENSOR, UART, TIMER, CLOCKS

Toolchains: MCUXpresso IDE

Table of Contents

1. Software
2. Hardware
3. Setup
4. Results
5. FAQs
6. Support
7. Release Notes

1. Software

• MCUXpresso v11.8.0
• LPCXpresso55S69 SDK from the MCUXpresso SDK Builder or from the MCUXpresso SDK repository.
• Python 3.11+
• Jupyter notebook or VS Code

2. Hardware

• LPCXpresso55s69
• LPCXpresso55s28
• Grid-EYE CLICK
• Micro USB cable

3. Setup

3.1 MCU Application Setup

1. Add the LPCXpresso55S69 SDK to the MCUXpresso IDE by right clicking on Installed SDKs and selecting Import archive... for the SDK downloaded from the SDK Builder or Import remote SDK Git repository... for the MCUXpresso SDK repository.
   
2. Clone this repository anywhere in your drive.
3. Open MCUXpresso and select File -> Open Projects from File System....
   
4. Under Import Source, select Directory..., navigate to <repo_location>/mcu_app/<board> and click on Select Folder.
5. Make Sure that the project is selected in the window and click Finish.
   
6. Right click on the project in the Project Explorer then navigate to Build Configurations -> Set Active -> Release.
   
7. Connect the Grid-EYE CLICK board to P23 and P24 as shown in the image below.
   
8. Connect the board to your computer through the Debug Link (P6) connector on the board.
9. Build the application by clicking on build and then flash it to the board by either clicking on Debug or by selecting the gray rectangle on the tool bar.
   
10. Mount the board to the ceiling for best results. Refer to Fig. 2 in this research paper.

3.2 Training Setup

1. We will use VS Code to open and run the Jupyter Notebook.
2. Open VS Code, click on File -> Open Folder, and navigate to <repo_location>/training and click Select Folder. Once opened, the Explorer should look like below.
   
3. Open the create_model.ipynb file and then click on Select Kernel in the top right.
   
4. Follow the instructions in the pop up to create a virtual environment.
5. After successfully creating a virtual environment, follow the instructions in the notebook to retrain or create a new model.
6. Once you have a new model, modify mcu_app/common/model/model_config.h and mcu_app/common/model/model_ops.cpp according to your new model.
7. Rebuild the MCU application, flash it to the board, and run it.

4. Results

• When running the model on the board, open a serial terminal and connect to the board to see the inference results.
  
• When running the model on your computer using real-time data from the board, the animation on the Jupyter Notebook will look like below.
  

5. FAQs

1. How do I generate the operations for model_ops.cpp?

   You can use the eIQ Toolkit's Model Tool to view all of the required operations for your model and manually add them to the ops resolver.

6. Support

Please submit any issues within this GitHub repository.

Project Metadata

Questions regarding the content/correctness of this example can be entered as Issues within this GitHub repository.

Warning: For more general technical questions regarding NXP Microcontrollers and the difference in expected funcionality, enter your questions on the NXP Community Forum

7. Release Notes

Version	Description / Update	Date
1.0	Initial release on Application Code Hub	September 26nd 2023