/Seeed_Grove_Vision_AI_Module_V2

Primary LanguageCMIT LicenseMIT

Himax examples for Seeed Grove Vision AI Module V2

This is a repository which step by step teaches you how to build your own examples and run on Seeed Grove Vision AI Module V2. Finally, teach you how to restore to the original factory settings and run SenseCraft AI from Seeed Studio.

Outline

scenario_app project name
face mesh tflm_fd_fm
yolov8n object detection tflm_yolov8_od
yolov8n pose tflm_yolov8_pose
pdm mic record pdm_record
KeyWord Spotting using Transformers kws_pdm_record
imu read imu_read
peoplenet from TAO tflm_peoplenet

How to build the firmware?

This part explains how you can build the firmware for Grove Vision AI Module V2.

Build the firmware at Linux environment

Note: The following has been tested to work on Ubuntu 20.04 PC

  • Step 1: Install the following prerequisites 
    sudo apt install make
  • Step 2: Download Arm GNU Toolchain (arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz) 
    cd ~
wget https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
  • Step 3: Extract the file 
    tar -xvf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz
  • Step 4: Add arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/: to PATH 
    export PATH="$HOME/arm-gnu-toolchain-13.2.Rel1-x86_64-arm-none-eabi/bin/:$PATH"
  • Step 5: Clone the following repository and go into Seeed_Grove_Vision_AI_Module_V2 folder 
    git clone --recursive https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2.git
cd Seeed_Grove_Vision_AI_Module_V2
  • Step 6: Compile the firmware 
    cd EPII_CM55M_APP_S
make clean
make
  • Output elf file: ./obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf alt text
  • Step 7: Generate firmware image file 
    cd ../we2_image_gen_local/
cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
./we2_local_image_gen project_case1_blp_wlcsp.json
  • Output firmware image: ./output_case1_sec_wlcsp/output.img alt text

Back to Outline

Build the firmware at MacOS environment

Note: The steps are almost the same as the Linux environment except Step 1 and Step 7.

  • Step 1:
    • You should make sure your make is using GNU version make not BSD version make. 
      make --version
      alt text
    • If it is not GNU make , you should download it by following command. 
      brew install make
    • After installation, you can access it with the command gmake to avoid conflicts with the default make. 
      gmake
    • So, you should create an alias in your shell configuration file (like .bash_profile or .zshrc): 
      alias make='gmake'
  • Step 7: Generate firmware image file (using ./we2_local_image_gen_macOS_arm64 for MacOS) 
    cd ../we2_image_gen_local/
cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
./we2_local_image_gen_macOS_arm64 project_case1_blp_wlcsp.json

Back to Outline

Build the firmware at Windows environment

  • Step 1: Install the make command for prerequisites , you can reference here
  • Step 2: Download Arm GNU Toolchain arm-gnu-toolchain-13.2.rel1-mingw-w64-i686-arm-none-eabi.zip
  • Step 3: Extract the file 
    tar -xvf arm-gnu-toolchain-13.2.rel1-mingw-w64-i686-arm-none-eabi.zip
  • Step 4: Add arm-gnu-toolchain-13.2.rel1-mingw-w64-i686-arm-none-eabi/bin/: to PATH 
    setx PATH "%PATH%;[location of your gnu-toolchain-13.2 ROOT]\arm-gnu-toolchain-13.2.rel1-mingw-w64-i686-arm-none-eabi\bin"
  • Step 5: Clone the following repository and go into Seeed_Grove_Vision_AI_Module_V2 folder 
    git clone --recursive https://github.com/HimaxWiseEyePlus/Seeed_Grove_Vision_AI_Module_V2.git
cd Seeed_Grove_Vision_AI_Module_V2
  • Step 6: Compile the firmware 
    cd EPII_CM55M_APP_S
make clean
make
  • Output elf file: ./obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf alt text
  • Step 7: Generate firmware image file 
    cd ../we2_image_gen_local/
cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
we2_local_image_gen project_case1_blp_wlcsp.json
  • Output firmware image: ./output_case1_sec_wlcsp/output.img alt text

Back to Outline

How to flash the firmware?

This part explains how you can flash the firmware to Grove Vision AI Module V2.

System Requirement

  • Driver
    • If you find that the Grove Vision AI V2 is not recognised after connecting it to your computer, you should install the driver which can reference here.
  1. Grove Vision AI Module V2
  2. Connection cable
    • Micro usb cable: connect to EVB (as Power/UART)
  3. Software Tools Serial terminal emulation application
    • In the following description, TeraTerm and Minicom will be used.
      • Serial terminal emulation application Setting
        • Baud Rate 921600 bps
        • Data 8 bit
        • Parity none
        • Stop 1 bit
        • Flow control none
        • please check xmodem protocol is supported.
      • Minicom (for Linux PC)
        • Install minicom command 
          sudo apt-get install minicom
        • Burn application to flash by using xmodem send application binary.
          • Minicom will extra install "lrzsz" package to support xmodem protocol alt text
          • If you did not have “lrzsz” instruction, please install by following instruction. 
            sudo apt-get install lrzsz #(to support xmodem protocol)
        • Open the permissions to access the device 
          sudo setfacl -m u:[USERNAME]:rw /dev/ttyUSB0
# in my case
# sudo setfacl -m u:kris:rw /dev/ttyACM0
          alt text
        • Open minicom 
          sudo minicom -s
          alt text
        • Setup serial port and COM Port name alt text
          • Tips for finding the COM Port name.
            • You can use google chrome to connect to Seeed SenseCraft AI, select Grove Vision(V2) and press Connect. alt text
            • Then, you would get the COM Port name. alt text
      • TeraTerm (for Windows PC)
        • Setup serial port alt text alt text

Back to Outline

Flash Image Update at Linux Environment by python code

  • Prerequisites for xmodem
  • Disconnect Minicom
  • Make sure your Seeed Grove Vision AI Module V2 is connect to PC.
  • Open the permissions to acceess the deivce
sudo setfacl -m u:[USERNAME]:rw /dev/ttyUSB0
# in my case
# sudo setfacl -m u:kris:rw /dev/ttyACM0

alt text

  • Open Terminal and key-in following command
  • port: the COM number of your Seeed Grove Vision AI Module V2, for example,/dev/ttyACM0
  • baudrate: 921600
  • file: your firmware image [maximum size is 1MB] 
    python3 xmodem/xmodem_send.py --port=[your COM number] --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local/output_case1_sec_wlcsp/output.img

# example:
# python3 xmodem/xmodem_send.py --port=/dev/ttyACM0 --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local/output_case1_sec_wlcsp/output.img
  • It will start to burn firmware image. alt text
  • Please press reset buttun on Seeed Grove Vision AI Module V2. alt text
  • It will success to run the algorithm.

Back to Outline

Flash Image Update at Windows Environment by python code

  • Prerequisites for xmodem
  • Disconnect Tera Term
  • Make sure your Seeed Grove Vision AI Module V2 is connect to PC.
  • Open CMD and key-in following command
  • port: the COM number of your Seeed Grove Vision AI Module V2
  • baudrate: 921600
  • file: your firmware image [maximum size is 1MB] 
    python xmodem\xmodem_send.py --port=[your COM number] --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local\output_case1_sec_wlcsp\output.img 
# example:
# python xmodem\xmodem_send.py --port=COM123 --baudrate=921600 --protocol=xmodem --file=we2_image_gen_local\output_case1_sec_wlcsp\output.img
  • It will start to burn firmware image automatically. alt text
  • Please press reset buttun on Seeed Grove Vision AI Module V2. alt text
  • It will success to run the algorithm.

Back to Outline

Flash Image Update at Linux Environment

Following steps update application in the flash.

  • Step 1: Open Minicom, setup serial port and COM Port name-> connect to Grove Vision AI Module V2. (Please reference the minicom part of System Requirement) alt text
  • Step 2: Hold down any key on the keyboard (except the Enter key) and press the reset button to reset Grove Vision AI Module V2 and the startup options will be displayed. alt text
  • Step 3: Press button “1” and Grove Vision AI Module V2 will enter receiving mode after then.
    alt text
  • Step 4: Press Ctrl+A on keyboard to enter minicom menu, and then press s on keyboard to upload file and select xmodem. alt text
  • Step 5: Select the firmware image at Seeed_Grove_Vision_AI_Module_V2\we2_image_gen_local\output_case1_sec_wlcsp\output.img and press enter to burn. alt text
  • Step 6: After burning is compelete, press any key to be continue. alt text
  • Step 7: Then, you will see the message "Do you want to end file transmission and reboot system? (y)" is displayed. Press button y to restart. alt text
  • Step 8: You will see the uart on minicom which is runing your algorithm. alt text

Back to Outline

Flash Image Update at Windows Environment

Following steps update application in the flash.

  • Step 1: Open TeraTerm and select File -> New connection, connect to Grove Vision AI Module V2. alt text
  • Step 2: Hold down any key on the keyboard (except the Enter key) and press the reset button to reset Grove Vision AI Module V2 and the startup options will be displayed. alt text
  • Step 3: Press button “1” and Grove Vision AI Module V2 will enter receiving mode after then. Select target flash image(output.img) by File->Transfer->XMODEM->Send. alt text
  • Step 4: After the firmware image burning is completed, the message "Do you want to end file transmission and reboot system? (y)" is displayed. Press button y to restart. alt text
  • Step 5: You will see the uart on TeraTerm which is runing your algorithm. alt text

Back to Outline

Flash using Edge Impulse CLI tools

This method works on all supported operating systems (Windows/Linux/MacOS...)

  • Step 1: Install the Edge Impulse CLI tools
  • Step 2: Open any system terminal and run the following command 
    himax-flash-tool -d WiseEye2 -f <path_to_four_firmware_img_file>
  • Step 3: Wait until you see the following message: 
    [HMX] Press **RESET** to start the application...
[HMX] Firmware update completed

Note: if the flashing process hangs, just cancel it (Ctrl+C) and start once again.

Back to Outline

How to restore to the original factory settings

Linux Environment

  • Update the flash image Seeed_SenseCraft_AI*.img to Grove Vision AI Module V2 and press reset buttun. alt text

  • Disconnect the Minicom:

    • Please press Ctrl+A on keyboard and press z on keyboard to go to the menu of minicom. alt text
    • Then, press q on keyboard to quit with no reset minicom, and press yes to leave. alt text

  • Open the permissions to acceess the deivce

    sudo setfacl -m u:[USERNAME]:rw /dev/ttyUSB0
# in my case
# sudo setfacl -m u:kris:rw /dev/ttyACM0

    alt text

  • After doing the above steps, you can run the SenseCraft AI on Grove Vision AI Module V2.

    1. Introduction : https://wiki.seeedstudio.com/grove_vision_ai_v2/
    2. Connect Grove Vision AI Module to NB USB port
    3. Open "Google Chrome" browser
    4. Open SenseCraft Homepage
    5. Select "Grove Vision AI(WE2)" and connect (serial port) alt text alt text alt text

Back to Outline

Windows Environment

  • Update the flash image Seeed_SenseCraft AI*.img to Grove Vision AI Module V2 and press reset buttun. alt text
  • Disconnect the TeraTerm. alt text
  • After doing the above steps, you can run the SenseCraft AI on Grove Vision AI Module V2.
    1. Introduction : https://wiki.seeedstudio.com/grove_vision_ai_v2/
    2. Install CH343 UART driver (CH343SER.ZIP) (Optional)
    3. Connect Grove Vision AI Module to NB USB port
    4. Open "Microsoft Edge" browser
    5. Open SenseCraft Homepage
    6. Select "Grove Vision AI(WE2)" and connect (serial port)

Back to Outline

How to add support for raspberry pi camera?

You can reference the scenario app allon_sensor_tflm , allon_sensor_tflm_freertos and tflm_fd_fm. Take allon_sensor_tflm for example, you should only modify the allon_sensor_tflm.mk from cis_ov5647 to cis_imx219 or cis_imx477.

#CIS_SUPPORT_INAPP_MODEL = cis_ov5647
CIS_SUPPORT_INAPP_MODEL = cis_imx219
#CIS_SUPPORT_INAPP_MODEL = cis_imx477

So that, it can support cis_imx219 or cis_imx477 camera.

Back to Outline

How to use CMSIS-NN at the project?

  • Modify the setting at the makefile
    • Enable the flag LIB_CMSIS_NN_ENALBE to build CMSIS-NN library 
      LIB_CMSIS_NN_ENALBE = 1
    • You can reference the scenario app example about allon_sensor_tflm_cmsis_nn which is the example running the model without passing vela and using the CMSIS-NN library.
      • Change the APP_TYPE to allon_sensor_tflm_cmsis_nn at the makefile 
        APP_TYPE = allon_sensor_tflm_cmsis_nn

Back to Outline