/UNIFEI-IESTI01-TinyML

Primary LanguageJupyter NotebookGNU General Public License v3.0GPL-3.0

UNIFEI-IESTI01-TinyML

TinyML - Machine Learning for Embedding Devices

UPDATED to 2024 - 1st Semester

Instituto de Engenharia de Sistemas e Tecnologias da Informação – IESTI - Campus de Itajubá

For the previous versions of IESTI01 courses, please visit:

Material

  • All materials were uploaded to this repo at the end of last semester's course
    • Slides, Notebooks, Code, and Docs in English
    • Videos in Portuguese

Optional pre-course activities:

Parte 1: Fundamentals

Parte 2: Applications & Deploy

Relevance of TinyML Learning

Microcontrollers (MCUs) are affordable electronic components, usually with only a few kilobytes of RAM, and designed to consume minimal power. MCUs are embedded in nearly all residential, medical, automotive, or industrial devices. It's estimated that over 40 billion microcontrollers are sold annually, and probably hundreds of billions are currently in service. However, interestingly, these devices often need to receive the proper attention, as they are frequently used merely as replacements for the functionalities that older electromechanical systems used to perform in cars, washing machines, or remote controls.

More recently, with the advent of the Internet of Things (IoT), a significant portion of these MCUs has started generating "quintillions" of data, much of which goes unused due to the high cost and complexity of transmission (bandwidth and latency).

On the other hand, in the past few decades, we've witnessed the development of Machine Learning models (a sub-field of Artificial Intelligence) trained on large datasets using powerful mainframes. But what's happening now is that suddenly, it has become possible to extract "meaning" from noisy and complex data like images, audio, or vibrations, using neural networks. And importantly, we can run these neural network models on microcontrollers and sensors, using very little power to interpret much more of the sensor data we typically overlook. This is TinyML, a new field that enables extracting "machine intelligence" right in the physical world (where the data is generated).

Course Overview

TinyML is an introductory course at the intersection of Machine Learning and Embedded Devices. The proliferation of embedded devices with ultra-low power consumption (on the order of milliwatts), coupled with the introduction of machine learning frameworks dedicated to embedded devices, such as TensorFlow Lite for Microcontrollers (TFLite Micro or TFLM), enables the widespread adoption of AI-powered IoT devices, known as "AioT."

Furthermore, the explosive growth of machine learning and the user-friendliness of platforms like TensorFlow (TF) make it an essential subject of study for engineering students, especially in Electronics, Computer Science, and Control & Automation.

TinyML differs from conventional machine learning (e.g., server-cloud-based) because it requires software knowledge and expertise in embedded hardware. This course aims to provide a foundation for understanding this emerging field.

References

The current version of this pioneering course in Latin America is primarily based on:

The IESTI01 course is part of the TinyML4D, an initiative to make TinyML education available globally.

Course Topics

  1. Fundamentals of IoT
  2. Fundamentals of Machine Learning (ML)
  3. Fundamentals of Deep Learning (DL)
  4. Collecting Data for ML
  5. Training and Deploying ML Models
  6. Basics of Embedded ML
  7. Code behind some of the most widely used applications in TinyML
  8. Real-world applications of TinyML in the industry
  9. Principles of Automatic Speech Recognition (Keyword Spotting apps like Alexa, Hey Google, Siri, etc.)
  10. Principles of Automatic Image Classification (Visual Wake Words)
  11. Anomaly Detection Concepts and Applicable ML Models
  12. Principles of Data Engineering applied to TinyML
  13. Overview of microcontroller-based device hardware
  14. Overview of software behind microcontroller-based devices
  15. Real projects using market platforms
  16. Design, Development, and Deployment of Responsible AI

Course Load:

  • 30 hours (Recorded videos)
  • 15 hours de assignments/labs
  • 15 hours of individual research and study + final group project

Approval Process:

  • Individual Quiz Responses: 20%
  • Code Assignments (ML/DL) (Jupyter Notebook / CoLab): 25%
  • Practical Projects (Laboratory Reports): 25%
  • Final Group Project (with presentation): 30%

Prerequisites:

  • Proficiency in the English language (for reading).
  • Basic programming knowledge in C/C++ (Arduino IDE) and Python. All class tasks will involve one or both programming languages.
  • Familiarity with command-line tools on Mac, Windows, or Linux. Projects will require some operations in a terminal.
  • Understanding linear algebra, signal analysis, basic probability, and statistics. Many ML topics revolve around grasping vector and matrix operations, notation, and concepts like Gaussian distributions, means, standard deviations, etc.

Methodology/Resources:

  • Students will be able to take classes sequentially at their convenience.
  • Exercises and projects can be developed on personal computers with TensorFlow v2.x installed or by utilizing online tools like Google Colab (desirable).
  • Real TinyML models will be developed and trained using the Edge Impulse Studio.
  • For initial data capture, deployment of trained models, and familiarization with market tools, personal smartphones equipped with at least an accelerometer, microphone, and camera sensors will be used.
  • In the second part of the course, an Arduino BLE Sense (Cortex-M4) Kit and a digital camera model OV7675 will be used. (Each student will receive a Kit the university provides for personal use during the course.)

Professor:

Supervision and support: