Tiny machine learning

Arduino Education
June 23rd 2021
 University

Agenda

● What is Machine Learning

● Applications
● Tiny Machine learning
● Tiny Machine learning kit
● Examples
Machine Learning

● Day to day tasks like scrolling through social media, taking a picture, checking the weather, all depend on
machine learning models.

● Machine learning is a subset of AI.

● Machines take data and ‘learn’ for themselves.

● Machine learning systems can quickly apply knowledge and training from large datasets to excel at facial
recognition, speech recognition, object recognition, translation, and many other tasks.

● Machine learning allows a system to learn to recognize patterns on its own and make predictions,
contrary to hand-coding a software program with specific instructions to complete a task.

https://www.expert.ai/blog/machine-learning-definition/
 Machine Learning

● It started from games.

● Computers were trained to play games for

example checkers with humans.

● The term machine learning was coined in 1959 by

Arthur Samuel, pioneer in the field of computer
gaming and artificial intelligence.

● Blue an IBM computer beat world chess champion

Garry Kasparov in 1996.

Photograph: Stan Honda/AFP/Getty Images

4
Machine Learning applications for everyday life

● Fraud detection

● Online recommender systems

● Google search algorithms

● The self-driving cars

● Face recognition
https://www.cnet.com/news/

5
Why to learn ML?
● Predictive inventory ● Alerts & diagnosis from real ● Aircraft scheduling
planning time patient data ● Customer complaint resolution
● Recommendation engines ● Disease identification ● Traffic patterns
● Market segmentation & ● Proactive health ● Safety monitoring
targeting management ● Dynamic pricing

Retail Healthcare Travel &

transportation

● Risk analytics
● Power usage analytics ● Predictive maintenance
● Sales & marketing campaign
● Energy demand & supply ● Propensity to buy
management
administration ● Demand forecasting
● Customer segmentation
● Smart grid management ● Process optimization
● Fraud detection

Financial services Energy Manufacturing

6
Challenges of Machine Learning
Machine Learning allows a computer to internalize
concepts found in data to form predictions for new
situations.

Training these models are computationally expensive.

Running inference on these models are computationally

expensive as well.

We need computing systems that are fast enough to

handle it.

Most of these models run on huge data centres with

clusters of CPUs and GPUs

To reach reliable levels of accuracy, models require large Photo by Taylor Vick on Unsplash
datasets to ‘learn’ from.

7
Challenges of Machine Learning

When you take a picture, you want the machine learning magic to happen instantly.

In this case, you we would want the machine learning model to run locally.

When you say “Alexa” or “Ok, Google”, you want your devices to respond to instantly.

1. Energy: Efficiency issues / large data centers of computers

2. Privacy & data acquisition: Machine Learning requires massive data sets to train on, and
these should be inclusive/unbiased, and of good quality

3. Latency: Send the data up and down takes time

4. Reliability: What happen if the network is down

8
 University

Tiny machine learning

Tiny Machine Learning

Collaboration of machine learning and embedded ultra-low power internet of things devices that
explores the types of models you can run on small, low-powered devices like microcontrollers.

The philosophy of TinyML is doing

more on the device with less
Machine Tiny
resources – in smaller
Learning ML Microcontrollers
form-factors, less energy and
lower cost.

10
Advantages of Tiny Machine Learning
1. Low Power Consumption: Microcontrollers consume very little power. This enables them to run
without being charged for a really long time.

2. Low Latency: Since the model runs on the edge, the data doesn't have to be sent to a server to run
inference. This reduces the latency of the output.

3. Low Bandwidth: As the data doesn’t have to be sent to the server constantly, less internet bandwidth is
used.

4. Privacy: Since the model is running on the edge, your data is not stored in any servers
(privacy by design)

11
Tiny machine learning
Tiny ML started with Ok google.
Ok google it is a machine learning model that runs on all of our phones, does not run on a big data centre.
It runs in the single processing little chip that is always on.

How they do that?

TinyML is focused on inferencing, we want a response to an event coming in.
The training of this model, the million of voices and keywords to distinguish between the phrases,
happens somewhere in the cloud, but the final inference can happen on this small devices.

12
 What do we need?

1. Hardware:

Arduino Nano 33 BLE Sense

It provides enough power to run TinyML models. It has a colour, brightness,
proximity, gesture, motion, vibration, orientation, temperature, humidity, and
pressure sensors. It also contains a digital microphone and a Bluetooth low
energy(BLE) module.

2. Software - Framework
TensorFlow Lite is the most popular and has the most community support.
Using TensorFlow Lite Micro, we can deploy models on microcontrollers.

Arduino IDE: There is an arduino library available for TensorFlow Lite, on the
Nano 33 BLE Sense
It comes with 4 examples showing how to run speech, accelerometer and
image machine learning on a microcontroller.

3. Resources

Since TinyML is an emerging field, there aren’t many learning materials as of

today. Harvard University’s Course on TinyML by Vijay Janapa Reddi

13
Arduino content NOT included TINY Machine Learning KIT

● Arduino Nano 33 BLE Sense

that sense movement, acceleration, rotation,
temperature, humidity, barometric pressure,
sounds, gestures, proximity, color, and light
intensity.

● A camera module (OV7675)

● Custom Arduino shield to attach your

components and create unique TinyML projects.

● University students can use the kit to explore

practical ML use cases using classical algorithms
as well as deep neural networks powered by
TensorFlow Lite Micro.

14
 Arduino content NOT included
TINY Machine Learning KIT
Arduino Nano 33 BLE Sense
● Senses: movement,
COURSE 1 acceleration, rotation,
● Understand what machine learning (ML) Is temperature, humidity,
● Deep learning and embedded machine barometric pressure,
learning
● Understand neural networks sounds, gestures,
● Background responsibilities and Real proximity, color, and light
Examples intensity.
COURSE 2
● Train your device using your own datasets Custom Arduino shield to attach
● Using external training models for your your components
project
● Basics of AI, real user cases and scenarios
● Know and play with some of the most used
technics
● Real world industry applications

COURSE 3
● Hardware basics Camera module (OV7675)
● Code your own projects with TinyML
● Train your TinyML device
● Deploy, test, and correct your TinyML External content:
projects ● There is NO Arduino content for this kit
● Use the freely available content & Arduino IDE
courses from Harvard University Tensor Flow
at EdX Google Colab
15
edX: Tiny machine learning course

16
edX: Tiny machine learning course

17
18
 University

Tiny machine learning

Examples with Arduino

Tiny machine learning examples

Fruit identification using Arduino and TensorFlow Gesture classification

TensorFlow Lite Micro library and the Arduino Nano 33 Training your own gesture classification model, output as
BLE Sense’s colorimeter and proximity sensor to classify emojis
objects

20
Tiny machine workflow

Re-train Model

Raw data Prepare data Train model Evaluate model Deploy model

● Classification
Extract features ● Anomaly detection
● Forecasting

Data processing Model building Deployment and

Monitoring
21
 Gesture classification
The objective of the project is to create a virtual keyboard that turns a gesture into an emoji.

What you need: Nano Ble 33 sense, Arduino IDE, Tensorflow and Jupiter notebook.

-Open the Arduino IDE

-Go to Sketches/IMU_Capture/IMU_Capture.ino

-We will get data from the gyroscope

-Compile the sketch and upload to the board

-Open the serial monitor

22
Gesture classification

-We can also get the data in

from of a graph in real time

- if we use the serial plotter on

the Arduino IDE

23
Gesture classification

With the board in our hand we do the punch and flex

gesture 10 times

We copy that info and create 2 csv files: one for punch and
other for flex

Go to a Jupyter notebook, hosted on google colab

You can go through the code that does all the training and
also the snipet of code that takes the model generated by
tensorflow and gets the file converted into a “ .h” file for
Arduino.

There is a section on the notebook where you can drag

your two files. And then you get the model.h

24
Gesture classification

You go to the IDE, there is another

example called IMU classifier paste the
model.h files from google and you will
see this on the serial monitor which the
level of confidence of each gesture.

Practical use of this example:

Someone with disability could operate an

equipment with gestures

25
Tiny machine Learning example - Portenta

Creating a Basic Face Filter With OpenMV editor The Haar Cascade Algorithm
Algorithms can be trained to detect the desired type of
In this tutorial you will build a MicroPython application
object.
with OpenMV that uses the Portenta Vision Shield to
detect faces and overlay them with a custom bitmap
image.

26
Tiny Machine Learning
applications

Epilet
TinyML-powered bracelet for detecting
epileptic seizures

TapLock
Uses tinyML on Arduino to protect
your bike from thieves

PUPPI
A tiny, portable, edge ML device
ready to interpret a dog's mood
based on vocal signals.

27
 University

Arduino Tiny machine

learning

Coming soon
What do you need to get started?
● ML theory: Knowing the basics of ML theory will give you a foundation to build on,
and help you troubleshoot when something goes wrong.

● Coding skills: Building ML requires coding in order to do the data management,

parameter tuning, and parsing results needed to test and optimize your model.

● Math and stats: ML is a math heavy discipline, so if you plan to modify ML models
or build new ones from scratch, familiarity with the underlying math concepts is
crucial to the process.

Build your own projects: Getting hands on experience with ML is the best way to
put the concepts learned to the test.

29
Arduino Education

Enabling anyone to innovate by making

complex technologies simple
to use.

30
Arduino Tiny Machine Learning Kit

Research:
-Interview with teachers, experts on the field.
-Needs, content, learning approach.

-University students / High School 17+

-Arduino Content & libraries to learn the core concepts of machine learning
-Software
-Hardware
-Step by step activities and projects

31
32
 University

Thank you!