uTensor - embedded devices and machine learning models

Lars Gregori, SAP CX
September 30, 2019
uTensor 
Embedded Devices And Machine Learning Models

© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ "2
SpeedAvailabilityPrivacy
Why?

• Energy is the limiting factor
• CPUs and sensors use almost no power
• radios and displays use lots
• We capture much more sensor data than we use
Why the Future of Deep Learning is Tiny
https://petewarden.com/2018/06/11/why-the-future-of-machine-learning-is-tiny/

“ML on tiny, cheap battery  
powered chips is coming  
and will open the door for  
some amazing new applications!”
-- Pete Warden

IoT Gateway
(credit: http://sappress.de/4795)

Machine Learning Models
uTensor + Code
Examples
Agenda

XOR
input_a input_b output
0 0 0
0 1 1
1 0 1
1 1 0

1. XOR model with Keras
2. XOR model with TensorFlow
3. MNIST model with TensorFlow
Machine Learning Models
https://github.com/choas/utensor_examples
MNIST:

TensorBoard - XOR model (Keras)

step 17000, training accuracy 0.94
test accuracy 0.9272
step 99000, training accuracy 1
test accuracy 0.9546
MNIST training (20,000 steps and 100,000 steps)
training steps don’t influence the model size
size:
• # layers
• # nodes
• value type (e.g. float, int)

uTensor
https://github.com/uTensor/uTensor

• Add
• ArgMax
• Dropout
• MatMal
• Max
• Min
• Placeholder
• Quantization Ops
• ReLu
• Reshape
uTensor - What is supported?

ARM Mbed OS 5
https://os.mbed.com/docs/mbed-os/v5.12/introduction/index.html

{
ctx.add(new RamTensor<int>(), "y_pred:0");
ctx.push(new ArgMaxOp<float, int>(),
{ "add_1:0", "y_pred/dimension:0" },
{ "y_pred:0" });
}
Code

const float inline_Variable_2_0 [ 20 ] = {
-0.07518502, -0.094865195, -2.2947302, 2.420463, -0.39691946,
0.2372103, 1.1672685, -1.1217928, -1.3336784, 1.2321934,  
-2.725729, 2.5416267, 0.8984613, -1.0757952, -0.064421736,  
-0.10880947, 0.57309926, -0.8162274, -0.01253005, -0.038079493
};
Code (weight and bias)

Elf2Bin: XOR
| Module | .text | .data | .bss |
|--------------------|-----------------|-------------|---------------|
| [fill] | 530 | 7 | 32 |
| [lib]/c.a | 69431 | 2548 | 127 |
| [lib]/gcc.a | 7456 | 0 | 0 |
| [lib]/m.a | 788 | 0 | 0 |
| [lib]/misc | 252 | 16 | 28 |
| [lib]/nosys.a | 32 | 0 | 0 |
| [lib]/stdc++.a | 173167 | 165 | 5676 |
| main.o | 1558 | 4 | 321 |
| mbed-os/cmsis | 1033 | 0 | 0 |
| mbed-os/components | 78 | 0 | 0 |
| mbed-os/drivers | 976 | 0 | 0 |
| mbed-os/hal | 1432 | 4 | 66 |
| mbed-os/platform | 4215 | 260 | 253 |
| mbed-os/rtos | 7243 | 168 | 5973 |
| mbed-os/targets | 6709 | 4 | 364 |
| models/xor_relu.o | 23342 | 4 | 1 |
| uTensor/uTensor | 6709 | 28 | 7 |
| Subtotals | 304951 | 3208 | 12848 |
Total Static RAM memory (data + bss): 16056 bytes
Total Flash memory (text + data): 308159 bytes
Binary XOR

Elf2Bin: MNIST_LCD
| Module | .text | .data | .bss |
|----------------------------------------|---------------|-------------|---------------|
| DISCO_L476VG_GlassLCD/BSP_DISCO_L476VG | 1960 | 0 | 76 |
| DISCO_L476VG_GlassLCD/LCD_DISCO_L476VG | 34 | 0 | 0 |
| [fill] | 553 | 3 | 42 |
| [lib]/c.a | 68359 | 2548 | 127 |
| [lib]/gcc.a | 7456 | 0 | 0 |
| [lib]/m.a | 788 | 0 | 0 |
| [lib]/misc | 252 | 16 | 28 |
| [lib]/nosys.a | 32 | 0 | 0 |
| [lib]/stdc++.a | 173167 | 165 | 5676 |
| main.o | 4424 | 4 | 30 |
| mbed-os/cmsis | 1033 | 0 | 84 |
| mbed-os/components | 156 | 4 | 4 |
| mbed-os/drivers | 692 | 4 | 100 |
| mbed-os/hal | 2734 | 8 | 247 |
| mbed-os/platform | 2021 | 256 | 136 |
| mbed-os/rtos | 7678 | 168 | 6289 |
| mbed-os/targets | 12882 | 8 | 913 |
| models/deep_mlp.o | 138837 | 4 | 1 |
| uTensor/uTensor | 6709 | 28 | 7 |
| Subtotals | 429767 | 3216 | 13760 |
Total Static RAM memory (data + bss): 16976 bytes
Total Flash memory (text + data): 432983 bytes
Binary MNIST_LCD

NUCLEO-F411RE 
https://os.mbed.com/platforms/ST-Nucleo-F411RE/
DISCO-L476VG 
https://os.mbed.com/platforms/ST-Discovery-L476VG/
DISCO-F413ZH 
https://os.mbed.com/platforms/ST-Discovery-F413H/
Hardware

uTensor is young and under going rapid development.  
Many exciting features are on the way:
• Convolution
• Pooling
• CMSIS-NN integration (https://arm-software.github.io/CMSIS_5/NN/html/index.html)
• Smaller binary
• More efficient Run-time
uTensor README.MD

https://twitter.com/zach_shelby/status/1126176019449147392

• Why
• Machine Learning Models
• uTensor
• C++ code
• TensorFlow Lite Micro
Summary

Contact information:
Lars Gregori
@choas
Thank you.

uTensor - embedded devices and machine learning models

Recommended

More Related Content

Similar to uTensor - embedded devices and machine learning models (20)

More from Lars Gregori (20)

Recently uploaded (20)

uTensor - embedded devices and machine learning models