Sensitivity Analysis for Building Adaptive Robotic Software
0 likes338 views
The document discusses a sensitivity analysis approach for optimizing adaptive robotic software by utilizing transfer learning to enhance performance based on configurable parameters. It highlights methods for runtime adaptation and performance debugging to improve robotic control under varying conditions. The findings emphasize the value of combining inexpensive simulated data with limited expensive real-world data to produce better predictive models.
![SENSITIVITY ANALYSIS FOR
BUILDING ADAPTIVE ROBOTIC SOFTWARE
Pooyan Jamshidi, Miguel Velez and Christian Kästner
INTENT DISCOVERY: SENSITIVITY ANALYSIS FOR CONFIGURATION OPTIMIZATION
REDUCING COSTS WITH TRANSFER LEARNING
USE CASES
Systematic System Evolution
To automate or guide intelligent design choices.
Runtime Adaptation
To enable runtime adaptation of software configurations
to maintain quality of performance under dynamic
conditions (changing environment, goals, and tasks).
Performance Debugging
To guide robot software developers to identify potential
bugs causing low quality of performance.
RESULTS
Motivation:
Robotic software expose configurable parameters.
These tunable parameters affect performance of robots.
This can be leveraged to optimize performance.
Source Response Target Response
Transfer learning combines:
Lots of data gathered cheaply from
the simulator
With much less data gathered
expensively from the target robot
To make better predictions overall
PUBLICATIONS
P. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer
learning for improving model predictions in highly configurable software. Int’l
Symp. Software Engineering for Adaptive and Self-Managing Systems
(SEAMS), 2017.
P. Kawthekar and C. Kästner. Sensitivity analysis for building evolving &
adaptive robotic software, Workshop on Autonomous Mobile Service Robots
(WSF), 2016.
Predictive
Model
Learn
Model
Measure
Measure
Data
Source
Target
Simulator (Gazebo) Robot (TurtleBot)
Predict
Performance
Predictions
Adaptation
Use for
analysis
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
0
5
10
15
20
25
CPU usage [%] CPU usage [%]
(a) (b)
(c) (d)
Prediction without transfer learning
5 10 15 20 25
5
10
15
20
25
10
15
20
25
Prediction with transfer learning
Using only a few real data points to
predict yields poor results across
configuration space
Using transfer learning to combine the
few real data points with lots of
approximate data yields a good model
Machine
Learning
Configuration
Parameters
Design of
Experiment
Configuration
Space
Predictive
Model
Sensitivity
Analysis
DataMeasurem
ents
Configuration
Space
Data
Accuracy
Energy
CPU
0 5 10 15 20 25 30 35
mean CPU utilization
0
500
1000
1500
2000
2500
3000
3500
numberofconfigurations
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
6
8
10
12
14
16
18
20
22
24
26
5 10 15 20 25
number of particles
5
10
15
20
25
numberofrefinements
5
10
15
20
25
30
35
40
45
CPU
Localisation Error](https://image.slidesharecdn.com/poster-170321123313/85/Sensitivity-Analysis-for-Building-Adaptive-Robotic-Software-1-320.jpg)
Sensitivity Analysis for Building Adaptive Robotic Software
- 1. SENSITIVITY ANALYSIS FOR BUILDING ADAPTIVE ROBOTIC SOFTWARE Pooyan Jamshidi, Miguel Velez and Christian Kästner INTENT DISCOVERY: SENSITIVITY ANALYSIS FOR CONFIGURATION OPTIMIZATION REDUCING COSTS WITH TRANSFER LEARNING USE CASES Systematic System Evolution To automate or guide intelligent design choices. Runtime Adaptation To enable runtime adaptation of software configurations to maintain quality of performance under dynamic conditions (changing environment, goals, and tasks). Performance Debugging To guide robot software developers to identify potential bugs causing low quality of performance. RESULTS Motivation: Robotic software expose configurable parameters. These tunable parameters affect performance of robots. This can be leveraged to optimize performance. Source Response Target Response Transfer learning combines: Lots of data gathered cheaply from the simulator With much less data gathered expensively from the target robot To make better predictions overall PUBLICATIONS P. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar. Transfer learning for improving model predictions in highly configurable software. Int’l Symp. Software Engineering for Adaptive and Self-Managing Systems (SEAMS), 2017. P. Kawthekar and C. Kästner. Sensitivity analysis for building evolving & adaptive robotic software, Workshop on Autonomous Mobile Service Robots (WSF), 2016. Predictive Model Learn Model Measure Measure Data Source Target Simulator (Gazebo) Robot (TurtleBot) Predict Performance Predictions Adaptation Use for analysis 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 0 5 10 15 20 25 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 0 5 10 15 20 25 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 0 5 10 15 20 25 CPU usage [%] CPU usage [%] (a) (b) (c) (d) Prediction without transfer learning 5 10 15 20 25 5 10 15 20 25 10 15 20 25 Prediction with transfer learning Using only a few real data points to predict yields poor results across configuration space Using transfer learning to combine the few real data points with lots of approximate data yields a good model Machine Learning Configuration Parameters Design of Experiment Configuration Space Predictive Model Sensitivity Analysis DataMeasurem ents Configuration Space Data Accuracy Energy CPU 0 5 10 15 20 25 30 35 mean CPU utilization 0 500 1000 1500 2000 2500 3000 3500 numberofconfigurations 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 6 8 10 12 14 16 18 20 22 24 26 5 10 15 20 25 number of particles 5 10 15 20 25 numberofrefinements 5 10 15 20 25 30 35 40 45 CPU Localisation Error