ME3220: Design and Manufacturing II

Sensors and Sensor Algorithms

Peter B. Shull
School of Mechanical Engineering
Shanghai Jiao Tong University
 Outline

1. Sensors

2. Sensor Fusion

2
 What is a Transducer?
• Transducer: Device that converts a signal in one
form of energy into another form of energy

Microphone:

Speaker:

3
 What is a Transducer?
• Examples
– Antenna: electromagnetic waves to electrical
– Load cell: force to electrical
– Potentiometer: position to electrical
– Accelerometer: acceleration to electrical signals
– Ultrasonic transceiver: sound to electrical signals
– LED: electrical to light
– Photoresistor: light to electrical
– Thermocouple: heat to electrical
– Resistor: electrical to heat

4
 What is a Sensor?

• Sensor: A transducer that converts a physical

stimulus to a output measurement
– Input could be: heat, light, sound, pressure,
magnetism, movement, etc.
– Output is usually electrical

5
Sensor Examples

6
 Important Sensors for Measuring
Human Movement
• Accelerometer
• Gyroscope
• Compass/Magnetometer
• GPS
• Ultrasound (Proximity Sensor)
• Optical Encoder
• Magnetoresistive Angle Sensor
• Potentiometer

7
 Accelerometer

• Acceleration: rate of change of velocity

– Either the direction or the speed can change

• Newton’s 2nd Law:

8
 Accelerometer
so, can measure force and deduce
acceleration

9
 Types of Accelerometers

• Most Common
– Capacitive
– Piezoelectric

• Less Common
– Piezoresistive
– Hall effect
– Heat transfer

10
 Capacitive Accelerometer
• Basic Idea: Spring-mass system with moving
plates causes a change in capacitance

Input: mass motion (M)

Output: Vout

11
 Capacitive Accelerometer
• Single capacitor
– Capacitance function
of gap or area
– Typically nonlinear

• Differential capacitor
– One capacitor
increases, the other
decreases
– Typically linear
– More common

12
Multiple Differential Electrodes

13
Multiple Differential Electrodes

Images from Analog Devices

14
 Piezoelectric Accelerometer
• Piezoelectricity: electric charge that accumulates
in certain solid materials (some crystals and
ceramics) in response to mechanical stress

15
 Piezoelectric Accelerometer
• Piezoelectricity: electric charge that accumulates
in certain solid materials (some crystals and
ceramics) in response to mechanical stress

16
 Piezoelectric Accelerometer
• Usable region produces linear output

17
 How to Make an Accelerometer?

• MEMS: Micro-Electro-Mechanical Systems

– Very small components (1 to 100 micrometers)

18
 Watch Video

How to Make an Accelerometer

19
 Other Types of Accelerometers

• Piezoresistive (or strain gauge accelerometers)

– Measure electrical resistance when mechanical stress
is applied
– Can measure accelerations down to 0 Hz
• Hall Effect
– Measure voltage variations from changing magnetic
fields
• Heat Transfer
– Measure internal changes in heat gradients due to
acceleration
– Thermoresistors on 4 sides of a heat source

20
 Accelerometer Problem: Noise
• If you want to get position, you have to integrate
twice!!

𝑣 = න 𝑎𝑑𝑡 𝑥 = න 𝑣𝑑𝑡

a: acceleration v: velocity
v: velocity x: position

21
 Gyroscope
• Many Applications

22
 How Does a Gyroscope Work?
• Gyroscope: Measures orientation based on
principles of angular momentum
– Mechanical gyroscopes typically comprise a spinning
wheel or disc with two gimbals
– Two gimbals allow the flywheel (rotor) to point in any
direction

23
 How Does a Gyroscope Work?

• Spin axis is always spinning

• Input axis is in direction of physical input movement
• Output axis is in direction of measurement output

24
 MEMS Gyroscope
• Uses the Coriolis Effect at a micro scale
• Two Main types
– Resonating Mass: mass resonates back and forth, the
amount of displacement increases as the rotation
speed increases
– Vibration Gyroscope: Sense angular velocity from the
Coriolis force applied to a vibrating object

25
 Resonating Mass Gyroscope
• Coriolis acceleration
– Moving outward on a spinning disk causes tangential acceleration
or an increase in tangential velocity
– A person moving northward toward the outer edge of a rotating
platform (below) must increase the westward speed component
(blue arrows) to maintain a northbound course. The acceleration
required is the Coriolis acceleration.

www.analog.com/library/analogdialogue/archives/37-03/gyro.html
26
 Resonating Mass Gyroscope
• When a resonating mass moves toward the outer edge
of the rotation, it is accelerated to the right and exerts on
the frame a reaction force to the left (orange arrow).
• When it moves toward the center of the rotation, it exerts a
force to the right (orange arrow).

www.analog.com/library/analogdialogue/archives/37-03/gyro.html
27
 Resonating Mass Gyroscope
• To measure the Coriolis acceleration, the frame
containing the resonating mass is tethered to the
substrate by springs at 90° relative to the resonating
motion.
• Coriolis sense fingers are used to capacitively sense
displacement of the frame in response to the force exerted
by the mass.

www.analog.com/library/analogdialogue/archives/37-03/gyro.html
28
 Resonating Mass Gyroscope
• The frame and resonating mass are displaced laterally in
response to the Coriolis effect.
• The displacement is determined from the change in
capacitance between the Coriolis sense fingers on the
frame and those attached to the substrate.

www.analog.com/library/analogdialogue/archives/37-03/gyro.html
29
 Vibration Gyroscope
• Uses piezoelectric effect

– Example materials:

www5.epsondevice.com/en/sensing_device/gyroportal/about.html
30
 Vibration Gyroscope
• How it works:

www5.epsondevice.com/en/sensing_device/gyroportal/about.html
31
 Watch Video

• How a Vibrating Gyroscope Works

32
 Gyroscope Problem: Drift
• Drift varies by size/type of gyroscope and
application

www5.epsondevice.com/en/sensing_device/gyroportal/about.html
33
 Magnetometer
• Chinese invented
1st compass in
206 BC
– Measures the
earth’s magnetic
field
– Magnetic and
Geographic North
Pole 11.5°
different

34
 Basic Idea – Faraday’s Law

• Faraday’s Law (Electromagnetism): Changing

magnetic field causes electrons to flow (current)

35
 Hall Effect Magnetometer

• Hall effect: when a current-carrying conductor is placed in

a magnetic field, a voltage will be generated perpendicular
to both the current flow and the field

No Magnetic Field Magnetic Field (B)

sensing.honeywell.com/index.php?ci_id=47847
36
 Hall Effect Magnetometer

• Signal is small and needs to be amplified

sensing.honeywell.com/index.php?ci_id=47847
37
 Hall Effect Magnetometer

• Need to use in the linear region

– Saturation occurs in the amplifier, not the Hall Element, so large
magnetic fields will not damage sensors but will saturate them

sensing.honeywell.com/index.php?ci_id=47847
38
 Example: AK8975
• Asahi (Japan)
• 3-Axis magnetometer
• Low power consumption: 350 uA
• Small: 4 mm x 4 mm x 0.75 mm

www.akm.com/akm/en/file/datasheet/AK8975.pdf
39
 Types of Magnetometers

• Hall Effect
• Magnetoresistive
• Magnetostrictive
• Magnetic Tunneling
• Proton
• Overhauser
• Optically Pumped
• Superconducting Quatum Interference
• Lorentz Force
www.engineersgarage.com/articles/magnetometer
www2.electronicproducts.com
40
 Magnetometer Problem:
Unwanted Magnetic Fields
• Power lines
• Computers
• Transformers
• Electrical Wiring
• Fluorescent
Lights
• Microwave Ovens
• Cell phones
• Electric Razors
• Etc…

41
 Watch Video: Sensors on a Phone

• AndroidDevices_SENSORS_DavidSachs2010

42
 Problems

• Accelerometer Problem: Noise

• Gyroscope Problem: Drift

• Magnetometer Problem: Unwanted Magnetic Fields

• Solution: Combine sensors together

– Use strength of each sensor
– Need sensor fusion algorithms

43
 IMU
• Accelerometer + Gyroscope
– IMU: Inertial Measurement Unit
– Very common to combine these
– Come in all different shapes and sizes

44
 MARG
• Accelerometer + Gyroscope + Magnetometer
– MARG: Margnetic Angular Rate and Gravitational
Sensor
– Not as common as IMU but growing in popularity
– IMU: more commercial devices, MARG: more lab
prototypes
– *Sometimes MARG is also referred to as IMU

45
 Outline

1. Sensors

2. Sensor Fusion

46
 Why Do We Need Sensor Fusion?

• When one sensor is not accurate enough to

measure the variable of interest
– When is a sensor not accurate enough?
• Sensor too noisy
• Sensor drops some data
• Sensor update rate is too slow
• One sensor is not sufficient to measure the variable of interest

• Example: measuring knee flexion angle

– When would you need sensor fusion?
– When would you not need sensor fusion?

47
 Watch Video

• SensorFusionBasics_StevanDobrasevic2013

48
 Example: Oculus Rift
• Wearable head-mounted display for Virtual Reality

49
 Oculus Rift
• Question: How do you estimate tilt axis of the
head?
• Answer: Sensor fusion with an accelerometer +
gyroscope

www.oculus.com/blog/sensor-fusion-keeping-it-simple/
50
 Tilt Compensation
Sensor Fusion Algorithm
Use accelerometer measurement to estimate the gravity
vector and assume it is correct if the following conditions are
met:
1. Magnitude of acceleration vector is close to 9.81 𝑚Τ𝑠 2
2. Gyroscope rotation measurements are small

www.oculus.com/blog/sensor-fusion-keeping-it-simple/
51
 Tilt Compensation
Sensor Fusion Algorithm
To correct error between estimated and actual tilt
1. Slowly adjust (imperceptible) while head is stationary or
moving slowly
2. Quickly adjust (imperceptible) while head is moving quickly

www.oculus.com/blog/sensor-fusion-keeping-it-simple/
52
 Types of Sensor Fusion

• Complementary Fusion

• Cooperative Fusion

• Redundant Fusion

53
 Complementary Fusion

Complementary Fusion
Sensor 1 Sensor 2
Type 1 Type 2
Cell phones Oculus Rift

Final
Result

Quadrotors Autonomous Cars

54
 Cooperation Fusion

Cooperation Fusion
Sensor 1 Sensor 2
Type 1 Type 1

Final
Result Motion Capture

Satellite Positioning (GPS)

55
 Redundant Fusion

Redundant Fusion
Money Counting
Sensor 1 Machine
Type 1

Final
Result

Sensor 2 Compare
Type 1 Results
Ballot Counting
Machine
Final
Result

56
 Types of Sensor Fusion

Complementary Fusion Cooperation Fusion Redundant Fusion

Sensor 1 Sensor 2 Sensor 1 Sensor 2 Sensor 1
Type 1 Type 2 Type 1 Type 1 Type 1

Final
Result
Final Final
Result Result

Sensor 2 Compare
Type 1 Results
When people say “Sensor Fusion”
they usually mean Complementary Final
Sensory Fusion Result

57
 Watch Video

• SENSOR-FUSION_DavidSachs2010.mp4

58
 Sensor Fusion Algorithms
• Probabilistic, Bayesian Recursive Algorithms
– Bayesian Networks
– Kalman Filter
– Extended Kalman Filter
– Particle Filter (i.e. Sequential Monte Carlo or bootstrap
filter)

• Other
– Interval Calculus
– Fuzzy Logic
– Evidential Reasoning (i.e. Dempster-Shafer theory)
– Central limit theorem
59
 Hidden Markov Model
• Basis of Bayesian Recursive Algorithms
– Goal: Estimate Actual System States and/or Actual
System Model from Measured System States and
Assumed System Model

Actual System States

(Unmeasurable): Hidden

Actual System Model

(Unmeasurable): Hidden

Measured System States

60
 Bayesian Networks
prior (prior probability
• Conditional Probability distribution)

P(𝐴 ∩ 𝐵)
𝑃 𝐴𝐵 =
𝑃(𝐵)
𝐏(𝐀) 𝐏(𝐀 ∩ 𝐁) 𝐏(𝐁)
P(𝐴 ∩ 𝐵)
𝑃 𝐵𝐴 =
𝑃(𝐴)

𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 𝐵 𝑃 𝐵 = 𝑃(𝐵|𝐴)

𝑃 𝐵 𝐴 𝑃(𝐴)
Bayes' Theorem! 𝑃 𝐴|𝐵 =
𝑃(𝐵)
61
 Bayesian Networks

• Definition: A probabilistic graphical model that

represents a set of random variables and their
conditional dependencies via a directed acyclic
graph
– directed acyclic graph: there is
no way to cycle back to a starting
vertex
• E.g. graph on right, starting at vertex
11, could never return to vertex 11

62
 Bayesian Network Example
• Conditional Probability Tables
𝑆, 𝑆ҧ 𝑅, 𝑅ത

𝐺, 𝐺ҧ

Conditional
Probability Tables

63
 Bayesian Network Example
• What is the probability that it’s raining?
𝑆, 𝑆ҧ 𝑅, 𝑅ത

𝑃 𝑅 =?

𝑃 𝑅 = 0.2 𝐺, 𝐺ҧ

64
 Bayesian Network Example
• If it’s raining, what is the probability that the
sprinkler is on?
𝑆, 𝑆ҧ 𝑅, 𝑅ത

𝑃 𝑆|𝑅 = ?

𝑃 𝑆|𝑅 = 𝟎. 𝟎𝟏 𝐺, 𝐺ҧ

65
 Bayesian Network Example
• If the sprinkler is on, what is the probability that
it’s raining?
𝑆, 𝑆ҧ 𝑅, 𝑅ത
𝑃 𝑅|𝑆 = ?

𝑃(𝑆 ∩ 𝑅) 𝑃(𝑆 ∩ 𝑅)
𝑃 𝑅|𝑆 = =
ത 𝐺, 𝐺ҧ
𝑃(𝑆) 𝑃 𝑆 ∩ 𝑅 + 𝑃(𝑆 ∩ 𝑅)
𝑃 𝑆 𝑅 𝑃(𝑅)
𝑃 𝑅|𝑆 =
𝑃 𝑆 𝑅 𝑃(𝑅) + 𝑃 𝑆 𝑅ത 𝑃(𝑅)
ത

(0.01)(0.2)
𝑃 𝑅|𝑆 = = 𝟎. 𝟎𝟎𝟔𝟐
0.01 0.2 + (0.4)(0.8)

66
 Kalman Filter

• History – Rudolf Kalman

– 1930: Born in Budapest, Hungary
– 1953: B.S. from MIT
– 1954: M.S. from MIT
– 1957: Ph.D. from Columbia University
– 1958, 1960, 1961: First published
papers describing the Kalman filter,
Swerling (1958), Kalman (1960),
Kalman and Bucy (1960)

67
 What is a Kalman Filter?
• Optimal estimator for linear system models with
additive independent white noise in the
measurement and model

• Optimal estimator
– minimizes mean square error of estimated variables

• Computes current state variables based on noisy

measurements

68
When Should I Use a Kalman Filter?

• Sensor measurements are noisy

• Sensor measurement noise is Gaussian (bell

curve)

• Sensor measurements occur at a constant rate

• You have a linear model of the system

69
 Kalman Filter Inputs/Outputs
• Input:
– Previous estimates
• don’t need any other previous or future estimates or
measurements
– Current measurements
– Underlying assumed linear system model

• Output:
– Current estimates in real-time

70
Kalman Filter – Simplified

71
Example 1: Voltage Measurement

http://greg.czerniak.info/guides/kalman1/
72
Example 2: Cannonball in Flight

http://greg.czerniak.info/guides/kalman1/
73
Kalman Filter Flow Chart

http://en.wikipedia.org/wiki/Kalman_filter
74
 Example 3: 1-DOF Cart

1) One degree of freedom cart with a noisy measurement and a

prediction estimate

Faragher, Understanding the basis of the Kalman filter… IEEE Signal Processing Magazine (2012): 128-132.

75
 Example 3: 1-DOF Cart

2) Initial conditions at t = 0. Red Gaussian distribution of probability

density function (pdf) for initial position estimate. Arrow represents
known initial velocity.

3) Prediction location at t = 1 and the red Gaussian pdf showing

uncertainty of the prediction; pdf has spread showing less confidence
because we are uncertain of cart accelerations from t = 0 to t = 1.

Faragher, Understanding the basis of the Kalman filter… IEEE Signal Processing Magazine (2012): 128-132.

76
 Example 3: 1-DOF Cart

4) Measurement of train location at t = 1. Uncertainty in the noisy

measurement represented by the blue Gaussian pdf.

5) New green pdf is calculated by multiplying the red (prediction) and

blue (measurement) pdfs for t = 1. New pdf provides the best
estimate of the cart location by fusing prediction and measurement
data.
Faragher, Understanding the basis of the Kalman filter… IEEE Signal Processing Magazine (2012): 128-132.

77
 Extended Kalman Filter
• Kalman Filter is for linear systems using linear
models (e.g. x = Ax + b)

• Most systems in real-life are non-linear

• Extended Kalman Filter works for non-linear

systems composed of differentiable functions
(e.g. y = cos(x))

𝑥𝑘 = 𝑓 𝑥𝑘−1 , 𝑢𝑘−1 + 𝑤𝑘−1

𝑧𝑘 = ℎ 𝑥𝑘 + 𝑣𝑘

78
 Extended Kalman Filter

• When the non-linear model is well-understood,

the Extended Kalman Filter is a widely used state
estimator (e.g. applications like navigation
systems and GPS)

• Disadvantages
– Doesn’t work well when the system model is inaccurate
– If the initial state is wrong or if the model is wrong, the
filter may diverge quickly
– Estimated covariance matrix tends to underestimate
the true covariance

79
 Particle Filter
• Also called Sequential Monte Carlo methods
• Bayesian recursive algorithm to estimate the
posterior density of the state variables
• Uses repeated sampling and simulation
approach

• Requirements
– System can be non-linear
– Noise can be non-Gaussian
– Initial state is unconstrained

80
 Particle Filter

1. Scatter 100s or 1000s of particles (hypotheses)

across all possible states
2. Use current measurements and change in
measurements from last time step to determine
the likelihood of each particle being correct
3. Weight more likely particles higher and less
likely particles lower
4. Resample particles with more near highly
probable states
5. Return to Step 2

81
 Particle Filter Applications

Autonomous driving vehicle navigation

SLAM (Simultaneous
Localization and Mapping)
for robot navigation

Image recognition
82
 Kalman Filter Particle Filter

Simplified Model Complex Model

Computationally Efficient Computationally Expensive

Exact Solution Approximate Solution

Linear Systems Non-Linear Systems

Gaussian Noise Any Kind of Noise

Initial Conditions Accuracy Important Initial Condition Accuracy Not Important

83
 Other Algorithms

• Interval Calculus
• Fuzzy Logic
• Evidential Reasoning (i.e. Dempster-Shafer
theory)
• Central limit theorem
• ….

84