System_approach_A_paradigm_for_robotic_t
System_approach_A_paradigm_for_robotic_t
Sensing
Ravinder S. Dahiya1, 2, Maurizio Valle2, Giorgio Metta1, 2
1
Italian Institute of Technology, Genoa, Italy
2
University of Genoa, Genoa, Italy
E-mail: [email protected]
Abstract-In the pursuit of developing touch sensors or tactile Optical, Ultrasonic, Magnetic, and Piezoelectric. A range of
sensing arrays, the emphasis has only been on the sensors. This sensors that can detect object shape, size, presence, position,
led to a large number of ‘bench top’ sensors, very few of which
have actually been used in robotic systems. And those that have forces and temperature have been reported in the reviews on
seen the actual use have almost invariably been used in static tactile sensing [3-5]. A very few examples of sensors that
contact point imaging rather than the active manipulation or could detect surface texture [7], hardness or consistency [8] are
exploration. Perhaps the lack of the system approach rendered reported. Most of the reported devices are either of the scalar
many of them unusable. In this work, we present the design of a single point contact variety (for intrinsic tactile sensing) or are
tactile sensing system taking into account not only the parameters
to be sensed but also the physical and operational constraints of linear or rectangular arrays of sensing elements (for extrinsic
robotic system. tactile sensing). The production of new designs and improved
configurations of tactile sensors still continues apace, but, the
I. INTRODUCTION touch sensor technology largely remains unsatisfactory for
robotics either because the developed sensors are single big
Like humans, the interaction behaviors of robot with
size touch elements and are too big to be used without
environment can be better understood by physically interacting
sacrificing the dexterity of robot, or because they are slow, or
with it. Without touching the objects, it would be difficult for a
fragile and also in some cases due to the digital nature of touch
robot to know their interaction behaviors which depend on how
sensors i.e. touch or no touch.
heavy and hard the object is when hold, how its surface feels
Clearly, the emphasis, ‘only’ on the sensor development has
when touched, how it deforms on contact and how it moves
resulted in a large number of ‘bench top’ sensors, very few of
when pushed etc. Availability of high performance video
which have actually been used in robotic systems. This is
cameras and the significant research in the area of computer
surprising, considering the long history of gripper design for
vision has made robot interaction with environment mainly
manipulative tasks. We believe that the lack of the system
through visual sensing techniques [1], which, at times can be
approach has rendered many of them unusable, despite their
misleading due to lack of direct physical interaction. Surely,
good design. It is evident from the fact that very few works on
some of the information about real world objects e.g. shapes of
tactile sensing has taken into account the system constraints,
objects etc. can be obtained from the vision cameras [2] and a
like those posed by other sensors or by robot controller etc. To
further detail can be obtained by moving them around the
the best of our knowledge, only [9] has reported the design of
object. But, moving the robot around the object is not always
tactile sensing system that also considers system constraints.
possible, as can happen with humans as well. And, even if it is
Overall system performance is dictated not only by the isolated
possible to move it around the object under observation, the
quality of the individual system elements, but also by how
presence of visual inaccuracies due to large distance between
system elements integrate to achieve a goal. As an example,
robot cameras and the object can make it difficult to explore or
the development of tactile sensing arrays for fingertips of a
manipulate a given object. Of course, such inaccuracies can be
humanoid robot should also take into account system level
reduced by keeping the cameras close to the object, or in other
issues like – availability of space on the fingertip (which
words, somewhere close to fingers on the robot hand (e.g. Eye-
decides size of the array), nature of signal going out of array
in-Hand Configuration) [1] – but not without paying the price
(analog/digital), position of analog sensor front end (on the
in terms of loss of dexterity.
same chip with array or on separate chip), sources of noise
Despite an important role, the lesser use of tactile sensing in
(there are many motors on robots), time response of the sensor
robotics, as compared to other sensory modalities e.g. vision
with respect to other sensors involved in the closed loop
and auditory sensing, could partly be attributed to the complex
control of the robot, division of functions to be performed
and distributed nature of tactile sensing and partly also to the
locally and centrally etc. Much work needs to be done at
non availability of satisfactory tactile sensors. In last more than
system level before artificial touch can be used in real world
two decades many new touch sensors have been reported [3-6];
environment. This will also serve as basis for the development
by exploring nearly all modes of transduction viz:
of practical and economic tactile sensing system in future.
Resistive/Piezoresistive, based on Tunnel Effect, Capacitive,
Inclusion of tactile arrays in the control loop of robot will help
The work presented in this paper has been supported -in part- by the
ROBOTCUB project (IST-2004-004370), funded by the European
Commission through the Unit E5 -Cognitive Systems.
Authorized licensed use limited to: IEEE Xplore. Downloaded on March 23, 2009 at 04:19 from IEEE Xplore. Restrictions apply.
in exploring deeper issues involved both in the exploration and
manipulation. This will not only advance research in robotics Control
Action Controller
but will also help in understanding the human interaction with (signal to
actuators)
environment through touch.
In this work, we present the design of a complete tactile
sensing system taking into account not only the parameters to High Level Computat ion
be sensed but also the physical and operational constraints of Construction of world model
robotic system. Although the work presented here mainly
refers to tactile sensing for parts like fingertips that require
high-density of touch sensors, it can also be applied to low- Sensor Fusion
density touch sensors areas like large skin applications. This
Vision Audio
paper is organized as follows. Section II presents an overview Touch Vision Comput ing
(including H/W
of the robot tactile sensing system. Section III discusses the intrinsic
sensors)
design constraints and specifications involved in acquiring the T ouch Audio
Functional Levels
the stringent system constraints. Conclusions are then drawn in T actile Data
section V. Selection
3) Data transmission.
4) Model construction of contact object.
5) Control action. Read Out
&Signal
Rea d Out
&Signal
Read Out
&Signal
Read Out
&Signal
Re ad Out
& Signa l
Analog Sensors
frontend
Condition Condition Condition Condition Condition
111
Authorized licensed use limited to: IEEE Xplore. Downloaded on March 23, 2009 at 04:19 from IEEE Xplore. Restrictions apply.
etc. In order to maintain a better signal to noise ratio, it is
desirable to keep the signal conditioning circuit close to the
transducers array (for example, if the transducer arrays are
placed on the distal phalange of robot fingers, then
conditioning circuits can be placed on the adjacent phalange, if
space constraint doesn’t allow the same to be placed on the
distal phalange). A System on Chip (SoC) approach would be
ideal in this case which is the also the goal of the work
presented here. The initial choice of transduction method and
conditioning circuit are important from system point of view,
as they set the bandwidth limits of data accessed by the higher
levels of the system.
The third level involves the transmission of collected
Fig. 2 Humanoid robot, ‘icub’ with the hand shown in inset.
information to higher levels through communication interface.
The desired operation speed, noise and number of wires put a Higher level computations are done at the seventh level to
constraint on the type of communication channel used for obtain the model/image of the environment (object in contact),
interaction with higher levels. The transmission of digital data based on the data obtained from earlier level (from
can be done either serially or by CAN bus. CAN bus is independent sensing modalities or fused data of different
generally a preferred choice due to high real-time capabilities, sensing modalities). This level doesn’t impose any major
fast transmission (up to 1 Mbit/s) and high transmission constraint on design of lower levels of tactile sensing system.
reliability. High transmission reliability makes CAN bus A dedicated computing hardware is required to perform the
preferred choice over wireless transmission also because of the functions of this level and those of earlier two level i.e. data
safety issues involved during robotic interaction with the selection, sensor fusion and model construction.
environment – even though wireless transmission would be an At the highest level, the control algorithms are implemented.
ideal solution as it helps in reducing wiring problems. For a reliable control of complex tasks, the tactile sensing
The fourth level involves the multiplexing of tactile data parameters like sensor density, resolution, speed and location
coming from different parts, for example, from different are particularly important. Thus, the final design of tactile
fingers during a typical manipulation/explorative task. sensor and associated electronics circuitry, is the result of
Due to large number of sensing elements, the data size also many trade-offs.
multiplies. Not all the data collected from various parts is Our approach for development of tactile sensing system is to
useful and hence useless data can be rejected. This is basically climb the hierarchical ladder of tactile sensing system from
the function of fifth level in the hierarchy of tactile sensing hardware intensive bottom. The work presented here is related
system shown in Fig.1. For example, a grasp may not involve to the three lowest levels of the functional diagram shown in
all the fingers and hence the data obtained from the fingers Fig. 1. For the lowest level, we have developed POSFET
other than those involved in the grasp can be rejected. This (Piezelectric Oxide Semiconductor Field Effect Transistor)
argument is also valid for certain patches on the tactile array on based tactile sensing arrays which are to be placed on the
a particular finger involved in grasping. Based on the task, fingertips of the humanoid robot, ‘icub’ [14], shown in Fig. 2.
involvement of a scheme for reading data from certain To perform the second and third level functions, we are
predetermined tactile sensor elements can be useful. This developing analog sensors frond end, digital core and
requires addressing of all the touch sensor elements; which is communication interface.
the reason why the data transfer in Fig. 1 is shown as
bidirectional.
III. TOWARDS IMPLEMENTATION
The next level is Sensor Fusion. At this level, the signals
from different kind of sensors are collected. In case of The first step towards implementation of tactile sensing
humanoid robot, these signals could be from touch sensors system is to fix the system requirements. The system
(both extrinsic and intrinsic), from vision sensors and from requirements presented below are divided here into two parts;
audio sensors. In humans, the interaction with environment those related to sensor and those related to conditioning
involves the statistical combination of sensory data from electronics.
different sensing modalities [11], for example, touch and A. Sensor Requirements
vision, as shown in Fig. 1. Some attempts of robot control In absence of any rigorous artificial tactile sensing theory
involving different sensing modalities has also been reported in that can help in specifying important system parameters such
past [12, 13]. Availability of fast and efficient vision and audio as sensor density, resolution, location, bandwidth etc. one can
sensors places a constraint on the speed with which tactile data turn to human tactile sensing to get some initial cues.
should be obtained, if the data from different sensing Following the information on human tactile sensing system,
modalities are involved in robot control. one can formulate some basic design features of artificial
112
Authorized licensed use limited to: IEEE Xplore. Downloaded on March 23, 2009 at 04:19 from IEEE Xplore. Restrictions apply.
tactile system for a general robotic system intended to be used Noise: Apart from many sources of noise in robot, the
in real world environment. Few such studies have been variation in temperature can be a source of noise in our case,
reported in literature [3-5], following which some design due to choice of piezoelectric polymer as transducer material.
factors for artificial tactile sensing are presented below: Total noise from the system puts a constraint on the resolution
of ADC and hence on the resolution of parameters to be
1) The distributed nature of receptors calls for using various measured. To get 1000:1 dynamic range of forces, a 10 bit
kinds of miniaturized sensors arranged in matrix. Number ADC is required.
of elements in the array may vary with its desired Cross talk: In order to measure the value of force at
physical location on the robot. addressed touch element, the read out circuitry must be
2) The spatial resolution of the array of sensors should be insensitive to parasitic due to touch elements in neighbouring
about 1-2 mm, which translates to an approximately 10 x rows and columns. In our earlier sensor design [16], a 25%
15 element grid on a fingertip-sized area. mechanical cross talk was observed. One reason for high cross
3) In general, the sensor should demonstrate high sensitivity talk was the presence of uniform metal layer on one side of
and broad dynamic range. Force sensitivity range of 0.01 polymer. The cross talk is expected to go down in the POSFET
– 10 N (~1g – 1 Kg) with a dynamic range of 1000:1 based tactile sensing array, as in this case the metal electrodes
would be satisfactory. have been patterned to be present over the touch element only.
4) It should be multifunctional i.e. in addition to the Any electrical cross talk (change in capacitance of polymer due
detection of forces, touch sensor should be able to detect to adjacent touch elements) can be reduced by grounding touch
other interaction behaviors like hardness, temperature etc. elements other than the one which is being read.
5) Linearity and low hysteresis are desired. Although non- Read out time: While interacting with environment, the
linearity can be dealt with through inverse compensation, image of contact object may be reconstructed if tactile sensing
the handling high hysteresis is difficult. Output from the array is scanned with some minimum frames per second. This
tactile sensor should be stable, monotonic and repeatable. must also take into account the read out time of other sensors
It is interesting to note that the human tactile sensing is and also the bandwidth of controller. For the 100 Hz
hysteric, nonlinear, time varying and slow. But, perhaps frequency of robot arm controller and 30 frames per second
the presence of large number of these ‘technologically reading of vision sensor, for example, assuming 100 frames per
poor’ biological receptors enables central nervous system second, the read out time for 5 x 5 array, at the 100 frames per
to extract useful information. second is 0.4 ms (= 1/ (5x5x100)). For very dense arrays, the
6) The artificial tactile sensor should be fast. This is read out time is very less and at times can be unrealistic. Thus,
particularly true, if the tactile sensor is part of the control the number of touch elements on array depends on the read out
loop. In general, for real time contact details, each touch circuitry.
element should have a response time lesser than 1 ms, or Some low level computations like temperature
a similar value related to the total number of elements. compensation, averaging etc. can be performed on the chip
7) In addition to above factors, the artificial tactile sensors itself. For example, if the temperature variation is high, then
should be robust and thus must be capable of voltage output of each touch element needs to be compensated
withstanding harsh conditions of temperature, humidity, (if the response due to force only, is desired). The system has
chemical stresses, electric field etc.
113
Authorized licensed use limited to: IEEE Xplore. Downloaded on March 23, 2009 at 04:19 from IEEE Xplore. Restrictions apply.
First PCB on distal phalange Second PCB on next phalange
5 x 5 POSFET based tactile sensing array ASIC
analog force Low noise
out amplifiers & Analog to Digital
signal Converter
temp.out
conditioning
10
taxel Serial
selection address bus Communication
Decoder Interface
25
5 Dedicated
Digital control serial line
bias current block
signal Current
temperature sensor reference
Connector in between
Fig. 3: System architecture (first phase) of tactile sensing system. The tentative location of different chips of the robot finger is also shown.
to perform a total of 5 x 5 x (2 address write +1 touch element ratio. As an example, with the extended gate approach used in
read + 1 temperature compensation) = 100 operations for each [15], the 8 x 8 tactile sensors array was scanned in around 50
measurement. Furthermore, the device may be required to store ms and thus the response bandwidth of 25 Hz was achieved.
the computed value for comparison, if required. Another With POSFET based touch sensors and whole system on chip
example of on chip operation could be the detection of slip (SOC), which is our final goal, the bandwidth can be pushed to
when robot picks an object. Such an operation requires a total >100 Hz, which is desired for involving touch sensing into
of 5 x 5 x (2 address write +1 touch element read) + Store the robotic arm control. As an example, for 5 address lines for 25
data to compare it with the touch element values obtained in (Ns) touch elements, 10 bits of data per touch element (Nd),
next frame. Change in values of touch elements along a line assuming POSFET response time as 50 µs (Tr), delay of 50 µs
will reflect the slip perpendicular to this line. Working at frame during addressing (Ta) and delay of 50 µs during transmission
rate of 100 frames per second, the resulting minimum system of data (Tt), the scanning frequency of entire array can be
clock frequency should be about 10 KHz. obtained by substituting the corresponding values in following
equation:
IV. SYSTEM ARCHITECTURE
Fs = 1/ (Ns*(Ta + Tr + Tt)) (1)
The architecture of lowest three levels of the tactile sensing
system is shown in Fig. 3. The total tasks are divided into three Thus, assuming number of data lines to be equal to the
parts: development of the POSFET based tactile sensing array; transmitted data bits, the scanning frequency is about 270 Hz
development of dedicated electronic circuitry and integration and the communication bandwidth is 67.5 Kbits/sec. It should
of whole system - in a single package (SIP) in the first phase be noted that with POSFET based touch elements and SOC
and on a single chip (SOC) in the second phase approach, the delay and response times are expected to less
The POSFET based tactile sensors arrays have been than those assumed in above example. In other words, the
designed for the fingertips of robot and they are now in scanning rate of array would be faster.
fabrication. The tactile sensing array, as shown in Fig. 3, For the feasibility and reliability of prototype
comprises of 5 x 5 POSFET (Piezoelectric Oxide implementation, the ad hoc System in Package (SIP), with
Semiconductor Field Effect Transistors) based tactile sensors dedicated chips - tactile sensing array, analog frontend, Digital
[16, 17], obtained depositing the piezoelectric polymer (PVDF- Core and Interface (all in one package), could be the starting
TrFE) film on the gate area of MOSFET. The charge of the point. The main functional blocks of the SIP, as shown in Fig.
piezoelectric polymer generated due to applied force, 3, are: a) POSFET based tactile sensing array; b) analog
modulates the charge in the induced channel of MOSFET, sensors frontend; c) digital core and serial communication
which is then converted into a voltage value by means of interface. To optimize the performance a dedicated
readout circuitry that can be embedded into the chip. While the implementation of the analog sensors frontend is mandatory.
piezoelectric polymer film as sensing element improves Thus, an Application Specific Integrated Circuit (ASIC) will
sensors time response; the tight coupling of sensing material be designed and manufactured for this purpose, taking into
(PVDF-TrFE) and electronics using MOS technology will account the system requirements given in Table I. The analog
improve force resolution, spatial resolution and signal to noise
114
Authorized licensed use limited to: IEEE Xplore. Downloaded on March 23, 2009 at 04:19 from IEEE Xplore. Restrictions apply.
sensors frontend will provide sensors, the necessary bias- ACKNOWLEDGMENT
voltage and currents to acquire the sensors signals with
We are thankful to Dr. Leandro Lorenzelli, FBK-IRST,
minimal noise (i.e. on chip filtering to remove out of band
Trento, Italy for the support on the development of tactile
noise components). And, if necessary, the signals are
sensing arrays.
amplified, to make the noise introduced by subsequent stages,
less critical. The signals are then converted to digital values.
Functions of digital core are to address touch sensors; to REFERENCES
extract and compress information from the tactile sensors [1] M. Saedan and M. H. Ang Jr, "3D Vision-Based Control On An
array; to compensate for non linear and pyroelectric effects Industrial Robot," presented at IASTED International Conference on
which may affect measurement; and to drive the analog Robotics and Applications, Florida, USA, 2001.
[2] M. J. Schlemmer, G. Biegelbauer, and M. Vincze, "Rethinking Robot
frontend control signals, etc. Moreover, the digital core Vision - Combining Shape and Appearance," International Journal of
manages the interface to the communication channel. The Advanced Robotic Systems, vol. 4, pp. 259-270, 2007.
prototype tactile sensing system will be interfaced either with [3] R. D. Howe and M. R. Cutkosky, "Dynamic Tactile Sensing: Perception
of Fine Surface Features with Stress Rate Sensing," IEEE Transactions
CAN bus or with a dedicated digital serial line (a digital On Robotics And Automation, vol. 9, pp. 140-151, 1993.
interface is almost mandatory to protect the sensor data from [4] M. H. Lee and H. R. Nicholls, "Tactile Sensing for Mechatronics - A
noise due to the fact that the robot controller cards can be State of the Art Survey," Mechatronics, vol. 9, pp. 1-31, 1999.
[5] P. Dario and D. de Rossi, "Tactile Sensors and Gripping Challenge,"
further away from the sensors). The bandwidth and IEEE Spectrum, vol. 22, pp. 46-52, 1985.
connectivity (constrained by the overall size of the fingertip) [6] J. Dargahi and S. Najarian, "Human Tactile Perception as a Standard for
with the electronics existing on the humanoid robotic hand/arm Artificial Tactile Sensing - A Review," International Journal of Medical
Robotics and Computer Assisted Surgery, vol. 1, pp. 23-35, 2004.
will be considered according to system requirements. [7] V. Maheshwari and R. F. Saraf, "High-Resolution Thin-Film Device to
A major breakthrough in robot tactile sensing would be Sense Texture by Touch," Science, vol. 312, pp. 1501-1504, 2006.
‘System on Chip’ (SOC) implementation of tactile sensing [8] M. Shikida, T. Shimizu, K. Sato, and K. Itoigawa, "Active Tactile Sensor
for Detecting Contact Force and Hardness of an Object," Sensors and
system. Presence of analog sensor front end, digital core and Actuators A, vol. 103, pp. 231-218, 2003.
communication interface along with tactile sensors array on [9] S. C. Jacobsen, I. D. McCammon, K. B. Biggers, and R. P. Phillips,
same chip is expected to improve (among others) the speed, "Design of Tactile Sensing Systems for Dextrous Manipulators," in IEEE
Control System Magazine, vol. 8, 1988, pp. 3-13.
bandwidth, signal to noise ratio (keeping in view many sources [10] J. M. Wolfe, K. R. Kluender, D. M. Levi, L. M. Bartoshuk, R. S. Herz, R.
of noise on humanoid), overall sensitivity, efficiency and L. Klatzky, and S. J. Lederman, Sensation and Perception. Sunderland,
robustness. Apart from these, with SOC approach, the problem Massachusetts USA: Sinauer Associates Inc., 2006.
[11] M. O. Ernst and M. S. Banks, "Humans integrate visual and haptic
of wiring complexity- a key robotics problem - can also be information in a statistically optimal fashion," Nature, vol. 415, pp. 429-
effectively dealt with. Thus, rather than an alternative solution, 433, 2002.
SOC is the requirement for the tactile sensing system. [12] G. Milighetti, T. Emter, and H. B. Kuntze, "Combined Visual-Acoustic
Grasping for Humanoid Robots," presented at IEEE International
Conference on Multisensor Fusion and Integration for Intelligent
Systems, Heidelberg, Germany, 2006.
V. CONCLUSION [13] P. K. Allen, A. Miller, P. Y. Oh, and B. Leibowitz, "Integration of
Vision, Force and Tactile Sensing for Grasping," International Journal of
The tactile sensing system of robot is presented. Instead of Intelligent Machines, vol. 4, pp. 129-149, 1999.
coming up with ‘Yet another touch sensor,’ only, the need to [14] http://www.robotcub.org/.
develop the tactile sensing system for robot has been presented. [15] E. S. Kolesar, C. S. Dyson, R. R. Reston, R. C. Fitch, D. G. Ford, and S.
D. Nelms, "Tactile Integrated Circuit Sensor Realized with a
The system requirements are outlined, based on which, the Piezoelectric Polymer," presented at 8th IEEE International Conference
system architecture is presented. The system architecture on Innovative Systems in Silicon, Austin, TX, USA, 1996.
(tactile sensing arrays + analog front end +digital core+ [16] R. S. Dahiya, M. Valle, G. Metta, L. Lorenzelli, and C. Collini, "Tactile
Sensor Arrays for Humanoid Robot," presented at IEEE PRIME'07, The
communication interface) will be implemented by SIP (System 3rd International Conference on PhD Research in Microelectronics and
in Package) approach, in the first phase and by SOC (System Electronics, Bordeaux, France, July 2007.
on Chip) in the second phase. SIP approach is preferred in first [17] R. S. Dahiya, M. Valle, G. Metta, and L. Lorenzelli, "POSFET Based
Tactile Sensor Arrays," presented at IEEE ICECS'07, The 14th
phase, to study the feasibility and reliability of system, as SIP International Conference on Electronics, Circuits and Systems,
allows simpler designs, easy design verification, processes Marrakech, Morocco, Dec 2007.
with minimal mask steps and the use of optimized technologies
for different functions. This will also provide an opportunity to
compare the SIP and SOC approaches, in terms of cost and
performance improvement for this application.
115
Authorized licensed use limited to: IEEE Xplore. Downloaded on March 23, 2009 at 04:19 from IEEE Xplore. Restrictions apply.