1998 IEEE InternationalConference on Robotics and Automat ion

The Roles of Shape and Motion in Dynamic Manipulation:

The Butterfly Example
Kevin M. Lynch Naoji Shiromay Hirohiko Araiz Kazuo Taniez
Mechanical Engineering Department yInstitute of Engineering Mechanics zBiorobotics Division
Northwestern University University of Tsukuba Mechanical Engineering Laboratory
Evanston, IL 60208 USA 1-1 Tennodai, Tsukuba, 305 Japan Namiki 1-2, Tsukuba, 305 Japan

Abstract
We are studying a juggler’s skill called the “butterfly.”
Starting with a ball resting on the palm of his/her open hand,
a skilled juggler can accelerate and shape his/her hand so
that the ball rolls up the fingers, over the top, and back down
to the back of the hand. This paper describes a robotic imple-
mentationof the butterfly. The hand’s shape and motion com-
bine to effect the rolling motion of the ball, and we find that
the shape and motion parameters enter the dynamic equa-
tions in a similar way. We define parameterized spaces of
hand shapes and motions, and using a simulation based on
the rolling equations, we identify shape and motion solutions
that roll the ball from one side of the hand to the other. We
describe an implementation of the butterfly on our planar dy- Figure 1: Sketch of a juggler doing a “butterfly.”
namic manipulationtestbed F LATLAND. This example is our
first step toward exploring the roles of shape and motion in nipulation, a robot can cause motion of the part relative to
dynamic manipulation. the end-effector, thereby controlling more part degrees-of-
freedom (Lynch and Mason [9]).
1 Introduction This paper focuses on rolling manipulation, where an ob-
ject rolls freely on the surface of the manipulator. We find
The robot manipulation problem is to find a set of controls that the derivative of the curvature of the manipulator sur-
that map the current state of the world to a goal state, where face and the acceleration of the manipulator enter into the
this mapping M is governed by the laws of physics. Typi- dynamic equations of rolling in a similar way. This implies
cally the robot controls are specified as motions or forces and that, at least locally, we can trade freedom in the manipula-
the system is engineered to simplify the mapping M, mak- tor trajectory for freedom in the manipulator surface shape.
ing it easier to construct robot plans. For instance, a grasped To obtain a desired rolling motion of the object, we may be
object tracks the motion of the hand, simplifying M to an able to reduce the number of actuators required by properly
identity relationship between hand and object motions. designing the shape of the surface. This is a type of dynamic
For manipulation by pushing, throwing, tapping, rolling, cam—we can use rolling dynamics and freedom in design-
and catching, however, the mapping M is a function of ma- ing the shape of the manipulation surface to transform sim-
nipulator shape, compliance, friction, restitution, etc. If we ple rotational or translational actuator motions to the desired
treat these as design (control) variables, then it is possible motion of the object.
to exploit the information embedded in M to simplify robot
hardware. Instead of constructing powerful dexterous robots
to force M to be a simple function, we can construct simple
2 The Butterfly
robots which work with, not against, the natural environmen- We have begun our investigation into the roles of shape
tal dynamics. and motion in dynamic manipulation by examining a jug-
In this paper we consider the relative roles of manipula- gler’s skill called the “butterfly” (Figure 1). Starting with a
tor shape and motion in dynamic nonprehensile manipula- ball resting on the palm of his/her open hand, a skilled jug-
tion. By nonprehensile we mean that the manipulated object gler can accelerate and shape his/her hand so the ball rolls up
is not grasped. By dynamic we mean that the robot exploits the fingers, over the top, and down to the back of the hand.
dynamics to help control the motion of the part. An example Our goal was to perform a planar version of this skill with
of dynamic nonprehensile manipulation is shooting a basket- a one joint robot, as shown in Figure 2. The design problem
ball. As the hand moves, dynamic forces cause the ball to is to find a hand shape and 1 DOF revolute motion profile
roll up the hand and off the fingertips into a free-flight tra- that rolls the disk from one side of the hand to the other while
jectory toward the hoop. With dynamic nonprehensile ma- maintaining rolling contact at all times.
 h

ch
xh

w
n^h

Figure 3: Coordinate conventions for the hand.

case. We define the 2x1 (or 3x2) Jacobian
Figure 2: A simulated robotic butterfly. Gravity acts downward.
The contact force is shown. @c :
J = @s h
h
h
3 Dynamic Rolling Equations
The outward-pointing unit contact normal at sh is given by
In this section we present the dynamic equations of rolling n^ (s ). In the planar case, n^
h h h is simply Jh =jjJh jj rotated by

between a moving surface (the “hand,” which could be any 90 degrees so that it is the outward-pointing normal. (Alter-
manipulator surface) and an object rolling with point contact natively, considering Jh =jjJh jj as a 3-vector with zero third
on the surface. We assume a hard contact model—no torque component, n ^ h is this vector crossed with a unit vector out
can be applied about the contact normal, and spin about the of the plane.) In the spatial case, n
^ h is the normalized cross-
contact normal is not explicitly prohibited. Only slip is dis- product of the two columns of Jh . The 2x1 (or 3x2) Jacobian
allowed. of n
^ h defines the curvature of the surface and is written
K = @^
n :
The kinematic equations of rolling have been derived pre-
h
viously in the context of rolling within a grasp. First-order
analysis relating the relative velocity of two objects to the @s h
h

change in contact coordinates has been carried out by Kerr The contact point in a world frame Fw is given by
and Roth [7], Montana [10], and Cole et al. [3]. Sarkar et
al. [11] built on Montana’s work by deriving the second- x +R c (s );
h h h h
order relationship between the relative motion of the contact-
ing bodies and the acceleration of the contact coordinates. where xh is the location of Fh in Fw and Rh is a rotation
This information is used in the dynamic control of rolling matrix (xh 2 Rn , Rh 2 SO(n) where n = 2 in the planar
motion in a grasp. Cai and Roth [2] derived the equations case and n = 3 in the spatial case). The time derivatives of
of motion in a manner allowing higher-order analysis. xh are written x_ h = vh and xh = ah . The angular velocity
Our derivation builds on this work, but differs in two and angular acceleration of Fh in Fw are written !h and h ,
ways. 1) The motion of the object relative to the hand is not respectively. In the planar case, we have
specified, but rather is determined by the acceleration (and  0 ?! 
resultant contact forces) of the hand. The input to the equa-
tions of motion is the acceleration of the hand, and the out- R_ = (! )R =
h h h
! 0 R;
h
h
h

puts are the acceleration of the object and the acceleration

of the contact coordinates. This is a second-order dynamic and in the spatial case, we have
analysis. 2) Our development applies to both the planar and 0 0 ?! ! 1
R_ = (! )R = @ ! 3 0 ?! 1 A R :
the spatial case. For this reason, we adopt notation similar 3 2 h h

to that of Cole et al. [3] (see Figure 3). In the spatial case, h h h h h h

simple transformations provide the metric tensor, curvature ?! 2 ! 1

h h 0
tensor, and torsion form used by Montana [10].
The hand is a one-dimensional curve (planar case) or a The matrix (h) may be defined similarly to (!h).
two-dimensional surface (spatial case). The hand is locally We can make similar definitions for the object being ma-
parameterized by sh , where sh 2 R in the planar case and nipulated, replacing the subscript “h” (for hand) with the sub-
sh 2 R2 in the spatial case. In a coordinate frame Fh at- script “o” (for object). The object frame Fo is fixed to the
tached to the hand, the contact position is given by ch (sh ), center of mass of the object. The mass of the object is m and
where ch 2 R2 in the planar case and ch 2 R3 in the spatial its inertia matrix is Io expressed in Fo .
 We are now ready to derive the rolling equations. of A, allowing us to simulate dynamic nonprehensile rolling
manipulation of smooth objects. In practice we also add a
1) Contact position constraint. The contact points on the ob- “snapping” routine to make sure the object stays on the sur-
ject and the hand must be coincident: face of the hand. We have written a rolling simulator in C.
x +R c ? (x +R c ) = 0:
h h h o o o
The simulation enforces rolling contact; we check the con-
tact force implied by ao and o to see if contact and friction
This constraint differentiates twice to yield the followingtwo constraints are satisfied.
The curvature derivative shape information in the term
(three) linear equations in the planar (spatial) case.
_Kh = @K h s_ (Equation 2) gives us a way to design the shape
@sh h

a +(  +!  ! )R c +2!  R J s_
h h h h h h h h h h of the hand to affect the rolling motion in a way similar to the
+R (J s + J_ s_ ) ? (a +(  +!  ! )R c
h h h h h o o o o o o
acceleration of the hand. This curvature derivative informa-
+2!  R J s_ +R (J s + J_ s_ )) = 0:
tion integrates to yield the shape of the hand.
o o o o o o o o o (1)

2) Contact normal constraint. The unit contact normals must 4 Designing the Shape and Motion
be opposite: Using the rolling simulation, we would like to design a
R n^ +R n^ = 0:
o o h h hand shape and motion to solve the planar butterfly task. Be-
This constraint differentiates twice to yield the followingtwo low we describe a space of shapes and motions from which
(three) linear equations in the planar (spatial) case. One of we will find a solution.
these equations is redundant because n ^ o and n^ h are con- 4.1 Shape Space
strained to be unit.
We would like a symmetric hand with a stable well (local
(  +!  ! )R n^ +2!  R K s_
o o o o o o o o o
minimum in a gravity field) when the disk is at the beginning
+R (K s + K_ s_ )+(  +!  ! )R n^
or end of the roll, as in Figure 2. We chose simple polynomial
o o o o o h h h h h functions satisfying these conditions:
+2!  R K s_ +R (K s + K_ s_ ) = 0: (2)
h h h h h h h h h
x(s ) = a+cs2 +es4
h h h

3) Rollingconstraint. To maintain rolling, the acceleration of

the contact points on the object and the hand must be equal y(s ) = bs +ds3 +fs5 ;
h h h h

when projected to the contact tangent space. This yields one where s 2 [?1;1] describes the right half of the hand. (The
h
(two) linear equations in the planar (spatial) case. other half is the mirror image.) Note that x(s ) and y(s )
h h

R J (a +(  +!  ! )R c
h
T
h h h h h h h
are even and odd functions, respectively. The six coeffi-
cients must satisfy the two independent equations x(?1) =
?a ? (  +!  ! )R c ) = 0: (3) x(1) = 0 and y0 (?1) = y0 (1) = 0, where the constraint on
y0 (1) forces the hand to be flat at these points. We chose
o o o o o o

4) Force constraint. The contact force passes through the a = 18, b = ?18, leaving a two-dimensional shape space
contact point. This yields one (three) linear equations in the which can be parameterized by c and d (Figure 4). Essen-
planar (spatial) case: tially d controls the depth of the well and c controls whether
 ?rf = 0
or not the curvature is pushed out to the ends of the hand.
For our experiments, we chose a = 18:0cm, b = ?18:0cm,
or equivalently,
c = ?25:2cm, d = 21:6cm, e = 7:2cm, and f = ?9:36cm.
Two important points are worth noting. First, not all points
I  +!  I ! ? R c  ma = 0:
w o o (4) w o o o o
in the shape space correspond to appropriate hands. Some
choices of c and d yield self-intersecting curves, curves with-
I = R I R is the object inertia matrix expressed in the
T out wells, or wells with curvature too high to accommodate
the disk (radius of 5:1cm in the experiments). Second, the
world frame F . The !  I ! term vanishes in the planar
w o o o

curvature is discontinuous at sh = 1, because we have re-

w o w o
case.
We would like to solve Equations (1)–(4) for ao , o , so , quired only that n ^ h be continuous there. The rolling equa-
and sh as a function of the state of the system and the inputs tions are ill-defined at sh = 1. This problem can be solved
ah and h . (We assume the input is the acceleration of the by increasing the order of the shape polynomials and forcing
them to satisfy continuous curvature and curvature deriva-
tive constraints at sh = 1. Here we use the low order poly-
hand, not force/torque.) Rearranging the equations, we get
the form
Ax = b; nomials and simply take care in the simulation near these sin-
gularities.
where A is 6x5 (11x10), x (the variables to be solved for) is
a 5-vector (10-vector), and b is a 6-vector (11-vector) in the 4.2 Motion Space
planar (spatial) case. These equations may be solved by pre- We would like to find a symmetric hand motion that dips
multiplying each side by (AT A)?1 AT , the pseudo-inverse the hand to begin the disk rolling, rotates 180 degrees to roll
 d
14.4 18.0 21.6 25.2 28.8

-25.2

c
-18.0 0.4 1.299
sh 0.2
0
-10.8 -0.2 1.181
-1.18
sh= -1 -1.16
2T (sec)
-1.14
k 1.063
sh= 0 -1.12
-1.10
sh= 1
Figure 6: Surface plot of the contact parameter sh at time T dur-
Figure 4: Some points in the two-dimensional hand shape space for ing the butterfly as a function of k and 2T . The locus of k 2T ?
a = 18, b = ?
18, and the shape we chose for our experiments. solutions to the butterfly corresponds to the points where sh = 0.

k = -1.4 A “solution” is any motion that takes the disk to exactly zero
θ k = -1.13 velocity on the opposite side of the hand. Because the hand
shape and the motion are symmetric, the motion of the disk
π on the hand is symmetric for any solution.
In a neighborhood of a solution in the k ? 2T space, in-
creasing the initial dip (smaller k) or the motion time 2T
k = -0.667

causes the disk to roll further (past the goal position). Fig-
π/2 ure 6 shows the hand contact parameter sh at time T dur-
ing the butterfly as a function of k and 2T . Because of the
symmetry of solutions, a necessary condition for a solution
motion is that sh = 0 at time T . The smoothness and mono-
0 tonicity of the sh surface of Figure 6 allow us to quickly con-
T 2T verge on the locus of points satisfying this condition.
In full gravity (9:8m=s2), the fastest solution which main-
t
tains contact at all times is 2T = 1:095s, with k = ?1:1493.
Figure 5: Example motion profiles for different values of k. At this speed, the contact force becomes zero when the disk
reaches its apex. At any higher speed the hand will throw the
the disk to the other side of the hand, overshoots to stop the disk. This is the time-optimal rolling solution in the k ? 2T
rolling, and finally settles in a horizontal position. We chose motion space.
the following motion profile: Figure 2 shows the solution for 2T = 1:181s and k =
?1:1393. In this example, the contact force at the apex of the
(t) = (j(t ? T)+k(t ? T)3 +l(t ? T)5 ) +  ;t 2 [0;2T]
2(jT +kT 3 +lT 5 ) 2
roll is 17.5% of the gravitational force. This motion requires
a contact friction coefficient of 0.306 to maintain rolling con-
tact. As we increase 2T , the value of k that solves the but-
where 2T is the total time of the motion. This function ro- terfly problem increases, implying a smaller initial dip (Fig-
_ = (2T)
tates the hand from 0 to . We require (0) _ =0 ure 7). The friction coefficient required to maintain rolling
and we set j = 1. The remaining two-dimensional motion goes to zero as 2T goes to infinity. This is because there is
space can be parameterized by k and T , where k determines a locus of (;sh ) equilibrium configurations connecting the
how far the hand initially dips to begin rolling of the disk. start configuration (0; ?1) and the goal configuration (;1)
k = ?2=3 yields zero dip, and the dip increases with in- where the contact force is normal to the surface and through
creasingly negative values of k. Example motion profiles are the disk center of mass. Some of these configurations (in the
shown in Figure 5. wells) are stable, others are unstable.
4.3 Simulation
5 Experimental Testbed: F LATLAND
Using the simulation with the hand shape chosen in Sec-
tion 4.1 and a disk of uniform mass and a radius of 5:1cm, we We implemented the butterfly on F LATLAND (Figure 8).
found a one-dimensional locus of solutions to the butterfly We built F LATLAND to be a general testbed for experiments
problem in the two-dimensional k ? 2T motion space. (For a in planar dynamic manipulation. F LATLAND consists of a
given motion, we could similarly find a one-dimensional lo- tiltable air table, allowing us to perform planar dynamic ma-
cus of solutions in the two-dimensional c ? d shape space.) nipulation experiments with variable gravity; a set of ma-
 1.0
-1.06
rolling solutions. (Increasing the time-scale permits more vi-
sion data during the roll.) The hand was made to follow the
trajectory 2T = (1:181s)= 4:0s, k = ?1:1393, as shown in
minimum
-1.08 minimum contact force
force
k the simulation in Figure 2. Even without feedback, the but-
-1.10 0.5
terfly often successfully rolled the disk to the goal well. By
simply scaling 2T , the butterfly was performed at different
k-2T solutions
-1.12
table angles.
-1.14
0
The open-loop butterfly is not robust. Often the disk rolls
1.5 2.0 2.5 too far, overshooting the final position, or does not roll far
2T (sec) enough, never reaching the goal configuration. In both cases
?
Figure 7: Plot of k 2T solutions to the butterfly problem and the
the hand drops the disk. To make execution robust, we
implemented vision feedback control. A simple estimator
corresponding minimum contact force as a ratio to the gravitational
was used to estimate the disk’s position between vision data
force. frames, and this data was used in a 1 kHz control loop. The
control follows three stages:
1. Perform the initial dipping motion open-loop to get the
disk rolling along the hand.
Motor 2. Once the disk has passed a certain point (typically sh >
?0:9), simply servo the hand toward the angle and an-
OKK vision tracker
Air Table
gular velocity in the planned trajectory that corresponds
to the disk’s current position on the hand. In other
words, the contact parameter sh drives the hand’s mo-
Control Vision
PC
tion. During this stage sh is monotonically increasing,
PC

Motor so there is a one-to-one mapping between sh and the

Drivers planned manipulator angle and angular velocity. The
control law is written
Figure 8: The F LATLAND experimental setup.  = k ( (s ) ?  )+k (_ (s ) ? _ );
p p h;a a d p h;a a

nipulator modules that can be configured as 1 or 2 DOF where k ;k > 0, s

p d is the actual contact parameter
h;a

robots mounted on rails around or over the air table; and a (from vision feedback),  is the actual hand angle, and
a

30 Hz OKK vision tracking system to track the motion of  (s ) is the planned hand angle when the disk is at
p h;a

objects on the table. To the underside of each manipulator s . (In our control system, the commanded acceler-
ation  is used to calculate a new reference position 
h;a

link is attached a manipulation surface. The manipulation

surfaces are the parts of the robot actually making contact and velocity _, which are then used in a PD controller
with the laminar objects floating on the table, and they can be to calculate joint torque. This approach suppresses non-
changed easily to allow us to experiment with new manipu- linearities in the actuator due to friction, etc.)
lator shapes (Figure 9). The manipulators are controlled by a
Pentium PC which receives vision data from an NEC PC98 3. Once the disk has passed a certain point (typically sh >
which processes the data from the OKK vision tracker. 0:9), perform the final overshoot motion open-loop.
The hand was cut from aluminum by a CNC machine and Despite the simplicity of the controller, it significantly sta-
attached to a manipulator module configured as a 1 DOF bilizes the rolling motion to small errors in initial conditions.
robot. The disk is made of plastic and is encircled by rubber Figure 10 shows experimental data for a butterfly under feed-
O-rings to increase friction with the aluminum manipulation back control with 2T = 4:0s;k = ?1:1393. The  and sh tra-
p
surface. We set the air table to a 5 degree angle, introducing
a time-scaling  = 1=sin5 = 3:387 from the full-gravity
jectories are symmetric, but the execution time is extended
to approximately 4:23s. The controller slowed the motion
of the hand in stage 2 to allow the disk more rolling time to
link compensate for errors.
manipulation
If the rolling velocity after stage 1 is too large, the hand
surface
may throw the disk slightly when the disk reaches the top
of the hand. (The butterfly often succeeds despite this, as
the hand catches the disk and continues on the trajectory.)
A slower nominal motion, which maintains a higher nomi-
nal contact force, could alleviate this problem. Another fail-
ure mode is when the disk does not have enough velocity
from the initial dip to reach the top of the hand. We are cur-
Figure 9: Manipulation surfaces attached to the robot links actually rently investigating nonlinear feedback controllers based on
make contact with objects on the air table. the rolling dynamics to make the butterfly more robust.
 3.5 cludes automatic trajectory planning for dynamic rolling ma-
nipulation; more robust nonlinear feedback control; design-
3.0
ing manipulation surfaces to achieve a desired rolling motion
2.5 with a low degree-of-freedom robot; and investigating the
roles of shape and motion in other kinds of dynamics, such
2.0
θ as impact in vibratory parts feeding.
1.5 Acknowledgments
1.0 sh This work was performed while the first author was an
STA postdoctoral fellow at the Biorobotics Division of the
0.5 Mechanical Engineering Laboratory. We thank the Science
0.0 and Technology Agency of Japan and the Robotics Depart-
1 2 3 4 ment of MEL for their support. We especially thank Garth
-0.5 Zeglin and Matt Mason for suggesting the butterfly problem.
time (sec)
-1.0
Their air table setup at Carnegie Mellon inspired many of the
ideas used in F LATLAND.
-1.5 stage 1 stage 2 stage 3

Figure 10: Experimental data for a butterfly with 2T = 4:0s, k =

References
? 1:1393, and the table set at a 5 angle. The controller slows the [1] H. Arai and S. Tachi. Position control system of a two degree
execution of the hand trajectory during stage 2 to allow the disk of freedom manipulator with a passive joint. IEEE Transac-
more rolling time to compensatefor errors. The total execution time tions on Industrial Electronics, 38(1):15–20, Feb. 1991.
is approximately 4:23s. [2] C. Cai and B. Roth. On the spatial motion of a rigid body with
point contact. In IEEE International Conference on Robotics
6 Discussion and Automation, pages 686–695, 1987.
The butterfly system resembles a revolute two-joint [3] A. B. A. Cole, J. E. Hauser, and S. S. Sastry. Kinemat-
robot with an unactuated second joint (Arai and Tachi [1]; ics and control of multifingered hands with rolling contact.
Suzuki et al. [12]). Both systems are subject to a second- IEEE Transactions on Automatic Control, 34(4):398–404,
order nonholonomic constraint from a passive pivot joint. Apr. 1989.
Three important differences are: 1) The pivot point is rolling [4] A. W. Divelbiss and J. Wen. A global approach to nonholo-
along the surface of the manipulator with the butterfly. 2) nomic motion planning. In IEEE International Conference on
The butterfly is performed in a gravity field. This allows Decision and Control, pages 1597–1602, 1992.
equilibrium configurations to be stabilized, but trajectories
[5] C. Fernandes, L. Gurvits, and Z. Li. Attitude control of a space
cannot be time-scaled. The time scale is chosen by gravity.
platform/manipulator system using internal motion. Interna-
3) There is a limited friction cone of contact forces that can
tional Journal of Robotics Research, 13(4):289–304, 1994.
be applied into the disk. Contact force constraints make the
control problem particularly challenging. [6] H. Hitakawa. Advanced parts orientation system has wide ap-
We have broken the control of the butterfly into two stages, plication. Assembly Automation, 8(3):147–150, 1988.
as suggested by the nonholonomic nature of the system: [7] J. Kerr and B. Roth. Analysis of multifingered hands. Inter-
planning a nominal trajectory and feedback stabilization of national Journal of Robotics Research, 4(4):3–17, 1986.
that trajectory. Figure 6 suggests that gradient-descent ap- [8] K. M. Lynch and M. T. Mason. Dynamic underactuated non-
proaches could be applicable to the trajectory planning prob- prehensile manipulation. In IEEE/RSJ International Confer-
lem; see, for example, (Divelbiss and Wen [4]; Fernan- ence on Intelligent Robots and Systems, pages 889–896, 1996.
des et al. [5]; Lynch and Mason [8]). The optimization could
also solve simultaneously for shape parameters according to [9] K. M. Lynch and M. T. Mason. Dynamic manipulation with a
some cost function weighting shape and motion. Since the one joint robot. In IEEE International Conferenceon Robotics
motion of the disk is solved for by simulation, care must be and Automation, pages 359–366, 1997.
taken to avoid numerical problems, especially if finite differ- [10] D. J. Montana. The kinematics of contact and grasp. Interna-
ences are used to approximate gradients. tional Journal of Robotics Research, 7(3):17–32, June 1988.
[11] N. Sarkar, X. Yun, and V. Kumar. Control of contact in-
7 Conclusion teractions with acatastatic nonholonomic constraints. Inter-
national Journal of Robotics Research, 16(3):357–374, June
We have derived the dynamic equations of nonprehensile 1997.
rolling and used a simulation based on them to find shape [12] T. Suzuki, M. Koinuma, and Y. Nakamura. Chaos and nonlin-
and motion solutions to the planar butterfly problem. We ear control of a nonholonomic free-joint manipulator. In IEEE
have successfully implemented a solution on F LATLAND, International Conference on Robotics and Automation, pages
our testbed for dynamic manipulation. Feedback control is 2668–2675, 1996.
used to stabilize the planned trajectories. Future work in-