scirobotics.ado6187
scirobotics.ado6187
INTRODUCTION notable results in terms of flight speed and agility. Song et al. (5) used
Birds have long captivated humans with their exceptional flight capa- reinforcement learning (RL) techniques to achieve flight speeds ex-
bilities to navigate at high speeds through cluttered environments, ceeding 30 m/s on a human-made racing track, using external com-
with remarkably low failure rates. Similarly, micro air vehicles (MAVs), puting for control and a motion capture system for state feedback.
among the most agile machines created by humans (1), hold the po- Foehn et al. (6) proposed a strategy based on progress optimization
tential to achieve bird-like high-speed agile flights. The rapid and safe to compute a time-optimal trajectory offline and track the trajectory
arrival of MAVs at their destinations is a critical factor for successfully with an onboard computer, whereas Romero et al. proposed a
deploying MAVs in practical applications. Being fast implies that an sample-based online planning approach (7) and used model predic-
MAV is able to reach designated locations in a timely manner, en- tive contouring control (8) to track the trajectory, achieving high-
abling rapid response in time-critical missions like search and rescue speed racing flights greater than 18 m/s. Kaufmann et al. (9),
(2) or disaster relief (3, 4). Being safe means that the MAV could detect following a similar RL approach, combined onboard sensing and
and avoid obstacles on the way without a collision failure. In this work, computation to surpass the performance of champion-level human
we explore how to endow MAVs with bird-like capabilities, relying pilots in racing missions. Despite achieving notable flight speed and
solely on onboard sensing and computational units, to achieve safe agility, these methods depend on external sensing (such as a motion
and high-speed flights in unknown environments (Movie 1). capture system) (5–8), external computation during actual flights
Achieving safety-assured, high-speed flights in unknown environ- (5), or offline training (5, 6, 9). Moreover, these methods assume a
ments is a complex task that necessitates a holistic system design. fixed, known environment (such as the racing track) (5–9), where
First, to execute aggressive maneuvers that avoid previously unknown the control policy is exhaustively trained for the best performance.
obstacles during high-speed flights, the MAV must have high agility, Their applicability to unknown environments is not clear.
characterized by its compact size and high thrust-to-weight ratio Autonomous flights in unknown environments using only on-
(TWR). Second, the MAV must have a long detection range to pro- board sensing and computation have been extensively investigated
vide sufficient reaction time for avoiding obstacles at high speeds. in the literature. The first set of approaches prioritizes flight speed
Moreover, such detection ability should be achieved with lightweight, (10–17). Escobar-Alvarez et al. adopted an expansion rate (ER)–
compact sensors to not degrade the MAV’s agility. Last, the MAV has based method to achieve high-speed flight ranging from 6 to 19 m/s
to carefully balance flight speed with safety in its trajectory planning. in a relatively open area (17), using visual sensors and a single-line
Flight speed and safety are two mutually conflicting factors, and their time-of-flight (TOF) light detection and ranging (LIDAR) sensor.
balance has to be achieved efficiently with the limited computation Loquercio et al. (16) used imitation learning to enable autonomous
power onboard the MAV. flight under challenging conditions, using an end-to-end approach
Several existing works have focused on addressing the challenges and achieving a maximum speed exceeding 10 m/s.
in autonomous drone racing applications (5–9) and have achieved Another approach is the Bubble planner proposed by Ren et al. (13),
which used receding horizon corridors for trajectory optimization, re-
sulting in a maximum speed of 13.7 m/s in complex environments. An
Department of Mechanical Engineering, University of Hong Kong, Pokfulam, Hong inherent issue with these works is their exclusive focus on flight speed at
Kong, China.
†These authors contributed equally to this work. the expense of safety guarantees. To maximize flight speed, existing works
*Corresponding author. Email: fuzhang@hku.hk (10–15, 17) all assume that the unknown regions caused by occlusions or
A
Safety
Solid-state LiDAR
LiDAR FOV
(Livox Mid 360) Safety-assured
Speed in
Onboard Autonomous
Computing Flights
Optimistic
Power
Onboard Computer m/s)
High (20
(Intel NUC 12) High
/s)
Low (3m
Low
SUPER (Ours) Flight Controller
m)
Sm
(PX4)
.4
a
ll (
(0
1
Size: 280 mm TWR: > 5.0
.0)
diu
Me
La
ZJU Swarm Commercial Drone Agilicious
m)
rg
e(
0.1
5.0
o(
)
Size
cr
Mi
(Wheelbase) TWR
Size: 110 mm TWR: 2.4 Size: 380 mm TWR: 2.3 Size: 250 mm TWR: > 5.0
ts Avo
gh id
Fli ing
d T
ee
hi
-sp
n
Wi
h
Hig
re s
B(iii) B(iv)
Clutte
ht
Nig
ed r
at
En
vi
tio
n
ro
nm
ents v i ga
Na
Fig. 1. Overview of the proposed autonomous aerial system. (A) Radar chart comparing SUPER with other state-of-the-art autonomous aerial robots, including the palm-
sized autonomous MAV (ZJU Swarm) (35), the open-source agile quadrotor platform (Agilicious) (33), and a representative commercial drone (27). For Agilicious, we obtained the
data from its best performance in onboard autonomous flights as reported in (16). For the commercial drone, we obtained its size, TWR, and the maximum flight speed allowed
by its APAS from the official manufacturer’s website (27). The onboard computation power and safety strategy are not publicly disclosed and hence are not shown. (B) Demon-
stration of SUPER in real-world flights. (i) High-speed flights in the wild; (ii) avoidance of thin electrical wires; (iii) navigation through cluttered environments; (iv) flights at night.
average flight speeds. The performance of SUPER in real-world sce- collision free, and the environment was completely unknown before
narios has also been validated with a substantial number of field the flight, requiring SUPER to detect and avoid those obstacles in
tests. In all experiments, SUPER was tasked to reach a goal position the environment in an online manner. In these real-world tests,
with fully onboard perception, planning, and control. The line con- SUPER achieved a speed of 20 m/s in unknown, unstructured envi-
necting the goal and the MAV’s current position was usually not ronments and maintained a 100% success rate across eight different
trials (Fig. 1B, i). Moreover, SUPER outperformed advanced com- Both systems demonstrated successful detection and tracking of the
mercial MAVs regarding flight safety against small obstacles and target throughout the entire task, ensuring a fair comparison for
adaptability to environments of high clutter and low lighting condi- their navigation capabilities.
tions. In particular, SUPER was able to detect and avoid thin objects, The tracking trajectories of both MAVs are presented in Fig.
such as power lines and tree branches, in the wild (Fig. 1B, ii) and 4A. In the beginning stage where the target is in open areas, the
navigate through extremely cluttered environments (Fig. 1B, iii), commercial drone and SUPER can achieve a comparable tracking
even at night (Fig. 1B, iv). SUPER was also successfully applied in performance. After the target entered the first wooded area, the
object tracking, autonomous exploration, and waypoint navigation commercial drone failed to track the target and stopped in front of
missions (see Fig. 2), demonstrating its robustness and versatility in the woods (position A in Fig. 4, A and C, i). A likely reason is that
real-world scenarios. Video recordings of some flights are supplied the visual navigation used by the commercial drone had a limited
in Movie 1. map resolution and accuracy (typically tens of centimeters), which
necessitated more conservative collision-free trajectories to enhance
Safe high-speed navigation in unknown environments safety. Subsequently, the target exited the first wooded area and ran
We tested the flight speed of SUPER in an unknown forest environ- toward the second one. The commercial drone successfully resumed
ment (see Fig. 3 and movie S1). The environment covered an area of tracking by finding a path detouring the wooded area (the orange
around 280 by 90 m2 and featured trees of varying thicknesses and line between positions A and B in Fig. 4A). However, after the target
diverse natural vegetation (Fig. 3A, i). We conducted eight experi- entered the second wooded area with a higher density, the commer-
ments at different times of day, creating a wide range of lighting con- cial drone failed the tracking mission completely and disengaged
ditions from normal bright day to completely dark night (Fig. 3A, ii from the automatic tracking mode (Fig. 4D, i) for the same reason
to v). We also set different maximum speed constraints across the mentioned above. In contrast, SUPER planned its trajectory directly
Fig. 2. Additional applications of SUPER. Examples of object tracking including (A) tracking of a person running in a dense forest, (B) a dark indoor scene, (C) a large
scene, and (D) a car moving on a hilly village road. (A)(i) and (D)(i) show the point cloud maps, which were synthesized from online LIDAR measurements after the flights
were completed, with trajectories of the target and the MAV being colored in blue and red, respectively. (A)(ii) and (D)(ii) show snapshots in either third-person or first-
person view during the experiments. (E) (i) Autonomous exploration experiment in a 50 m–by–60 m area with the executed trajectory of SUPER shown in the white path.
(ii and iii) Snapshots of the scene and the corresponding reconstruction results, respectively. (F) Example of waypoint navigation. (i) The MAV starting from ps visited 11
waypoints, p1 ∼ p11, sequentially. The MAV trajectory is color coded on the basis of time. (ii) Time-lapse image composition capturing the changes in the environments
and the flight trajectories of the MAV. The black artifacts are caused by the two persons moving randomly in the area, and the MAV trajectory is highlighted in blue. (iii to v)
MAV flight trajectories at locations traveled by moving persons. SUPER can use the spaces previously traversed by moving objects.
28
0m A(iv) Test 7 A(v) Test 8
m
90
B(iv)
B(iv)
C D
Maximum Speed
Distribution of Speed [m/s]
Fig. 3. Safe high-speed navigation in unknown environments. (A) Bird’s-eye view of the experiment environment. (ii to v) Four representative lighting conditions from
four of the eight experiment tests. Specifically, (ii) was captured during test 1, and (iii to v) were captured in tests 6 to 8, respectively. (B) (i) The executed trajectories during
the eight tests are shown in curves with different colors. In all eight tests, the MAV flew through the waypoints ps, p1, p2, and p3 sequentially and stopped at position p4.
Overlaid with the trajectories is the point cloud map, which is synthesized offline with point measurements collected during the flight. (ii to iv) Three detailed views of the
environment on flight paths. (C and D) Violin plots illustrating the distribution of flight speed and position tracking error across the eight tests. The shape of the violin plot
represents the density of the data distribution, with the white point indicating the mean value. The black error bar represents the SD. The data for these plots were ob-
tained by uniformly sampling 2000 data points from the flight data of each test.
on these thin objects (Fig. 5D), and the point cloud map in our plan- Evaluation of safety, success rate, and efficiency
ning module effectively retains these points for trajectory planning. We evaluated the performance of SUPER’s planning module through
The robustness of SUPER when faced with small objects further a series of controlled experiments conducted in simulation. The
strengthens its safety for real-world missions. A video illustration of testing environments consisted of a set of randomly generated 3D
this experiment is shown in movie S3. forest-like scenarios at a fixed size of 110 m by 20 m (Fig. 6A). These
A B
Target
SUPER (Ours)
D Commercial drone (approx.)
B
C SUPER (Ours)
A
C
Commercial Drone
C(i) C(ii)
Fig. 4. Navigation in cluttered environments. (A) Trajectories of the target, the commercial drone, and SUPER during the target tracking experiment. After starting from
target target
position ps , the target passed sequentially through two wooded areas and ended at position pg . The commercial drone tracked the target closely until the target
entered the first wooded area, where the commercial drone failed to find a collision-free path to track the object and stopped at position A. Once the target exited the first
wooded area, the commercial drone resumed tracking by finding a path detouring the wooded area. However, the commercial drone failed the tracking mission again
and stopped at position B when the target entered the second wooded area. In contrast, SUPER tracked the target well without stopping throughout the whole process.
The point cloud map is synthesized offline with point measurements collected online during the flight of SUPER. (B) SUPER with extended size and the commercial drone
(27). (C) Tracking of the target when she entered the first wooded area. The commercial drone failed to track the target because of the tight spaces, whereas SUPER could
track the object closely. (D) Tracking of the target when she entered the second wooded area. The commercial drone failed the tracking again, whereas SUPER was able
to fly through the tight space where the person being tracked had to lower her body to pass through.
scenarios encompassed six distinct obstacle densities ranging from (23). The implementation details of these four planners are summa-
3.1 to 6.5 in terms of the traversability (30). Traversability is a metric rized in table S1. Each planner was evaluated 18 times in each map,
describing the obstacle density normalized by the robot’s size, spe- with maximum velocity varying from 1 to 18 m/s. The maximum
cifically its maximum radius (r = 0.2 m in our simulation), where a acceleration was fixed at 20 m/s2 for all experiments. In each experi-
higher traversability corresponds to a less challenging obstacle avoid- ment, a planner was given a goal 100 m away from the start position,
ance task. For each scenario, the positions and inclination angles of and the experiment terminated at any of the three conditions. A “suc-
the obstacles were randomly generated. To ensure diversity, we used ceed” occurs when the MAV reaches the goal without any collisions
different random seeds to generate 10 different maps for each density, and satisfies all kinematic constraints, including velocity and accel-
resulting in a total of 60 distinct maps for evaluation. We bench- eration. A “collision” happens when the MAV collides with obstacles
marked the performance of SUPER with three state-of-the-art base- during the flight. An “unfinished” outcome is recorded if, in any re-
line planners: Bubble planner, an optimistic planning strategy (13); planning cycle before reaching the goal, the planner fails to return a
Raptor, a safety-aware method (20); and Faster, a safety-assured method trajectory within 30 s.
Fig. 5. Demonstration of thin object avoidance. (A) (i to iv) Four thin wires of varying diameters were used in the experiments. (B) (i to iv) Time-lapse images capturing
the flights of DJI Mavic 3. It successfully avoided the thin wire of 30-mm diameter but failed to avoid wires with smaller diameters. The collision of the commercial drone
caused the wire to swing, leading to some artifacts in the time-lapse images despite only one actual wire being present in each experiment. (C) (i to iv) Time-lapse images
capturing the flights of SUPER. It successfully avoided all four types of thin wires. (D) Point cloud view of SUPER when facing a 2.5-mm thin wire. The wire was seen in the
current scan measurements (white points) and more evident in the accumulated point cloud (colored points).
Fig. 6. Evaluation of safety, success rate, and efficiency. (A) Benchmarking environments with varying traversability. (B) Flight results of the benchmarked methods in
1080 experiments. (C) Distributions of the ratio of switching to backup trajectories for SUPER and Faster at varying obstacle densities, with each distribution computed
from 180 tests. The white points represent the mean values, and the error bars indicate the SDs. (D) Success rate of the benchmarked methods at different obstacle densi-
ties and flight speeds. Empty columns indicate that the corresponding combination of speed and density was not achieved. (E) Time consumption of the benchmarked
methods, with squares representing the mean values and error bars indicating the SDs of the total computation time. Each mean and SD was computed from 180 tests.
We evaluated the safety of all benchmarked methods in Fig. 6B in parameters and can be solved efficiently. In contrast, the MIQP prob-
terms of the safe rate, which is the ratio of both succeed and unfinished lem solved in Faster for trajectory optimization is a nondifferentiable
cases over all experiments. Across all 1080 experiments with varying and hard to solve, taking a computation time of up to 41 ms. Con-
flight speed and obstacle density, SUPER had no collision or infeasible sequently, Bubble and SUPER took the least total computation time
trajectory, achieving a perfect safe rate. Faster (23) is a safety-assured (8 to 44 ms for Bubble and 10 to 47 ms for SUPER). The slightly
method in principle because of the planning of a backup trajectory in higher time consumption of SUPER when compared with Bubble is
each replan cycle, similar to SUPER. However, we found that its back- due to the planning of an extra backup trajectory. Raptor took 67 to
up trajectory was not constrained in the map region, where the un- 83 ms, and Faster took 67 to 91 ms; both were higher than SUPER.
known and occupied information is available, causing collisions when
the map was not updated in time. The collision rate of Faster reached Applications
12.78%, leading to an overall safe rate of 87.22%. Bubble (13) achieved We demonstrated the use of SUPER in three representative applica-
a safe rate of 68.80%; the remaining 31.20% of collision cases were tions: object tracking, autonomous exploration of unknown spaces,
caused by its optimistic planning strategy that treats unknown spaces and waypoint navigation in changing environments (movie S4). In
as free. Regarding Raptor (20), it suffered from a collision rate of 38.33%, object tracking, we commanded SUPER to track an object by repeat-
which was primarily due to the planned trajectory frequently violating edly assigning its local planning goal as a position behind the object.
kinodynamic constraints during high-speed flights. These trajectories We conducted extensive experiments to evaluate SUPER’s perfor-
are challenging for the simulated MAV to track, resulting in collisions mance in tracking both a person and a car, as depicted in Fig. 2 (A to
with obstacles. Consequently, the overall safe rate was merely 61.67%. To D). In all experiments, the target object was attached with a sheet of
sum up, SUPER outperformed other benchmarked methods by achiev- high reflectivity, enabling reliable detection within the LIDAR point
ing safe flights in all experiments, highlighting its exceptional safety- cloud based on reflectivity measurements. The target was then tracked
providing sufficient computation power for efficient, online optimi- performance even under low-light conditions, such as nighttime,
zation of both exploratory trajectories, attaining high-speed flights surpassing visual sensors’ capabilities in these scenarios.
and backup trajectories and ensuring safety. The high agility and Using 3D LIDAR sensors for MAV navigation is not new. For
onboard computation power have enabled SUPER to achieve high- example, several works (10, 11, 40–42) used the VLP-16 on multiro-
speed flights of greater than 20 m/s in unknown cluttered environ- tor UAVs, and other works (29, 43–45) used the Ouster OS1-128
ments without a collision failure. The safety-assured and high-speed (46). A crucial problem among these works lies in their LIDAR sen-
flying capability of SUPER places itself in a unique position among sors. Existing LIDAR sensors are bulky, heavy, and costly: The prior
existing autonomous aerial robots (Fig. 1A). Agilicious (33) exhibits two factors necessitate a UAV of a large size (wheelbase of 330 to 1133
a high TWR of greater than 5.0 and a size similar to SUPER, but its mm) and low TWR [for example, typically around 2.0 (11, 29, 40, 41, 43–
imitation learning–based planning (16) achieves a lower flight speed 45, 47, 48)], which restrict the agility of the UAV. The high cost of
of 10 m/s and lacks a safety guarantee, yielding a low success rate in LIDAR sensors has also prevented the wider adoption of them in UAV
high-speed flights (around 60% at 10 m/s). Moreover, the onboard navigation, not even to speak of the high-speed agile flights that pose
computing unit, an Nvidia TX2 (34) with a four-core 1.9-GHz CPU, a much higher risk to the system. The reported flight speeds of exist-
has limited computational power, which restricts the potential ap- ing LIDAR-based UAVs were merely 2 to 5 m/s (40, 41, 44). Zhang
plication of Agilicious for more computationally demanding tasks, et al. (10, 11, 42) achieved a flight speed of 10 m/s with a LIDAR-
such as autonomous exploration. On the other hand, ZJU Swarm based UAV, but their UAV was large (wheelbase of 1133 mm), heavy
(35) concentrates on designing small-size and lightweight autono- (more than 13 kg), and was only demonstrated in a static, sparsely
mous MAVs for swarm applications, achieving an impressive size of wooded environment (for example, the distance between obstacles
a 110-mm wheelbase with a weight less than 300 g. However, this was more than 10 m) with very smooth flights. Our SUPER is en-
design has a low TWR (only 2.4) and limited onboard computation abled by the rapid recent advancements in LIDAR technology,
Methods for a benchmark comparison). The accurate and high- SUPER’s trajectory optimization required only half the computation
resolution point map also enabled SUPER to avoid thin obstacles in time of Faster while adhering to all constraints. Another key differ-
the wild and navigate through tight spaces safely. Last, outdated ence between Faster and SUPER is the determination of switching
points caused by moving objects or LIDAR false measurements pres- time between backup and exploratory trajectories. Faster determines
ent another problem for point cloud maps. We addressed this prob- the switching time heuristically before the backup trajectory optimi-
lem using a spatiotemporal sliding mechanism, which has enabled zation on the basis of a wild assumption that the backup trajectory
SUPER to use the areas traversed by moving objects. would decelerate along the exploratory trajectory. In contrast, SUPER
optimizes the switching time along with the backup trajectory
Safety-assured high-speed trajectory planning optimization. The optimized switching time in SUPER is usually
SUPER achieved high-speed and safe planning by calculating two much larger than the heuristically determined one in Faster, reducing
trajectories at each time: an exploratory trajectory in both known the flight time on the backup trajectory and the chance of switching
free spaces and unknown spaces and a backup trajectory in known to it (fig. S6).
free spaces only. This two-trajectory strategy was first proposed in
Faster (23, 28). SUPER adopted such a two-trajectory strategy but Limitations and future directions
had completely different designs for sensing, mapping, and plan- At the hardware level, the emergence of smaller and lighter LIDAR
ning, which are optimized for higher flight speeds, success rates, and sensors with longer sensing ranges and denser point clouds may
computation efficiency. Ultimately, SUPER achieved flight speeds open up new opportunities for autonomous aerial systems. Although
exceeding 20 m/s in the unknown wild using only its onboard sens- the LIDAR adopted by SUPER is much smaller and lighter than pre-
ing and computing devices. vious ones, it remains larger and heavier than visual sensors, limiting
From sensing, Faster (23) used a depth camera (38) for obstacle the potential for further miniaturization of the MAV platform. This
MATERIALS AND METHODS (position, velocity, and attitude) at a frequency of 200 Hz and with a
Agile LIDAR-based MAV platform latency of less than 1 ms. It also produces world-frame registered
The design objective of SUPER is to achieve a high TWR while min- point clouds at a scan rate of 50 Hz, with around 4000 points per
imizing its size. After a thorough investigation, we finalized the de- scan, resulting in a total of about 200,000 points per second at a la-
sign of SUPER, as depicted in Fig. 7A. The platform has a takeoff tency of less than 10 ms.
weight of 1.5 kg and a compact wheelbase measuring only 280 mm. For the mapping module, we propose a point cloud–based spa-
For the motors, we selected the T-Motor F90 type (54), with each tiotemporal sliding point cloud map to represent the occupied spac-
motor capable of generating a maximum thrust of 23.6 N. This con- es of the environment. To prevent increasingly densified points over
figuration results in a maximum total thrust of 94.4 N and an esti- time accumulation, we downsampled the points in the map using a
mated ideal TWR of ~6.3. Because of factors like airflow interference uniform grid data structure. This structure divides the space into
and blockage, the actual TWR is slightly above 5.0, enabling highly uniform grids with a resolution of 0.05 to 0.3 m and retains only the
agile maneuvers. For computation, the MAV is equipped with a center point of each occupied grid cell. Also, to prevent overlarge
flight control unit (FCU) running PX4 Autopilot (55) and an Intel maps that are not necessary for local planning, we only maintained
NUC onboard computer that features a 12-core 4.7-GHz i7-1270P a local map of 50 to 100 m around the MAV using a zero-copy map
CPU. For sensing, the MAV equipped a lightweight 3D LIDAR (25) sliding strategy following our previous work (51). Furthermore, a
enabling object detection at ranges exceeding 70 m with a rapid well-known problem of a point cloud map is that LIDAR points be-
point rate of 200,000 Hz. The MAV also equipped an IMU on the longing to both the static environment and moving objects in past
FCU, providing acceleration and angular rate measurements at a locations are considered to be occupied straightly, which will falsely
frequency of 200 Hz. The LIDAR point clouds and IMU measure- classify the space traveled by moving objects as occupied. To address
ments are input to the navigation software, which consists of a this issue, a temporal sliding strategy was used to cull points outside
Fig. 7. System overview of SUPER. (A) Hardware configuration of SUPER. (B) Perception module comprises two components: state estimation and mapping. (C) The
planning module is a safety-assured planner based on point cloud maps. The planned trajectory includes an exploratory trajectory e, which spans both known free space
and unknown space, and a backup trajectory b, which starts from a state on e and lies entirely within known free space. Part of e and the entirety of b form the com-
mitted trajectory c, which is then executed by the MAV. (D) The control module is an accurate and efficient OMMPC.
effectively leverage the extended sensing range of LIDAR to accom- Flight corridor generation in configuration space
plish long-range updates. In contrast, volume mapping methods We represent both exploratory and backup corridors as a set of over-
such as OGMs (50) or ESDF-based maps (31) have to update all grid lapping polyhedra and extract them directly from LIDAR point
cells in the measuring range, leading to a much higher computation cloud inputs to enhance the computation efficiency. For the explor-
load and short mapping range, typically around 5 to 10 m. atory corridor, the objective is to find a set of overlapping polyhedra
that connect the start points ps to the goal position pg. To achieve
Safety-assured high-speed trajectory planning this, we first use an A* path search algorithm (57) to find a collision-
Framework overview free path connecting pc and pg (Fig. 8B, i); the resulting path con-
To achieve safe and high-speed trajectory planning, we used a two- sists of a set of discrete positions with a step equal to the map
trajectory planning strategy where two trajectories are planned at resolution (such as 0.1 m). Then, we search for a series of line seg-
each replan cycle (Fig. 7C). The first trajectory, called the explor- ments along the path by repeatedly finding the furthest visible
atory trajectory e : pc → pg, which expands from the MAV current point from the end of the last line segment. Specifically, starting
position pc to a goal position pg. With the aim to achieve high-speed from pc, the first point of , we find the furthest point s1 that is vis-
flight, the exploratory trajectory treats unknown space as free and is ible from pc and form a line segment (pc, s1). Then, starting from s1,
planned in both known free spaces and unknown spaces. The sec- we find its furthest visible point s2 and form the line segment (s1, s2).
ond trajectory, referred to as the backup trajectory b : ps → pl, be- This process is repeated until the goal position pg is visible (Fig. 8B,
gins from a position ps on the exploratory trajectory at time ts and ii). Last, each of the found line segments is used as a seed for convex
ends at a position pl with zero speed. With the aim to ensure safe decomposition, which extracts a polyhedron that contains the seed
flights, the backup trajectory and the segment of the exploratory tra- but not any obstacle points input to it (Fig. 8B, iii).
jectory from pc to ps must remain in known free spaces. In cases The convex decomposition has to be performed in the robot con-
from the map, and includes the seed, ensuring that the current posi- that the backup corridor, which is extracted by the CIRI algorithm
tion pc is within the corridor as required. As a result, this polyhe- and should contain the seed (pc, s), is aligned with the exploratory
dron can serve as the backup corridor. trajectory and can hence contain a substantial portion of the explor-
On the basis of theorem 1, the detailed process is illustrated in Fig. atory trajectory. Meanwhile, considering that practical LIDAR sen-
8C. For the seed, we select the line segment that starts from the MAV’s sors are not infinitely dense, to make a sufficiently dense depth image,
current position, pc, and ends at the furthest point s on the explor- we accumulate recent LIDAR scans (for example, 1 to 2 s) and input
atory trajectory e that is visible from pc (Fig. 8C, i). The purpose is them to CIRI, along with the line seed (pc, s), to obtain a convex
polytope (Fig. 8C, ii). The area within the generated polytope is ̇ 22 ≤ vmax
‖p‖ 2
̈ 22 ≤ amax
, ‖p‖ 2
(5)
guaranteed to be known free according to theorem 1. If the LIDAR
has a limited FOV, the intersection between the polytope obtained [ ]
by CIRI and a convex subset of the LIDAR FOV region can be used pi (t) ∈ i , ∀ i ∈ [1, M], t ∈ 0, Ti (6)
(Fig. 8C, iii). This intersection forms another polyhedron within the
known free space. Moreover, the first polytope of the exploratory cor- where ρT > 0 is the weight used to penalize the total flight time to
ridor must also be intersected with the MAV’s FOV to ensure that the achieve high flight speed and ρu > 0 is the penalty weight for the soft
initial portion of the exploratory trajectory lies in known free space, penalty term. Constraint Eq. 5 guarantees compliance with kinody-
thereby enhancing the feasibility of the backup trajectory optimiza- namic constraints, whereas constraint Eq. 6 constrains the position
tion. A detailed method for selecting the convex subset of the FOV trajectory within the flight corridor. The term in the objective
and an analysis of the FOV’s influence are provided in the Supple- function is defined as
mentary Materials.
Trajectory optimization M
∑ ( )
Given the exploratory and backup corridors generated in the previ- = μ ‖ ‖2 (7)
‖qi − ai ‖2
ous section, we consider how to optimize smooth trajectories within i=1
them. For exploratory trajectory generation, optimizing high- where ai is the center location of the overlapping area between i
speed trajectories is challenging because it requires not only spatial and i+1. This term is used to prevent the trajectory from being too
optimization of the trajectory shape but also temporal optimiza- proximate to the boundaries of the flight corridor, which is often
tion for
[ time] allocation. The objective is to find a smooth trajectory where the obstacles are located. Maintaining a distance from obsta-
p(t): 0, tM → ℝ3 that minimizes the flight duration tM and control cles would enhance the visibility of the onboard sensor and hence
constraints, two additional optimization variables are added: the distribution, including its density and range. The kernel density es-
switching time ts and the terminal position pl. The terminal position timate ̂f (x) was calculated as
pl replaces the goal position pg in Eq. 2 and should be determined
within the optimization because the MAV could rest at any position 1 ∑
n (x −x )
̂
f (x) = K i
(10)
in the known free space. Moreover, because the backup trajectory nh i=1 h
lies within the backup corridor, the start point ps should lie in the back-
up corridor as well, necessitating an additional constraint tc < ts < to, where n is the number of data points, xi is the data point, h is the
where tc is the current time and to is the time when the exploratory bandwidth parameter, and K is the kernel function (a Gaussian ker-
trajectory exits the backup corridor, respectively (see Fig. 8D). To nel was used in our analysis). In each violin plot, the width corre-
eliminate the inequality constraints in the optimization, we param- sponds to the density of data points at different values, and the central
eterize ts as another variable η ∈ ℝ as follows: dot represents the mean value. The error bars, which indicate the SD,
( ) are shown alongside the mean to convey the data’s variability.
1
t s = t o − tc + tc (8) Box plots, used in fig. S8, summarize the data distribution by dis-
1 + e−η playing the median and interquartile range (IQR). The whiskers ex-
where η is completely unconstrained (see fig. S10). As a result, the tend to 1.5 times the IQR, and data points beyond this range were
complete optimization variables are (Q, T, pl, ts). Combining the considered outliers and are not shown in the plots.
fact that the trajectory pi(t) is differentiable to the Q, T, the initial
state s0, and the terminal ) sf containing pl [as proven in (29)]
( state Supplementary Materials
and the fact that s0 = e ts is differentiable to ts (and hence η), the The PDF file includes:
whole optimization problem containing the objective and con- Supplementary Methods
On-manifold model predictive controller Other Supplementary Material for this manuscript includes the following:
The position trajectory p(t) generated by the planning module is then Movies S1 to S4
transmitted to the control module for tracking (Fig. 7D). The position
trajectory is first transformed into a state trajectory s(t) on the basis of REFERENCES AND NOTES
the differential flatness of the MAV subject to drag effects (64). Then, 1. J. Verbeke, J. D. Schutter, Experimental maneuverability and agility quantification for
to accurately track this state trajectory at high speeds, we implement rotary unmanned aerial vehicle. Int. J. Micro Air Veh. 10, 3–11
(2018).
an on-manifold model predictive control (OMMPC) developed in 2. B. Mishra, D. Garg, P. Narang, V. Mishra, Drone-surveillance for search and rescue in
our previous work (65). The method linearizes the quadrotor system natural disaster. Comput. Commun. 156, 1–10 (2020).
by mapping its states from the state manifold to the local coordinates 3. S. M. S. M. Daud, M. Y. P. M. Yusof, C. C. Heo, L. S. Khoo, M. K. C. Singh, M. S. Mahmood,
of each point along the trajectory s(t), leading to a linearized system H. Nawawi, Applications of drone in disaster management: A scoping review. Sci. Justice
62, 30–42 (2022).
that is minimally parameterized while avoiding kinematic singulari-
4. B. Rabta, C. Wankmüller, G. Reiner, A drone fleet model for last-mile distribution in
ties, allowing the MAV to perform aggressive maneuvers with large disaster relief operations. Int. J. Disaster Risk Reduct. 28, 107–112
attitude angles, such as banked turns at sharp corners. The linearized (2018).
system is a linear time-varying system, which is then controlled by a 5. Y. Song, A. Romero, M. Müller, V. Koltun, D. Scaramuzza, Reaching the limit in
usual MPC. To enhance the control accuracy, we further consider the autonomous racing: Optimal control versus reinforcement learning. Sci. Robot. 8,
eadg1462 (2023).
rotor drag compensation in the control. Detailed derivations of the 6. P. Foehn, A. Romero, D. Scaramuzza, Time-optimal planning for quadrotor waypoint
OMMPC with rotor drag compensation and a quantitative analysis of flight. Sci. Robot. 6, eabh1221 (2021).
the compensation effectiveness are provided in Supplementary Meth- 7. A. Romero, R. Penicka, D. Scaramuzza, Time-optimal online replanning for agile
ods and fig. S13. quadrotor flight. IEEE Robot. Autom. Lett. 7, 7730–7737 (2022).
8. A. Romero, S. Sun, P. Foehn, D. Scaramuzza, Model predictive contouring control for
time-optimal quadrotor flight. IEEE Trans. Robot. 38, 3340–3356
Statistical analysis (2022).
We used three types of statistical analyses: error bars, violin plots, 9. E. Kaufmann, L. Bauersfeld, A. Loquercio, M. Müller, V. Koltun, D. Scaramuzza,
and box plots. Error bars were used to represent the SD around the Champion-level drone racing using deep reinforcement learning. Nature 620, 982–987
mean, reflecting the variability in the data. The SD σ was calculated as (2023).
10. J. Zhang, R. G. Chadha, V. Velivela, S. Singh, P-cal: Pre-computed alternative lanes for
√ aggressive aerial collision avoidance, paper presented at the 12th International
√ n Conference on Field and Service Robotics (FSR), Tokyo, Japan, 31 August 2019.
√ 1 ∑ ( )2
σ=√ x −μ (9) 11. J. Zhang, R. G. Chadha, V. Velivela, S. Singh, P-cap: Pre-computed alternative paths to
n − 1 i=1 i enable aggressive aerial maneuvers in cluttered environments, in 2018 IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2018),
pp. 8456–8463.
where n is the number of data points, xi is each data point, and μ is 12. Y. Ren, S. Liang, F. Zhu, G. Lu, F. Zhang, Online whole-body motion planning for quadrotor
the mean value. using multi-resolution search, in 2023 IEEE International Conference on Robotics and
Violin plots, used in Figs. 3 and 6 and fig. S12, were used to il- Automation (ICRA) (IEEE, 2023), pp. 1594–1600.
lustrate the distribution of various performance metrics (e.g., flight 13. Y. Ren, F. Zhu, W. Liu, Z. Wang, Y. Lin, F. Gao, F. Zhang, Bubble planner: Planning highspeed
smooth quadrotor trajectories using receding corridors, in 2022 IEEE/RSJ International
speed, trajectory tracking error, and the ratio of backup trajectory Conference on Intelligent Robots and Systems (IROS) (IEEE, 2022), pp. 6332–6339.
execution) across multiple tests. These plots combine features of error 14. X. Zhou, Z. Wang, H. Ye, C. Xu, F. Gao, Ego-planner: An esdf-free gradient-based local
bars and kernel density estimates, providing a depiction of the data’s planner for quadrotors. IEEE Robot. Autom. Lett. 6, 478–485 (2020).
15. H. Ye, X. Zhou, Z. Wang, C. Xu, J. Chu, F. Gao, Tgk-planner: An efficient topology guided 44. P. De Petris, H. Nguyen, M. Dharmadhikari, M. Kulkarni, N. Khedekar, F. Mascarich,
kinodynamic planner for autonomous quadrotors. IEEE Robot. Autom. Lett. 6, 494–501 sK. Alexis, Rmf-owl: A collision-tolerant flying robot for autonomous subterranean
(2020). exploration, in 2022 International Conference on Unmanned Aircraft Systems (ICUAS) (IEEE,
16. A. Loquercio, E. Kaufmann, R. Ranftl, M. Müller, V. Koltun, D. Scaramuzza, Learning 2022), pp. 536–543.
high-speed flight in the wild. Sci. Robot. 6, eabg5810 (2021). 45. Y. Cai, F. Kong, Y. Ren, F. Zhu, J. Lin, F. Zhang, Occupancy grid mapping without raycasting
17. H. D. Escobar-Alvarez, N. Johnson, T. Hebble, K. Klingebiel, S. A. Quintero, J. Regenstein, for high-resolution LiDAR sensors. IEEE Trans. Robot. 40, 172–192
N. A. Browning, R-advance: Rapid adaptive prediction for vision-based autonomous (2023).
navigation, control, and evasion. J. Field Robot. 35, 91–100 (2018). 46. Ouster, Ouster OS1-128 sensor (2024); https://ouster.com/products/scanning-lidar/
18. L. Quan, Z. Zhang, X. Zhong, C. Xu, F. Gao, Eva-planner: Environmental adaptive os1-sensor.
quadrotor planning, in 2021 IEEE International Conference on Robotics and Automation 47. DJI, DJI Matrice 300 RTK (2024); https://enterprise.dji.com/zh-tw/matrice-300.
(ICRA) (IEEE, 2021), pp. 398–404. 48. LiDAR USA, LiDAR USA surveyor 32 (2024); https://lidarusa.com/.
19. L. Wang, Y. Guo, Speed adaptive robot trajectory generation based on derivative property 49. Intel, Product Family D400 Series Datasheet (2024); https://intelrealsense.com/download/
of b-spline curve. IEEE Robot. Autom. Lett. 8, 1905–1911 (2023). 21345/?tmstv=1697035582.
20. B. Zhou, J. Pan, F. Gao, S. Shen, Raptor: Robust and perception-aware trajectory 50. A. Hornung, K. M. Wurm, M. Bennewitz, C. Stachniss, W. Burgard, Octomap: An efficient
replanning for quadrotor fast flight. IEEE Trans. Robot. 37, 1992–2009 (2021). probabilistic 3d mapping framework based on octrees. Autonom. Robot. 34, 189–206
21. H. Oleynikova, Z. Taylor, R. Siegwart, J. Nieto, Safe local exploration for replanning in (2013).
cluttered unknown environments for microaerial vehicles. IEEE Robot. Autom. Lett. 3, 51. Y. Ren, Y. Cai, F. Zhu, S. Liang, F. Zhang, Rog-map: An efficient robocentric occupancy grid
1474–1481 (2018). map for large-scene and high-resolution lidar-based motion planning. arXiv:2302.14819
22. S. Liu, M. Watterson, S. Tang, V. Kumar, High speed navigation for quadrotors with limited [cs.RO] (2023).
onboard sensing, in 2016 IEEE International Conference on Robotics and Automation (ICRA) 52. F. Kong, W. Xu, Y. Cai, F. Zhang, Avoiding dynamic small obstacles with onboard sensing
(IEEE, 2016), pp. 1484–1491. and computation on aerial robots. IEEE Robot. Autom. Lett. 6, 7869–7876
23. J. Tordesillas, B. T. Lopez, M. Everett, J. P. How, Faster: Fast and safe trajectory planner for (2021).
navigation in unknown environments. IEEE Transact. Robot. 38, 922–938 (2021). 53. H. Wu, Y. Li, W. Xu, F. Kong, F. Zhang, Moving event detection from LiDAR point streams.
24. E. Mueggler, H. Rebecq, G. Gallego, T. Delbruck, D. Scaramuzza, The event-camera dataset Nat. Commun. 15, 345 (2024).
and simulator: Event-based data for pose estimation, visual odometry, and slam. Int. J. 54. T-MOTOR, T-MOTOR F90 series racing drone motor specifications (2024); https://store.
73. Y. Cui, D. Sun, K.-C. Toh, On the r-superlinear convergence of the KKT residuals generated whereas G.L. and Y.R. implemented the control module with the assistance of N.C. The
by the augmented lagrangian method for convex composite conic programming. Math. experiments were designed by Y.R. and F. Zhu and conducted by Y.R., F. Zhu, G.L., N.C., and F.K.
Program. 178, 381–415 (2019). Y.R. and J.L. performed all data analyses. The manuscript was written by Y.R. and F. Zhang, with
74. W. Wu, mockamap, https://github.com/HKUST-Aerial-Robotics/mockamap. input and advice from all authors. F. Zhang provided funding for the research and supervised
75. Dronecode Foundation, PX4 software in the loop simulation with Gazebo (2024); https:// the project. Competing interests: The authors declare that they have no competing interests.
docs.px4.io/v1.12/en/simulation/gazebo.html. Data and materials availability: The data and source code supporting the conclusions of this
study are available at https://doi.org/10.5281/zenodo.14528604 and https://github.com/
Acknowledgments hku-mars/SUPER.
Funding: This work was supported by the Hong Kong Research Grants Council (RGC) General
Research Fund (GRF) 17204523 and a DJI donation. Author contributions: The research was Submitted 13 February 2024
initiated by Y.R. and F. Zhang. Y.R. was responsible for the design and manufacture of the UAV Accepted 23 December 2024
prototype, as well as the implementation of the planning modules for the UAV, with assistance Published 29 January 2025
from G.L. and L.Y. F. Zhu and Y.C. were responsible for implementing the perception model, 10.1126/scirobotics.ado6187