A universal 3D imaging sensor on a silicon photonics platform

Christopher Rogers1,∗ , Alexander Y. Piggott1,∗ , David J. Thomson2 , Robert F.

Wiser1 , Ion E. Opris3 , Steven A. Fortune1 , Andrew J. Compston1 , Alexander
Gondarenko1 , Fanfan Meng2 , Xia Chen2 , Graham T. Reed2 , and Remus Nicolaescu1,†
1
Pointcloud Inc, San Francisco, California 94107, USA
2
Optoelectronics Research Centre, University of Southampton,
University Road Southampton, Hampshire SO17 1BJ, UK
Opris Consulting, San Jose, California 95138, USA
∗ These authors contributed equally to the work. and
† Corresponding author: [email protected]
(Dated: July 27, 2020)
arXiv:2008.02411v3 [physics.app-ph] 11 Nov 2020

Accurate 3D imaging is essential for machines to map and interact with the physical world[1, 2].
While numerous 3D imaging technologies exist, each addressing niche applications with varying
degrees of success, none have achieved the breadth of applicability and impact that digital image
sensors have achieved in the 2D imaging world[3–10]. A large-scale two-dimensional array of coherent
detector pixels operating as a light detection and ranging (LiDAR) system could serve as a universal
3D imaging platform. Such a system would offer high depth accuracy and immunity to interference
from sunlight, as well as the ability to directly measure the velocity of moving objects[11]. However,
due to difficulties in providing electrical and photonic connections to every pixel, previous systems
have been restricted to fewer than 20 pixels[12–15]. Here, we demonstrate the first large-scale
coherent detector array consisting of 512 (32 × 16) pixels, and its operation in a 3D imaging system.
Leveraging recent advances in the monolithic integration of photonic and electronic circuits, a dense
array of optical heterodyne detectors is combined with an integrated electronic readout architecture,
enabling straightforward scaling to arbitrarily large arrays. Meanwhile, two-axis solid-state beam
steering eliminates any tradeoff between field of view and range. Operating at the quantum noise
limit[16, 17], our system achieves an accuracy of 3.1 mm at a distance of 75 metres using only
4 mW of light, an order of magnitude more accurate than existing solid-state systems at such
ranges. Future reductions of pixel size using state-of-the-art components could yield resolutions
in excess of 20 megapixels for arrays the size of a consumer camera sensor. This result paves the
way for the development and proliferation of low cost, compact, and high performance 3D imaging
cameras, enabling new applications from robotics and autonomous navigation to augmented reality
and healthcare.

This paper is dedicated to the memory of Sunil Sandhu.

The digital complementary metal–oxide– mance, such as in mobile devices and augmented reality
semiconductor (CMOS) image sensor revolutionized systems. Optical phased arrays are a promising solid-
2D imaging, borrowing technology from silicon micro- state approach, but developing long-range 2D-scanning
electronics to produce a flexible and scalable camera systems has proven challenging, with current demonstra-
sensor[18]. As a focal plane array (FPA), the digital tions limited to less than 20 metres[21–23]. As such, no
image sensor uses a lens to focus light and form an current technology addresses the needs of these diverse
image on the detector. A key advantage of this scheme use cases.
is that the field of view and light collection efficiency are
not set by the image sensor, but by the choice of lens. Here, we demonstrate a fully solid-state, integrated
Furthermore, the CMOS image sensor can be optimized photonic LiDAR based on the same FPA concept as the
for high performance or low-cost. The flexibility of this CMOS image sensor. By efficient use of light, our sys-
arrangement made the digital CMOS sensor the sensor tem achieves the range, depth accuracy and field of view
of choice for most 2D imaging. needed by demanding applications such as self driving
vehicles[1] and drone- or land-based 3D mapping[24, 25].
In contrast, 3D imaging is characterized by an assort- The centerpiece of our system is the coherent receiver
ment of competing technologies, each addressing a nar- array, a focal plane array of compact optical heterodyne
row niche. Long range and high precision applications detectors operating at the quantum noise limit[16, 17].
such as autonomous vehicles and construction site map- To eliminate tradeoffs between field of view and range,
ping are dominated by expensive and fragile mechani- the receiver is paired with solid-state beam steering that
cally steered LiDAR systems[3, 4]. Solid-state solutions sequentially illuminates the scene in small patches. The
such as structured light[5] and time-of-flight arrays[6– coherent receiver array allows our architecture to operate
10, 19, 20] are used when affordability, compactness, and using a frequency-modulated continuous-wave (FMCW)
reliability must be achieved at the expense of perfor- coherent LiDAR scheme[26, 27]. In contrast to time-of-
 2

flight LiDARs that use pulses of light, FMCW LiDAR This switching approach to beam steering is robust and
uses a linearly chirped laser. Scattered light received can be scaled up to arbitrarily large arrays, with optical
from the target is mixed with local oscillator light in a losses limited only by waveguide scattering[32] and the
heterodyne receiver, producing a beat frequency propor- extinction ratio of the switching trees. Meanwhile, the
tional to round-trip travel time and hence distance to the receiver consists of a dense FPA of miniaturized hetero-
target. dyne receivers. All receiver pixels that correspond to the
The FMCW scheme confers several advantages relative illuminated area are simultaneously read out in parallel.
to time-of-flight schemes. First, heterodyne detection is Since the angular resolution is defined by the point spread
immune to interference from sunlight and other nearby function of the lens, which drops off very quickly, there
LiDAR systems since it selectively detects light close in is negligible crosstalk between different receiver pixels.
frequency to the local oscillator light[11]. Second, co- To avoid wasting local oscillator light, a second switch-
herent LiDAR directly measures target velocity through ing tree provides only the activated subset of the receiver
Doppler shifts of the received light[26, 27]. Third, high FPA with local oscillator light.
depth accuracy, depending upon only chirp bandwidth Parallel readout in the receiver is fundamental to this
and signal-to-noise ratio[28], is achieved using relatively architecture’s scalability. First, resolution is defined by
low frequency receiver electronics. In contrast, depth ac- the number of pixels in the receiver FPA, rather than
curacy is limited by receiver bandwidth for time-of-flight the number of steering positions. This increases the
schemes. Finally, the FMCW system emits photons con- maximum resolution for a given chip size since hetero-
tinuously and thus is well suited for photonic integration, dyne receiver pixels are ∼ 10× smaller than thermo-optic
where non-linear effects constrain peak power[29, 30]. switches. Second, fast thermo-optic switching is unneces-
Conversely, time-of-flight schemes emit photons in short sary because the pixel rate is decoupled from the switch-
high-power bursts. ing rate. Finally, parallel readout proportionally reduces
Previous 3D imaging systems based on coherent re- the receiver signal frequencies for an FMCW scheme by
ceiver arrays have been limited to fewer than 20 pix- allowing longer ramp times, simplifying the readout elec-
els due to their reliance on direct electrical connections tronics.
to each pixel[12–15]. To address scalability, we imple-
mented our LiDAR system on a silicon photonics pro-
cess with monolithically integrated radio-frequency (RF) DESIGN OF HYBRID CMOS-PHOTONICS CHIP
CMOS electronics[31]. A highly multiplexed electronic
readout architecture is integrated into the receiver array, An optical micrograph of our demonstrator chip is
minimizing external electrical connections while main- shown in Fig. 1(c). The transmitter consists of a 1 × 16
taining signal integrity. Our 512-pixel prototype array thermo-optic switch tree with 16 grating couplers in
can thus be scaled to arbitrarily large numbers of pix- the transmit FPA. Meanwhile, the receiver consists of
els by increasing the size of the array. Furthermore, the a 32 × 16 (512) pixel array of heterodyne receivers, with
use of a standard commercial foundry process facilitates local oscillator light provided by a 1 × 8 switch tree. A
immediate mass production at minimal cost. subsection of the receiver is schematically illustrated in
Fig. 2(a). Each pixel collects scattered light from the
scene using a grating coupler. Local oscillator light is
SCALABLE 3D IMAGING ARCHITECTURE provided to each pixel via a network of silicon waveg-
uides. Scattered light and local oscillator (LO) are mixed
As shown in Fig. 1(a), our architecture is based on two on a balanced detector consisting of a 50-50 directional
FPAs. The first acts as a transmitter, and the second as coupler and germanium PIN photodiodes, producing a
a receiver. Chirped laser light for the FMCW scheme heterodyne tone corresponding to the target’s distance.
is generated externally by modulating a fixed-frequency The signal is then amplified by a transimpedance ampli-
1550 nm laser with a silicon-photonic IQ Mach-Zehnder fier (TIA) integrated within the pixel. A buffer amplifier
modulator (MZM), which is in turn driven by an arbi- at the end of each row transmits the signal to the edge
trary waveform generator. This approach ensures chirp of the chip, maintaining wide bandwidths while driving
linearity and enables the use of a simple, low-noise laser. the large parasitic capacitances of the wiring and mul-
Long-range performance is achieved by sequentially il- tiplexed circuitry. Finally, a set of 8 output amplifiers
luminating and reading out the scene in small patches. transmits the signal off-chip, enabling parallel readout.
By illuminating only the portion of the scene correspond- As shown in Fig. 2(b), the individual pixels are turned
ing to the pixels being read out, no illumination light on and off using a power switch built into each TIA.
is wasted. As illustrated in Fig. 1(b), the transmit- In general, minimizing the input-referred noise of the
ter consists of a switching tree terminated by a FPA of electronic signal chain improves the pixel’s sensitivity
grating couplers. Light is directed to one transmit grat- and detection probability. Furthermore, higher receiver
ing at a time, illuminating a small subset of the scene. bandwidths are desirable since this increases the max-
 3

a IQ modulator
Power splitter Laser
MZM

90° MZM Single-mode

Switch Transmitter chip Optical fiber
control amplifier
Chirp generator
Microlens array

Receiver chip
Digital
signal
processing

Beamsplitter Lens

b Active transmit grating Active receiver block

Transmitter chip Receiver chip

Heterodyne
detector pixel
Transmit Local
light oscillator

1×M switch Focal plane array 1×N switch Focal plane array
Receiver block
c

FIG. 1. Solid-state 3D imaging architecture. (a) Our architecture consists of two focal plane arrays (FPAs): a transmitter
FPA that sequentially illuminates patches of the scene, and a receiver FPA that detects scattered light from the scene. The
frequency-modulated continuous-wave scheme is used for ranging. An optional microlens array can be used to shape the
illumination pattern to more closely match the receiver array, thereby improving system efficiency. (b) On-chip steering of
light is provided by thermo-optic switching trees on both the transmitter and receiver chips. (c) Optical micrograph of our
demonstrator chip, showing the switching trees and focal plane arrays for both the transmit and receive functionality.

imum range for a given integration time. The TIA gain and bandwidths above 280 MHz, as shown in Fig.
feedback resistance determines gain, bandwidth, and 3(a)-(b). As seen in Fig. 3(d), our integrated TIA has
noise, with bandwidth and noise decreasing with larger a similar gain-bandwidth product and a 2 − 3× lower
resistance[33]. Compact waveguide-coupled photodiodes noise floor than conventional systems with photodiodes
and tight integration between the photodiodes and TIAs and amplifiers on separate chips.
yields a small parasitic capacitance of 1.5 fF. We thus The low noise on-chip amplifier chain allows the re-
achieve low noise in the electrical signal chain for 20 kΩ ceiver FPA to operate at the quantum limit for sensitiv-
 4

Power, Transimpedance
ground amplifier Power + ground
vias RF output wires
via

LO light from
switching tree
50/50
coupler

Photodiode

Grating
coupler

Receiver pixel

Si (undoped)
Si (n-doped)
Si (p-doped) Electronic amplifiers
Row RF
Ge Metal wires outputs

Row amplifier

b Heterodyne detector pixel c

Power bus
Block and
Grating column enable
coupler

Pixel signal
Directional output bus
couplers

Ground bus

LO bus
Ge Transimpedance Inter-
Optical
Balanced amplifier stage
Detector filter Electrical 50 μm

FIG. 2. Receiver focal plane array design. (a) Schematic of a receiver block in our receiver focal plane array. Within the
receiver block, local oscillator (LO) light is distributed to a dense array of heterodyne detector pixels via a network of silicon
waveguides. Meanwhile, each pixel collects scattered light from the scene using a grating coupler, which is combined with LO
light on a balanced detector to produce a detectable photocurrent. The photocurrent is amplified in two stages: first by a
transimpedance amplifier within the pixel, and again by an amplifier at the end of each row. For clarity, we have omitted
control wires from the diagram. (b) Electrical schematic of the heterodyne detector pixel. (c) Optical micrograph of a small
subset of the receiver focal plane array.

ity, which is reached when local oscillator shot noise dom- the size of the LO distribution tree since many receiver
inates all other noise sources[16, 17]. As shown in Fig. pixels can simultaneously share LO power.
3(e), shot noise reaches parity with amplifier noise with Monolithic integration of electronics into the receiver
only 5 µW of LO power for a typical receiver pixel. In FPA facilitates the use of an actively multiplexed read-
contrast, coherent receivers for telecommunications typ- out architecture, allowing the receiver to be scaled to
ically require ∼ 10 − 100× more LO power. Combined arbitrarily large numbers of pixels. In our demonstrator
with the excellent 30 − 40 dB common-mode rejection chip, multiple levels of multiplexing and amplification are
ratio of the balanced heterodyne detectors, as shown in used to map 512 pixels to 8 outputs while maintaining
Fig. 3(c), this makes the receiver array significantly less signal integrity. As illustrated in Extended Data Fig.
susceptible to LO noise sources such as laser relative in- 2(a-c), the pixels are read out in blocks of 8 at a time.
tensity noise, optical amplifier noise, and chirp generator The output analog signals are fed into a bank of off-chip
noise. Furthermore, the low LO power per pixel reduces analog-to-digital converters for digitization, followed by
 5

a b 120 c
f-3dB = 280 MHz 120

Receiver response (dB)

0
100 100

Occurrences

Occurrences
80 80
−5
60 60

−10 40 40
20 20
−15 0 0
1 2 3
10 10 10 0.8 1.0 1.2 1.4 20 30 40 50
Frequency (MHz) Noise floor (pA/√Hz) CMRR (dB)

d 2 e 5
10
4
Noise floor (pA/√Hz)

Noise floor (pA/√Hz)

3

2
1 Previous work
10

1 Amplifier

ise
noise

no
ot
0
This work

Sh
10
0.5
12 13 14 15 −2 −1 0 1
10 10 10 10 10 10 10 10
Gain-bandwidth product (Ω-Hz) LO power (μW)

FIG. 3. Receiver electro-optic performance. (a) Measured frequency response of the receiver readout chain for an optical signal
supplied to a single pixel, showing a cutoff frequency of 280 MHz. (b, c) Histograms of input-referred amplifier noise and
common-mode rejection ratio (CMRR) respectively throughout the full array, showing tight distributions for both parameters.
(d) Largely due to tight integration between our photodiodes and transimpedance amplifiers, we have achieved a high gain-
bandwidth product with significantly improved noise performance compared to previous designs. (e) Input-referred noise as a
function of optical local oscillator (LO) power for a single pixel, demonstrating shot-noise limited detection using < 10 µW of
LO power.

digital signal processing on a field-programmable gate- achieved a measurement precision of 1.8 mm at 17 m

array (FPGA). for a 85% reflectance target, and 3.1 mm at 75 m for a
The transmitter illumination pattern is closely syn- 30% reflectance target. Due to the effects of speckle,
chronized with the receiver readout pattern, as detailed which equally impacts all coherent radar and LiDAR
in the Methods and Extended Data Fig. 2(d). To further schemes[34], the detection probability was 97% for the
improve the optical efficiency of the system[34], a mi- 17 m target, and 42% for the 75 m target. Meanwhile, the
crolens array was placed in front of the transmitter array velocity precision for slowly moving objects is 1.0 mm/s,
as illustrated in Extended Data Fig. 3. This produced a as shown in Fig. 4(c). Point clouds of a rotating basket-
structured illumination pattern that exactly matched the ball at 17 m, stacked boxes at 55 m, and an exterior wall
grating coupler positions in the receiver FPA, as shown at 75 m are illustrated in Fig. 4(d-h). The point clouds
in Extended Data Fig. 4, yielding a 24× improvement in were generated by stacking 3 sequential frames to mini-
signal strength. mize the number of missing pixels due to speckle effects.
No incoherent averaging was used. Detection probabil-
ities were calculated without any frame stacking. The
3D IMAGING AND VELOCIMETRY missing band of points in the middle of the point clouds
is due to a narrow gap in the receiver array for electri-
LiDAR system operation is presented in Fig. 4 us- cal and optical routing, as shown in Fig. 1(c). In future
ing an emitter power of 4 mW at the aperture, a chirp designs, this gap can be reduced or eliminated by more
bandwidth of 4 GHz, and up- and down-chirp lengths aggressive chip layout and routing.
of 850 µs. As shown in Fig. 4(a), distance and veloc-
ity are encoded in the frequency tones detected by each
pixel[26, 27]. As demonstrated in Fig. 4(b), the system
 6

a Frequency (MHz) b c
0.40 0.45 0.50 0.55 Rotating basketball at 17 m (1 rpm)
3000 200
3 Static object 18 m target (85% reflectance) Pixel at left edge of ball
10

Occurrences

Occurrences
150
Up-chirp 2000
Photons

2
10
1 Down-chirp 100
10 1000
0 50
10
0 0
3 Moving object 75 m target (30% reflectance) Pixel at right edge of ball
10 400

Occurrences

Occurrences
150
Photons

2
10 300
1 100
10 200
0
100 50
10
0 0
15 16 17 18 19 −10 −5 0 5 10 −15 −10 −5 0 5 10 15
Distance (m) Depth error (mm) Velocity (mm/s)

d e Rotating basketball at 17 m
10
Velocity (mm/s)

0.2
0

0.1
-10

y (m)
0.0

−0.1
1 rpm

−0.2

17.8
f 10
17.7
Velocity (mm/s)

17.6 5

17.5 0
)
(m

17.4
z

−5
−0.2 17.3
−0.1
0.0 17.2 −10
x (m) 0.1 −15 −10 −5 0 5 10 15
0.2
x (cm)
g h
Boxes at 54 m Exterior wall at 75 m

0.4
0.6
0.2 0.3
y (m)

y (m)

0.0 0.0
−0.2 −0.3

−0.4 −0.6 Field of view

76.0
−0.6 −0.4 54.8 −1.0
−0.2 0.0 −0.5 75.5
54.6 0.0
)
(m

0.2 0.4 54.4 x (m) 0.5 75.0

)

x (m)
(m

0.6 1.0
z

FIG. 4. 3D imaging system characterization. (a) Representative signals from a receiver pixel, showing Doppler splitting between
the up- and down-chirps for the moving target. (b) Depth noise for targets at 18 m and 75 m, with standard deviations of
1.8 mm and 3.1 mm respectively. (c) Velocity histograms for a basketball rotating at 1 rpm, exhibiting a standard deviation of
1.0 mm/s. (d) Velocity annotated point cloud of a basketball at 17 m rotating about its vertical axis at 1 rpm. (e) Photograph
of the basketball setup. (f) Horizontal linecut of velocity across the middle of the basketball. (g, h) Point clouds of (g) stacked
cardboard boxes at 54 m, and (h) an exterior wall at 75 m. Distance to the target is indicated by colour in (g) and (h). The
missing band of points in the middle of the point clouds is due to a narrow gap in the receiver array for electrical and optical
routing.
 7

DISCUSSION AND OUTLOOK robotics and autonomous navigation to consumer prod-

ucts such as augmented reality headsets. Our results sug-
gest that the equivalent of the CMOS image sensor for
We have demonstrated a scalable solid-state 3D imag-
3D imaging is imminent, ushering in a broad range of ap-
ing architecture that achieves > 70 m range and
plications which were previously impractical or unimag-
millimetre-class accuracy with only 4 mW of transmit-
inable.
ted power. Our 3.1 mm precision is an order of magni-
tude higher than existing solid-state 3D imagers at these
ranges, with state-of-the-art flash LiDAR systems lim- Acknowledgements
ited to an accuracy of several centimetres for distances
greater than 50 metres[7–10]. This level of performance
D. J. Thomson acknowledges funding from the Royal
meets the needs of a variety of demanding applications
Society for his University Research Fellowship. We thank
previously out of reach for solid state 3D imaging sys-
Andy Stricker, Andy Watts, Mehrdad Djavid and the
tems. The long range and eye safety requirements of
rest of the Global Foundries team for assistance in device
autonomous driving[1], and the millimeter-class accu-
fabrication.
racy required by 3D mapping of constructions sites and
buildings[2, 24, 25, 35] can be simultaneously achieved
by our system. Contributions
System range could be improved by correcting the 4×
mismatch between the number of transmitter and re- C.R. and A.Y.P. contributed equally to this work. C.R.
ceiver positions as discussed in Methods, and through in- conceived, built and tested the free-space portion of the
creases in transmitter power. The latter can be achieved LiDAR system, performed the final LiDAR measure-
by optimizing the silicon photonic elements to minimize ments, and calibrated the optical switching trees. A.Y.P
transmission losses. Reducing two-photon absorption and C.R. performed the electro-optic characterization.
with wider waveguides or reverse-biased PIN junctions A.Y.P. conceived, built and tested the fiber-optic portion
would further increase transmitter power, with previous of the LiDAR system, wrote the LiDAR system software,
demonstrations reaching ∼ 1 W optical power [30]. Since tested the coherent receiver array, and contributed to
the range of a coherent LiDAR scales as the square root of the architecture and layout of the photonic chip. D.J.T.
transmitter power[34], this implies that our architecture designed and laid out the silicon photonics modulator.
could operate at ranges of up to 1 km. Conversely, for a R.W. contributed to design verification including circuit
fixed distance of 75 m, increasing the transmitter power simulations, and contributed to the embedded software
to 1 W and eliminating the mismatch between trans- control systems. I.O. designed the electronic circuits on
mitter and receiver would increase the point rate from the photonic chip and performed their layout and verifi-
4.7 × 103 to 5 × 106 points per second. Finally, depth ac- cation. S.A.F developed the signal acquisition and con-
curacy could be improved by increasing chirp bandwidth. trol systems, and contributed to the system architecture.
Demonstrations of 50 GHz silicon photonic modulators A.J.C. designed and verified the circuit boards used to in-
[36] imply that ∼ 200 µm precision is feasible. terface with the photonic chip. A.G. contributed to the
Our 3D imaging architecture naturally scales to large architecture and performed the layout of the photonic
arrays. By using a 1 ×N switching tree to steer light, the chip. F.M and X.C. contributed to the fabrication and
number of active thermo-optic switches and associated testing of the modulator. R.N. conceived the receiver,
DACs and drivers needed for control is O(log N ). On the switching, and photonic system architecture. R.N. and
receiver side, power consumption depends only upon the G.T.R. supervised the project.
number of parallel readout channels and is independent
of the size of the array since only active amplifiers are
Competing Interests
powered at any given time. The fundamental limit on
array size therefore comes from the size of the chip. At
the current receiver pixel pitch of 80 × 100 µm2 , chips In terms of competing interests, all authors with the
the size of a full-frame camera sensor (36 × 24 mm2 ) exception of F.M. and X.C. disclose that they are share-
would have QVGA (320×240 pixel) resolution. However, holders of Pointcloud Inc., a start-up company engaged in
using state-of-the-art designs 8×5 µm2 pixels are feasible, making laser ranging devices based on coherent receiver
corresponding to full-frame sensors with resolutions of arrays.
4500×4800 pixels. Further design refinements could yield
even higher resolutions.
We have developed a universal solid-state 3D imag-
ing architecture with the potential to meet the needs [1] C. Urmson, J. Anhalt, D. Bagnell, C. Baker, R. Bit-
of nearly all 3D imaging applications, spanning from tner, M. N. Clark, J. Dolan, D. Duggins, T. Galatali,
 8

C. Geyer, M. Gittleman, S. Harbaugh, M. Hebert, T. M. Lasers and Electro-Optics (Optical Society of America,
Howard, S. Kolski, A. Kelly, M. Likhachev, M. Mc- 2018) p. JTh5C.2.
Naughton, N. Miller, K. Peterson, B. Pilnick, R. Ra- [23] C. V. Poulton, M. J. Byrd, P. Russo, E. Timurdogan,
jkumar, P. Rybski, B. Salesky, Y.-W. Seo, S. Singh, M. Khandaker, D. Vermeulen, and M. R. Watts, IEEE
J. Snider, A. Stentz, W. R. Whittaker, Z. Wolkowicki, Journal of Selected Topics in Quantum Electronics 25, 1
J. Ziglar, H. Bae, T. Brown, D. Demitrish, B. Litk- (2019).
ouhi, J. Nickolaou, V. Sadekar, W. Zhang, J. Struble, [24] J. Wang, W. Sun, W. Shou, X. Wang, C. Wu, H.-Y.
M. Taylor, M. Darms, and D. Ferguson, Journal of Field Chong, Y. Liu, and C. Sun, Journal of Intelligent &
Robotics 25, 425 (2008). Robotic Systems 79, 417 (2015).
[2] Q. Wang and M.-K. Kim, Advanced Engineering Infor- [25] A. Kasturi, V. Milanovic, B. H. Atwood, and J. Yang,
matics 39, 306 (2019). in Laser Radar Technology and Applications XXI , Vol.
[3] D. D. Lichti, ISPRS Journal of Photogrammetry and Re- 9832, edited by M. D. Turner and G. W. Kamerman,
mote Sensing 61, 307 (2007). International Society for Optics and Photonics (SPIE,
[4] J. Kidd, Performance Evaluation of the Velodyne VLP- 2016) pp. 206 – 215.
16 System for Surface Feature Surveying, Master’s thesis, [26] H. D. Griffiths, Electronics Communication Engineering
University of New Hampshire (2017). Journal 2, 185 (1990).
[5] J. Salvi, J. Pagès, and J. Batlle, Pattern Recognition 37, [27] J. Riemensberger, A. Lukashchuk, M. Karpov, W. Weng,
827 (2004), agent Based Computer Vision. E. Lucas, J. Liu, and T. J. Kippenberg, Nature 581, 164
[6] A. Corti, S. Giancola, G. Mainetti, and R. Sala, Robotics (2020).
and Autonomous Systems 75, 584 (2016). [28] K. Thurn, R. Ebelt, and M. Vossiek, in 2013 IEEE MTT-
[7] P. McManamon, Optical Engineering 51, 1 (2012). S International Microwave Symposium Digest (MTT)
[8] P. F. McManamon, P. S. Banks, J. D. Beck, D. G. Fried, (2013) pp. 1–3.
A. S. Huntington, and E. A. Watson, Optical Engineer- [29] H. K. Tsang, C. S. Wong, T. K. Liang, I. E. Day, S. W.
ing 56, 1 (2017). Roberts, A. Harpin, J. Drake, and M. Asghari, Applied
[9] S. W. Hutchings, N. Johnston, I. Gyongy, T. Al Abbas, Physics Letters 80, 416 (2002).
N. A. W. Dutton, M. Tyler, S. Chan, J. Leach, and [30] H. Rong, A. Liu, R. Jones, O. Cohen, D. Hak, R. Nico-
R. K. Henderson, IEEE Journal of Solid-State Circuits laescu, A. Fang, and M. Paniccia, Nature 433, 292
54, 2947 (2019). (2005).
[10] A. Ronchini Ximenes, P. Padmanabhan, M. Lee, Y. Ya- [31] K. Giewont, K. Nummy, F. A. Anderson, J. Ayala,
mashita, D. Yaung, and E. Charbon, IEEE Journal of T. Barwicz, Y. Bian, K. K. Dezfulian, D. M. Gill,
Solid-State Circuits 54, 3203 (2019). T. Houghton, S. Hu, B. Peng, M. Rakowski, S. Rauch,
[11] B. Behroozpour, P. A. M. Sandborn, M. C. Wu, and J. C. Rosenberg, A. Sahin, I. Stobert, and A. Stricker,
B. E. Boser, IEEE Communications Magazine 55, 135 IEEE Journal of Selected Topics in Quantum Electronics
(2017). 25, 1 (2019).
[12] F. Aflatouni, B. Abiri, A. Rekhi, and A. Hajimiri, Opt. [32] Y. Shen, N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-
Express 23, 5117 (2015). Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle,
[13] A. Martin, D. Dodane, L. Leviandier, D. Dolfi, D. Englund, and M. Soljačić, Nature Photonics 11, 441
A. Naughton, P. O’Brien, T. Spuessens, R. Baets, G. Lep- (2017).
age, P. Verheyen, P. De Heyn, P. Absil, P. Feneyrou, and [33] B. Razavi, Solid-State Circuits Magazine 11, 10 (2019).
J. Bourderionnet, Journal of Lightwave Technology 36, [34] J. Y. Wang, Appl. Opt. 21, 464 (1982).
4640 (2018). [35] D. Rebolj, Z. Pučko, N. Čuš. Babič, M. Bizjak, and
[14] D. Inoue, T. Ichikawa, A. Kawasaki, and T. Yamashita, D. Mongus, Automation in Construction 84, 323 (2017).
Opt. Express 27, 2499 (2019). [36] A. Samani, E. El-Fiky, M. Morsy-Osman, R. Li, D. Patel,
[15] C. Li, X. Cao, K. Wu, X. Li, and J. Chen, Opt. Express T. Hoang, M. Jacques, M. Chagnon, N. Abadı́a, and
27, 32970 (2019). D. V. Plant, Journal of Lightwave Technology 37, 2989
[16] M. Collett, R. Loudon, and C. Gardiner, Journal of Mod- (2019).
ern Optics 34, 881 (1987). [37] A. Stove, IEE Proceedings F (Radar and Signal Process-
[17] M. A. Rubin and S. Kaushik, Opt. Lett. 32, 1369 (2007). ing) 139, 343 (1992).
[18] A. El Gamal and H. Eltoukhy, IEEE Circuits and Devices [38] V. Winkler, in 2007 European Radar Conference (2007)
Magazine 21, 6 (2005). pp. 166–169.
[19] B. L. Stann, K. Aliberti, D. Carothers, J. Dammann, [39] H.-S. Chen and C. R. N. Rao, Journal of Physics D: Ap-
G. Dang, M. M. Giza, W. B. Lawler, B. C. Redman, plied Physics 1, 1191 (1968).
and D. R. Simon, in Laser Radar Technology and Appli- [40] Z. Sheng, Z. Wang, C. Qiu, L. Li, A. Pang, A. Wu,
cations IX , Vol. 5412, edited by G. W. Kamerman and X. Wang, S. Zou, and F. Gan, IEEE Photonics Jour-
G. W. Kamerman, International Society for Optics and nal 4, 2272 (2012).
Photonics (SPIE, 2004) pp. 264 – 272. [41] N. C. Harris, Y. Ma, J. Mower, T. Baehr-Jones, D. En-
[20] K. Hu, Y. Zhao, M. Ye, J. Gao, G. Zhao, and G. Zhou, glund, M. Hochberg, and C. Galland, Opt. Express 22,
IEEE Sensors Journal 17, 5547 (2017). 10487 (2014).
[21] C. V. Poulton, A. Yaacobi, D. B. Cole, M. J. Byrd, [42] M. Mendez-Astudillo, M. Okamoto, Y. Ito, and T. Kita,
M. Raval, D. Vermeulen, and M. R. Watts, Opt. Lett. Opt. Express 27, 899 (2019).
42, 4091 (2017). [43] M. G. Ahmed, T. N. Huynh, C. Williams, Y. Wang, P. K.
[22] S. A. Miller, C. T. Phare, Y.-C. Chang, X. Ji, O. A. J. Hanumolu, and A. Rylyakov, IEEE Journal of Solid-
Gordillo, A. Mohanty, S. P. Roberts, M. C. Shin, State Circuits 54, 834 (2019).
B. Stern, M. Zadka, and M. Lipson, in Conference on
 9

[44] S. Shahdoost, A. Medi, and N. Saniei, Analog Integrated

Circuits and Signal Processing 86, 233 (2016).
[45] S. S. Mohan, M. D. M. Hershenson, S. P. Boyd, and
T. H. Lee, IEEE Journal of Solid-State Circuits 35, 346
(2000).
[46] B. Razavi, in 2000 IEEE International Solid-State Cir-
cuits Conference. Digest of Technical Papers (Cat.
No.00CH37056) (2000) pp. 162–163.
[47] A. Romanova and V. Barzdenas, Electronics 8, 1073
(2019).
[48] J. Sun, E. Timurdogan, A. Yaacobi, E. S. Hosseini, and
M. R. Watts, Nature 493, 195 (2013).
[49] A. Y. Piggott, E. Y. Ma, L. Su, G. H. Ahn, N. V. Sapra,
D. Vercruysse, A. M. Netherton, A. S. P. Khope, J. E.
Bowers, and J. Vučković, ACS Photonics 7, 569 (2020).
[50] V. Sorianello, A. Perna, L. Colace, G. Assanto, H. C.
Luan, and L. C. Kimerling, Applied Physics Letters 93,
111115 (2008).
 10

Methods
Design and fabrication. velocity measurements were performed using a series of
fifty 17 µs up-chirps, followed by fifty 17 µs down-chirps,
The demonstration chips used as transmitter and as shown in Extended Data Figure 5(d). The multiple
receiver FPAs were fabricated using GlobalFoundries’ chirps were coherently combined using a two-dimensional
CMS90WG 300 mm silicon photonics process[31], which FFT to extract the beat frequencies, maintaining the
monolithically integrates photonic devices with 90 nm same signal-to-noise ratio as the single-chirp case [37, 38].
silicon-on-insulator (SOI) RF CMOS electronics. All
photonic devices used in the design, with the exception of
the directional couplers, were provided in the foundry’s
Optical setup.
standard process development kit (PDK). By doing so
the photonic architecture had correct-by-construction de-
vice placement and connectivity, verifiable using Mentor A narrow-linewidth (< 100 Hz) fiber laser (NKT Ad-
Graphics’ Calibre Design Rule Checker. The integrated justik) operating at 1550 nm was used as the seed laser for
electronics followed a standard design flow using Cadence the FMCW ranging system. A linear chirp was applied
Virtuoso and Spectre for circuit design and layout, and to the laser light using a silicon photonic IQ modula-
Mentor Graphics’ Calibre for verification of design rules, tor fabricated at the University of Southampton, which
comparing layout-versus-schematic, and extracting par- was driven by a microwave chirp produced by an arbi-
asitics. The two domains are merged into a single hi- trary waveform generator (Tektronix AWG70002A). The
erarchy enabling connectivity verification at the receiver chirped laser light was amplified by erbium-doped fiber
photodiodes along with design rule verification of closely amplifiers (EDFAs) in two stages (a Keopsys CEFA-C-
intertwined photonics and electronics across the chip. HG-PM followed by an NKT Boostik). The amplified
Due to the limited number of dies available from the light was then coupled on-chip via single-mode optical
multi-project wafer shuttle, we were able to fully test the fiber V-grooves into two identical demonstration chips,
functionality of 5 dies. We did not observe any defects used as a transmitter and receiver respectively. The light
such as dead pixels or inoperative thermo-optic switches emitted by the transmitter FPA was structured using a
across these dies. 32x16 microlens array (PowerPhotonic), which was pre-
cisely matched to the receiver array’s pixel pattern. This
created a structured illumination pattern and minimized
Optical chirp scheme. any waste of light due to transmit optical power being
incident on the gaps between pixels.
A linearly chirped optical field E(t) has the form To precisely match the fields of view of the transmitter
and receiver FPAs, we took advantage of the fact that
E(t) = exp i2πf0 t + iπrt2


each receiver grating coupler emits a small amount of LO
= cos πrt2 + i sin πrt2 exp(i2πf0 t),
 
(1) light due to backreflections from the balanced detectors.
where f0 is the carrier frequency, and r is the chirp ramp An infrared camera was then used to align the patterns
rate. Thus, by coherently modulating fixed-frequency of spots produced by the receiver and transmitter FPAs.
light with a microwave chirp of the form cos πrt2 + The physical aperture of the output lens is 25 mm in
i sin πrt2 , we produce a linear chirp in the optical do- diameter. The size of the mode corresponding to a single
main. grating coupler on the receiver focal plane array at this
In our demonstrator system, we use a series of lin- same lens (which can be thought of as the entrance pupil
ear up-chirps immediately followed by a series of down- of our system) is slightly elliptical and is 11 mm along the
chirps. The mean and difference of the up-chirp and first axis and 16 mm along the second axis. For the long
down-chirp beat frequencies allow separate measurement range 75 m measurements, the output lens was adjusted
of range and velocity of a target respectively[11, 26, 27]. such that the transmit and receive beams were essentially
For low-velocity measurements, which includes all the collimated to infinity. For shorter range measurements,
measurements presented here except those in Extended the lens position was adjusted to simulate the effects of
Data Figure 5(f), we use a single up-chirp and single a smaller aperture.
down-chirp, each with a length of 850 µs. This chirp Due to the use of coherent detection, the receiver de-
scheme is illustrated in Extended Data Figure 5(c). The tects a single polarization of scattered light. In coher-
beat frequencies for each chirp were then extracted us- ent LiDAR systems, the receiver polarization is typically
ing fast-Fourier transforms (FFTs). For fast moving ob- chosen to be the same as the transmitter polarization[11,
jects with sufficiently large Doppler shifts, however, the 26, 27] as is done here, since most materials preferentially
beat frequency wraps around zero, resulting in ambigu- scatter light into the same polarization as the illuminat-
ous measurements. To compensate for this effect, high ing light[39].
 11

On-chip electronics. of 10.9 dB. Under typical transmitter operating condi-

tions, there is an extra 2 dB loss from two-photon ab-
As illustrated in Extended Data Fig. 2(a-c), there sorption.
are several levels of on-chip multiplexing in the receiver These losses can be significantly reduced in a straight-
which allow pixels to be read out in blocks of 8 at a forward manner. For example, by employing existing de-
time. The lowest level of multiplexing is achieved by signs for 2 × 2 couplers with a loss of 0.06 dB [40], and
making use of the power switch incorporated into each thermo-optic phase shifters with a demonstrated loss of
pixel’s TIA: only one pixel per row is activated at a time. 0.23 dB [41], the loss could be reduced to 0.35 dB per
The appropriate receiver block is then selected by acti- switch. Meanwhile, two-photon absorption can be min-
vating the set of eight buffer amplifiers associated with imized by using wider waveguides or reverse-biased PIN
that block. A final set of differential output amplifiers junctions[30].
drives eight off-chip 100 Ω loads. An inter-stage RC filter
flattens the frequency response to simplify downstream
Electronic control and signal processing.
signal processing.
The receiver bandwidth of 280 MHz in Fig. 3(a) is the
measured bandwidth through the packaged chip. The A field-programmable gate array (FPGA) with in-
simulated bandwidth of the on-chip amplifier chain is tegrated RF analog-to-digital converters (ADCs) and
750 MHz, suggesting a bandwidth limitation in the test digital-to-analog converters (DACs) was used for both
setup. system control and signal acquisition (Xilinx Zynq Ul-
traScale+ RFSoC). Thermo-optic switch control and re-
ceiver array multiplexing were coordinated by software
running on the system’s ARM Cortex-A53 processing
Thermo-optic switching tree operation.
core. On the signal processing side, the 8 receiver out-
put signals were first digitized in parallel using 8 inte-
Operation of the thermo-optic switching trees is grated ADCs, followed by decimation, application of a
demonstrated in Extended Data Fig. 1. Both the trans- Hann window, FFTs, and peak detection on a custom
mit and receive thermo-optic switch trees on our demon- digital signal processing (DSP) pipeline. Final data pro-
stration chip contained integrated photodiodes to mon- cessing and point cloud reconstruction were performed on
itor the flow of light through the switching trees. To a personal computer. To precisely measure the beat fre-
enable digital control and calibration, the monitor pho- quencies, we performed a least-squares fit of the expected
todiodes were directly connected to off-chip TIAs and lineshape to each peak in the measured power spectral
analog-to-digital converters (Analog Devices AD7091R- density. Target distance and velocity were computed us-
8), and the thermo-optic phase shifters were connected ing the beat frequencies f1 and f2 recorded during the
to off-chip digital-to-analog converters (Analog Devices up- and down-chirps respectively. The distance d is given
AD5391). The switch trees were calibrated one switch at by
a time by adjusting the control voltage to maximize the
optical power in each of the tree outputs. Due to minor c(f1 + f2 )
d= , (2)
thermal cross-talk between the thermo-optic switches, it 4r
was necessary to repeat this process for several iterations and the velocity v is
to converge on an optimal configuration. The average
power consumption of a single thermo-optic switch was λ0 (f1 − f2 )
v= , (3)
40 mW. This leads to an average power consumption of 4
approximately 160 mW and 120 mW for the transmitter
where r is the chirp ramp rate, c is the speed of light,
and receiver switching trees, respectively. The thermo-
and λ0 is the laser wavelength.
optic switching trees were found to be very stable, with
no recalibration required even after several months of op-
eration in an uncontrolled temperature environment. Optical Efficiency.

Ideally, the transmitter illumination pattern should ex-

Transmitter chip losses. actly match the readout pattern, so that only the receiver
pixels currently being read out are illuminated by the
Optical losses in the switching trees are estimated to be transmitter. However, in our current prototype, each
2.1 dB per switch layer, due in large part to sub-optimal transmitter steering position illuminates a field of view
2 × 2 couplers. For the transmitter, 4 layers of switches, of 32 receiver pixels, and 8 receiver pixels are read out
an additional 1 dB of waveguide routing loss, and 1.5 dB at a time. This mismatch was due to a combination of
of grating coupler loss leads to total linear on-chip losses chip area constraints and particularly large 1 mm long
 12

thermo-optic phase shifters, and can be resolved by using differences in on-chip optical waveguide lengths, in ad-
existing designs for compact thermo-optic shifters [42]. dition to subtly differing paths taken through the free
Correcting for the 4× mismatch between the number of space optics by light from different pixels. Since these
transmitter and receiver positions, our equivalent point path length differences are static, they can be eliminated
rate is 2 × 104 per second. Combined with a 4 mW emit- using straightforward calibration measurements.
ter power, this yields an optical efficiency of 0.2 µJ/point. Thus, the key parameter for our system is depth noise,
The combination of high optical efficiency and long range the variation in depth measurements due to stochastic
necessary for autonomous navigation applications[1] is noise in our system. Depth noise was measured by ac-
typically met using mechanically steered LiDARs, such quiring 40 sequential frames of a static test target for the
as the commonly used Velodyne VLP-16. This 100 m histograms shown in Fig. 4(b), and 20 sequential frames
class mechanical LiDAR uses the same 0.2 µJ of light for the depth precision measurements in Extended Data
per point as our system[4], and has a much poorer depth Fig. 5(b). The mean distance value for each pixel was
accuracy of 3 cm. taken to be the true distance, and depth error was de-
fined as the deviation from the true distance for each
pixel. Finally, we defined measurement precision as the
Electro-optic characterization. standard deviation of the depth error.

A 3.5 GHz oscilloscope (Tektronix DPO7354C) was

used for all electro-optic characterization of the receiver Contrast measurement.
array. The amplifier noise floor and shot noise were av-
eraged over a bandwidth of 1 − 3 MHz, avoiding low Imaging contrast for the system was measured us-
frequency 1/f noise from the amplifier, as well as the ing retroreflective sheeting (white 3M Scotchlite 680CR)
relative intensity noise peak of the laser. The measured as a target. The edge of the retroreflective sheet was
shot noise floor was used to determine the exact local os- placed across the field of view of the system, allowing for
cillator power at each receiver pixel, since the shot noise the measurement of pixel-to-pixel crosstalk. The edges
power spectral density depends only upon the photocur- were deliberately placed near the middle of the 8 × 8
rent. The common mode rejection ratio of the receiver pixel receiver blocks to maximize the expected crosstalk.
pixels was measured by modulating the amplitude of the Extended Data Fig. 5(a) illustrates the mean signal
local oscillator light at 10 MHz, and comparing the mea- strength as a function of distance from the retroreflector
sured electrical output amplitude to the expected ampli- edge. This was obtained by averaging the signal strength
tude given the local oscillator power and amplifier gain. of 12 pixels in the direction parallel to the retroreflector
Based on circuit modelling and independent verification edge.
using a test structure on the chip, the total gain of the The imaging contrast is slightly worse for horizontal
amplifier chain was 20 kΩ. The total electrical power con- edges. This is likely due to the greater numerical aperture
sumption of the receiver chip was 250 mW. This includes of the receiver grating couplers in the vertical direction,
all 8 receiver output channels of signal amplification and leading to additional blurring due to lens aberrations in
multiplexing. the vertical direction.

Characterization of measurement accuracy. Transimpedance amplifier comparison.

Measurement error in our 3D imaging system can be Fig. 3(d) plots input-referred noise current density
divided into two categories: systemic errors due to non- against the transimpedance gain-bandwidth product for
idealities in our system, and random fluctuations in the several state-of-the-art CMOS and BiCMOS optical re-
measured beat frequencies due to shot noise, laser rela- ceiver publications[43–47]. A custom design must simul-
tive intensity noise, laser frequency fluctuations, and elec- taneously meet requirements for gain, noise, and band-
tronic noise sources. Systemic errors in our system are width. Generally, the gain-bandwidth product will be
very tightly controlled. Since the frequency chirps in our constant for a target technology and power consumption.
system are generated using direct digital synthesis in an In a resistive shunt-feedback configuration, the input-
AWG, distance accuracy is fundamentally derived from referred current noise in,rms is typically dominated by
the speed of light, a fixed physical constant, and the tim- the feedback resistance RF :
ing accuracy of the clocks in the AWG and ADCs, which r
4kT
are controlled to within a few parts per million. The in,rms = · BW−3dB (4)
RF
only remaining source of systemic error comes from op-
tical path length differences between pixels, which mani- where k is the Boltzmann constant, T is the tempera-
fest as static offsets in measured depth. These are due to ture, and BW−3dB is the 3-dB bandwidth. The gain is
 13

approximately equal to the feedback resistance, and the Pixel Size Reduction.
bandwidth is determined by the pole at the input, where
CT is the total capacitance at the TIA input, and A0 is When considering the maximum resolution sensor that
the open-loop gain of the TIA. can fit on a chip with a given area, the pixel size is the
dominant factor. The switching trees are negligible in
size compared to the receiver FPA if existing compact
1 + A0 designs for thermo-optic phase shifters are used. For ex-
BW−3dB = (5) ample, efficient thermo-optic phase shifters as short as
2π · RF · CT
35 µm in length have been demonstrated [42].
The current pixel size is limited by the use of foundry
PDK devices, which were not designed to minimize foot-
The negative feedback acts to reduce the input print. In terms of individual photonic components, effi-
impedance looking into the TIA. Due to our low band- cient grating couplers[48] with a footprint of 3 × 3 µm2 ,
width requirement (< 1 GHz), and small diode and para- and 2 × 2 couplers[49] as small as 3 × 1 µm2 have been
sitic capacitance at the TIA input, we can use a large re- demonstrated. Meanwhile, photodiodes with a footprint
sistor to get high TIA gain resulting in a reasonable gain- of 3 × 1 µm2 are feasible due to the short absorption
bandwidth product while allowing a low input-referred length of germanium[50]. Lastly, using advanced deep
noise density. Having low-noise electronics improves the sub-micron CMOS technology nodes, the TIA could be
system’s detection probability, providing longer range for reduced to 3 × 3 µm2 in size. This should allow the com-
a given optical power. ponents for a single pixel to fit in an 8 × 5 µm2 footprint.
 14

a b c
Input
Heater 1

Output 1
Output 2

Heater 2
Heater 1

Heater 2

1 2
Outputs

d On Transient Off Transient

Extended Data Figure 1. Thermo-optic switching tree demonstration. (a) The thermo-optic switches consist of a Mach-Zehnder
interferometer with an electrical heater on each arm. (b) Tuning curve for a single thermo-optic switch, showing optical power
in the two outputs as a function of applied heater power. The use of two heaters allows the average electrical power consumption
per switch to be halved. (c) Output power distribution of the 1×16 transmitter switch tree for all switch settings, demonstrating
clean switching. Output power was monitored using a set of monitor photodiodes at the output of the switch tree. (d) On
and off transients for a representative thermo-optic switch, demonstrating 90% − 10% switching times of 9.1 µs and 12.1 µs
respectively. Due to minor thermal crosstalk between switches, the switching transients are not perfect decaying exponentials.
 15

a b
Output driver amplifiers
TX Gratings
Transmitter Ch0 Ch1 Ch2 Ch3 Ch4 Receiver
(+ – )(+ – ) (+ – )(+ – ) (+ – )
...
8 Output Drivers
0 8
RX Block 0 RX Block 4
1 9 3 4
column 0 1 2 5 6 7 row

Row Buffers
1×16 2 10 0 RX Block 1 RX Block 5
TX Switch 1
3 11
2 RX Block 2 RX Block 6
4 12 3
RX Block 0
5 13 4 RX Block 3 RX Block 7
5
6 14 6
1×8
7 15 7 LO Switch

Active RX Pixel Column Row buffer amplifiers

c d
Pixel Array Buffers Drivers
RX Block 0, Row 0 Channel 0 TX Optical Steering TX Grating 0 TX Grating 1 TX Grating 2

RX LO Optical Steering RX Block 0 RX Block 1

RX Block 7, Row 0
RX Pixel & Buffer Enable RX Block 0 RX Block 1

RX Column Select 0 1 7 0 1 7 0
RX Block 0, Row 7 Channel 7
Acquire Pixel Column
RX Block 7, Row 7 Time
Readout 8 pixels
Column 0 1 7 Chip
Optical
Boundary
Active circuits during readout example Electrical

Extended Data Figure 2. Transmitter (TX) and receiver (RX) synchronization and readout architecture. (a) The TX steers
light through a 4 level tree of 1 × 2 switches to feed the focal plane array of 16 output grating couplers. Each leaf contains a
fractional tap and monitor photodiode enabling electronic calibration of the tree. (b) The RX array is divided into 8 blocks of 64
pixels. Imaging an 8-pixel column requires both steering the local oscillator (LO) light to the block and enabling the associated
electronics (pixel column and row buffer amplifiers). Signals from the active pixel column are driven by 8 output amplifiers
for parallel readout. (c) Several levels of multiplexing are used to map 512 pixels down to 8 output channels. An active RX
block has one active pixel per row, with the other disabled pixels within the row presenting high output impedance (no drive
strength). The row buffers are similarly passively multiplexed between the blocks. The 8 drivers are always activated during
readout. (d) Timing diagram showing synchronization between the optical switching trees (TX and RX) and the electrical
readout circuitry.
 16

Receiver
chip

Objective
lens

Lens

Polarizing
beamsplitter

Transmitter Telescope Microlens Telescope Faraday

chip array rotator

Lens

To target

Extended Data Figure 3. Free-space optics schematic of the demonstration system. Much of the complexity in the optical
system is to match the receiver and transmitter focal plane arrays, which can be corrected in the future by adjusting the chip
layouts. For inexpensive consumer versions of the system, the Faraday rotator and polarizing beamsplitter could be replaced
by a 50-50 beamsplitter, at the cost of a 4× reduction in signal strength. Although it is possible to implement this experiment
using a single chip for both transmit and receive functions, we have used two identical chips acting as the transmitter and
receiver respectively to simplify the experimental setup.
 17

a b

Extended Data Figure 4. Far-field infrared camera images of transmitter steering. (a) Images of several representative steering
positions. The receiver fields of view corresponding to the 16 steering positions are indicated by the dashed lines, with the
currently active block indicated by a solid outline. The light from the active transmit grating is first slightly defocused to
completely illuminate the active block, and then structured by the microlens array. Due to this defocusing and the soft edges
of the beam, a small fraction of the transmitted light falls outside of the active block. (b) A zoomed-in image showing the
structured illumination pattern produced by the microlens array. The locations of the bright spots coincide with the receiver
pixel grating couplers.
 18

a b
Vertical edge Horizontal edge

Field of view Field of view

Retroreflector
Retroreflector

Retroreflector Air

Vertical edge
Horizontal edge

Detection threshold

c Single chirp FMCW

e

4
Frequency (GHz)

Up-chirp Down-chirp
850 μs 850 μs

Time

d f
Multi-chirp FMCW
4
Frequency (GHz)

Up-chirps Down-chirps
50 × 17 μs 50 × 17 μs

Time

Extended Data Figure 5. Additional characterization of system performance. (a) Imaging contrast measured using retroreflec-
tive sheeting. Our system achieves > 25 dB contrast for a 1 pixel displacement, > 50 dB contrast for a 4 pixel displacement,
and reaches the system noise floor thereafter, illustrating the excellent pixel-to-pixel isolation in our system. Here, the error
bars represent the standard error. (b) Depth precision and detection probability as a function of distance for a 44% reflectance
target. The error bars on the depth precision represent the 95% confidence intervals. (c, d) Single-chirp (c) and multi-chirp
(d) frequency-modulated continuous-wave waveforms used for measuring slow and fast objects respectively. (e, f) Measured
velocity as a function of rotation rate for a 30 cm diameter styrofoam cylinder at a distance of 17 m using (e) single-chirp and
(f) multi-chirp waveforms, with error bars indicating the standard deviation.