Deep Learning Modeling Approach for Metasurfaces With High Degrees of Freedom
Deep Learning Modeling Approach for Metasurfaces With High Degrees of Freedom
Abstract: Metasurfaces have shown promising potentials in shaping optical wavefronts while
remaining compact compared to bulky geometric optics devices. The design of meta-atoms, the
fundamental building blocks of metasurfaces, typically relies on trial and error to achieve target
electromagnetic responses. This process includes the characterization of an enormous amount
of meta-atom designs with varying physical and geometric parameters, which demands huge
computational resources. In this paper, a deep learning-based metasurface/meta-atom modeling
approach is introduced to significantly reduce the characterization time while maintaining
accuracy. Based on a convolutional neural network (CNN) structure, the proposed deep learning
network is able to model meta-atoms with nearly freeform 2D patterns and different lattice
sizes, material refractive indices and thicknesses. Moreover, the presented approach features the
capability of predicting a meta-atom’s wide spectrum response in the timescale of milliseconds,
attractive for applications necessitating fast on-demand design and optimization of a meta-
atom/metasurface.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
Metasurfaces, the 2D version of metamaterials, provide a novel platform for the realization of
ultra-thin and large-scale optical components and systems. By manipulating the geometry of
individual meta-atoms, desired responses (e.g. phase and amplitude) can be realized at the unit-
cell level over the flat surface for full control of light propagation. The key challenge in the field
of metasurface/meta-atom design is the non-intuitive design process, which makes it difficult to
find optimal design parameters to meet specific requirements. Current design approaches include
trial-and-error methods and inverse design methods based on optimization algorithms or deep
neural networks (DNNs). For the traditional trial-and-error method, a commonly adopted design
process includes complete exploration of the entire design space and careful selection of results
that fit the design requirements. In this case, the design time is determined by the simulation time
of each trial design and the number of design degrees of freedom (DOFs). Therefore, exhaustive
#401960 https://doi.org/10.1364/OE.401960
Journal © 2020 Received 21 Jul 2020; revised 29 Aug 2020; accepted 30 Aug 2020; published 8 Oct 2020
Research Article Vol. 28, No. 21 / 12 October 2020 / Optics Express 31933
exploration of all design parameters is unrealistic without a fast evaluation tool when massive
design DOFs must be taken into consideration (e.g. in freeform meta-atom designs). For inverse
design approaches, which are based on either optimization algorithms [1–4] or neural networks
[5–13], their converging speed and design accuracy largely rely on the simulation speed and
accuracy of the local or commercial solver that is cascaded to the optimizer. One exception is
the recent deep learning metasurface design approach based on generative adversarial networks
(GANs) [14–20], which incorporates a self-evolving Critic, rather than a well-trained DNN
simulator to evaluate the performance of the generated designs. However, since a GAN requires
a noise vector as part of the input, the corresponding output designs have unstable performance
and thus still need to be verified by a simulator [14,17]. Therefore, modeling and characterization
tools play a pivotal role in almost all current metasurface/meta-atom design approaches. In this
context, reliable and time-efficient modeling tools are being heavily investigated to facilitate the
design of next generation meta-devices often characterized by large numbers of design DOFs and
multi-functionality.
One approach being considered is the development of analytical effective medium models,
such as the Lewin model [21] and the GEM model [22]. Although these models are simple
and efficient, their application is limited to metamaterials with spherical inclusions under the
long-wavelength approximations. Another widely adopted approach relies on iterative numerical
full-wave simulations based on different methods, including FEM (finite-element method), FDTD
(finite-difference time-domain) and FIT (finite integration technique). This approach provides
accurate results but requires considerable computing resources. A data-driven modeling tool
based on deep neural networks (DNNs) [9,11,13,23] emerged recently and has been proven
to be both accurate and time-efficient. Previous work has employed fully-connected layers
(FCLs) to realize accurate spectral response predictions for plasmonic [5,9,10] or all-dielectric
[6,8,12,13] nanophotonic structures with bulk layers [6], cavities [7] and meta-atoms in the
shape of cylinders [8,9,12,13,24], elliptic cylinders [25], spheres [11] and bars [5,10]. While
most works [5,6,8–11,25] has been focused on amplitude response predictions, recent work
[13] has demonstrated that neural networks are also capable of predicting meta-atoms’ phase
responses, which is crucial to the design of phase-gradient meta-optical devices. After being
fully-trained with sufficient data, the DNN models are highly accurate and are able to generate
electromagnetic (EM) responses on the time scale of milliseconds, which enables fast, on-demand
meta-atom/metasurface designs. However, there are remaining issues with these existing networks
constructed with FCLs. Firstly, these works mainly deal with simple meta-atom structures that
are described by only 3-5 parameters, which heavily constrains the accessible meta-atoms design
DOF. Secondly, these DNN models are operating in a very restricted design space. Design
parameters including lattice sizes, meta-atom thicknesses and material properties are fixed in
these networks. Once one of these design parameters falls outside the design space, a new dataset
needs to be collected and the model must be re-trained, a process which can be time-consuming.
The current DNN approach is therefore inadequate as a general design scheme.
In this paper, we present a new DNN approach for the modeling and characterization of
three-dimensional (3D) meta-atoms, which addresses the issues and limitations discussed above.
To expand the design space and demonstrate the network’s generality, our approach considers
high DOFs structures and accounts for the meta-atom’s two-dimensional (2D) geometrical
pattern, material refractive index, thickness and lattice size. After being trained with sufficient
data, the proposed network is able to generate accurate phase and amplitude predictions of
meta-atoms with complex shapes across a wide spectrum. Furthermore, we demonstrated our
network’s generalizability by testing it with meta-atoms which contain features that do not
exist in the training data. To show the efficacy of the proposed method, it has been applied for
practical metasurface/meta-device design and optimization. The performance of the resulting
metasurface/meta-device prototypes corroborates that the network achieves two important
Research Article Vol. 28, No. 21 / 12 October 2020 / Optics Express 31934
features for DNN-based meta-atom modeling: 1) fast and accurate performance evaluation of
geometrically complex meta-atoms and metasurfaces; and 2) a modeling tool that covers an
extensive design space of 3D meta-atoms. It is envisioned that the proposed deep learning
network can be readily applied to various meta-atom modeling and optimization tasks, as well
as be extended to other fields such as the characterization and design of dielectric resonator
antennas, optical circuits and chiral metamaterials.
Fig. 1. Network architecture. Meta-atom design parameters were processed through two
separate input neural networks, divided into a 1D property vector (circled in yellow) and 64
by 64 pixel 2D images (circled in blue). 2D images were processed with two convolution
layers and then combined with the 1D properties (with the size of 8 × 8 × 64 through
spatial tiling). Combined results (8 × 8 × 128) were further processed with convolution and
pooling layers then flattened into a 1D array (1 × 1024). After being processed with 2 more
FCLs, the real/imaginary parts (1 × 51) of the transmission coefficient over the 30-60 THz
spectrum were ready for evaluation. All convolution layers in the network are followed by a
batch normalization layer. More detailed network architecture can be found in section I in
Supplement 1.
We demonstrate the validity of the well-trained PNN with two meta-atom samples (Fig. 2(a))
that were randomly selected from the test dataset. Their transmissive amplitudes (blue dots)
and phase responses (red dots) were evaluated and compared with the results derived from
FEM-based simulations (blue and red curves). As indicated by the minimal test error, the PNN
prediction results agreed well with the full-wave electromagnetic simulations. Moreover, to
verify the PNN’s applicability to new, previously unseen data, we tested the PNN with meta-atom
designs possessing features that did not exist in either the training or test datasets. This included
meta-atoms with the shapes of circles (Fig. 2(b)) and rings (Fig. 2(c)), which cannot be obtained
using the needle drop method adopted here. The radii defining the circles and rings in Figs. 2(b)
and 2(c) were randomly generated. The patterns showing as insets in Figs. 2(b) and 2(c) are
pixelated images that can best describe the generated circles and rings with a resolution of 64
by 64 pixels. Other parameters, including refractive index, meta-atom thickness and lattice
size, were randomly selected within the preset parameter range. Similarly, as in Fig. 2(a), the
amplitude and phase responses of both designs in Figs. 2(b) and 2(c) were evaluated using the
PNN (dots), and then compared with the full-wave electromagnetic simulation results (curves).
The PNN maintained its accuracy even with those meta-atoms that have previously unseen
features, indicating its broad generalization ability. More importantly, once trained with enough
data, the proposed PNN can produce the predictions in milliseconds, which makes it an appealing
Research Article Vol. 28, No. 21 / 12 October 2020 / Optics Express 31936
substitute for conventional simulation tools for applications such as high-throughput device
evaluation and optimization.
Fig. 2. Examples of PNN predictions compared to full-wave simulation results. (a) PNN
predictions on meta-atoms selected from the test dataset. (b) PNN predictions on circle-
shaped meta-atoms. (c) PNN predictions on ring-shaped meta-atoms. Blue curves represent
the PNN predictions, while red curves are simulation results. Refractive index, meta-atom
thickness and lattice size are shown on top-right corner of each subplot, respectively (lengths
in µm). 2D cross sections of each meta-atom are included as insets. Additional PNN
prediction results can be found in Supplement 1, Section V.
3. Discussion
The well-trained PNN can be adopted in various application scenarios. Below, we discuss
the possibility of utilizing this fast and accurate modeling tool to solve problems, including
meta-atom inverse designs and meta-device optimizations.
overall efficiency and phase coverage of all-dielectric meta-atoms. In Fig. 3, four meta-atom
pattern datasets of circles, rectangles, “H”-shapes, and quasi-freeform shapes were constructed,
each containing 3,000 randomly generated patterns. Each pattern was pixelated into a 64 by
64 pixel image (Fig. 3(a)), which is in accordance with the input dimensions of the PNN. We
compare the performance of these meta-atoms, as predicted by the PNN, at two different working
frequencies, where materials, thicknesses and lattice sizes are considered as design variables.
In these two cases (Figs. 3(b) and 3(c)), the meta-atoms with quasi-freeform cross sections are
clearly superior to the meta-atoms from the other three categories in terms of both efficiency
and phase coverage. It is apparent from the superior amplitude/phase coverage obtained by the
complex meta-atom shapes that the extra design DOFs would largely improve the performance of
metasurfaces, especially multifunctional metasurfaces where a single meta-atom must provide
good performance for multiple conditions such as changes in frequencies, polarizations, materials,
etc. This further highlights the need to model freeform metasurfaces and the efficacy of the
proposed DNN approach. Below we discuss several practical applications enabled by this
Fig. 3. Comparing the performance of meta-atoms with different shapes using the PNN. (a)
Randomly generated pixelated patterns (with the resolution of 64 by 64) in the shape of circles,
rectangles, “H”s and quasi-freeforms. (b) Transmissive responses of meta-atoms at 60 THz,
with index = 3.7, thickness = 0.8 µm and lattice size = 3 µm. (c) Transmissive responses of
meta-atoms at 58 THz, with index = 5, thickness = 0.7 µm and lattice size = 2.5 µm. (d)
Examples of randomly-generated patterns for each shape.
Research Article Vol. 28, No. 21 / 12 October 2020 / Optics Express 31938
approach that are prohibitively time-consuming using conventional methods (e.g. full-wave
simulations).
Fig. 4. Meta-atoms’ EM performance evaluated using the PNN. (a) Phase and amplitude
coverage with fixed index (n = 4.5), lattice size (2.6 µm) and changing thicknesses. (b) Phase
and amplitude coverage with fixed index (n = 5) and thickness (0.7 µm) but changing lattice
sizes. Areas that are sparsely populated by high-efficiency candidates are circled in red
dotted lines. Working frequency is set to be 57 THz in all six cases.
Research Article Vol. 28, No. 21 / 12 October 2020 / Optics Express 31939
between a meta-atom’s intrinsic electric and magnetic dipoles and reduce the overall phase
coverage. To address this issue, we randomly generated over 20,000 quasi-freeform meta-atom
patterns and combined them with different refractive indices, thicknesses and lattices sizes, and
evaluated their performance at 57 THz (5.26 µm in wavelength) using the proposed PNN. As
shown in Fig. 4(a), we fixed the index to be 4.5 and lattice size to be 2.6 µm, and evaluated the
influence of varying thickness on the phase coverage of the same group of meta-atom patterns. It
is observed that the phase coverage increased at first but started to decrease when the thickness
continued to increase. When the thickness is equal to 1 µm, it’s hard to pick a group of high
efficiency meta-atoms in the area highlighted in the red dashed line. Similarly, in Fig. 4(b), when
fixing the index to be 5 and thickness to be 0.7 µm, the meta-atom phase coverage drops as the
lattice size increases from 2.6 µm to 3 µm. In both cases, the proper meta-atom design parameter
combinations with maximum phase coverage can be identified in seconds using the presented
PNN, which highlights its efficacy in searching for new meta-atom designs.
Fig. 5. Metalens optimization using the well-trained PNN. Four different metalenses
are derived using the GAN-PNN optimization network and then evaluated with full-wave
simulations using the time domain solver in FEM simulation tool, CST Microwave Studio.
The inset 2D images are of the E-field on the plane perpendicular to the metasurface. 1D
curves are the E-field along white dotted lines in each subplot. Full-wave simulation results
show that the peak E-field increased with more optimization steps.
Research Article Vol. 28, No. 21 / 12 October 2020 / Optics Express 31940
process. The proposed PNN for quasi-freeform-shaped metasurfaces can significantly alleviate
the issue due to its fast prediction capability.
As a demonstration, we employed a well-trained GAN model [17] to design a transmissive
meta-lens working at 57 THz. The lens has a focal length of 80 µm, a numerical aperture (NA)
of 0.63, and is composed of 50 by 50 dielectric meta-atoms (refractive index 4.7 and thickness
0.75 µm) on a dielectric substrate (refractive index 1.4) with a lattice size of 2.6 µm (aka a total
lens size of 130 µm by 130 µm). After the phase mask of this lens was calculated, we trained
a generative meta-atom design network [17] to generate a meta-atom design for each unit cell
within the metalens. The EM responses of the generated meta-atoms were then evaluated using
the proposed PNN. This cascaded design-evaluate process was executed for several iterations so
that the best meta-atom designs generated during these iterations were selected to assemble the
final device. To verify the prediction accuracy of the PNN and the efficacy of this design-evaluate
process, we employ this cascaded network (GAN + PNN) to run 1, 2, 5 and 10 iterations to
generate 4 different meta-lenses (shown in Supplement 1, Section VI), and then tested their
performance using full-wave simulations. The metalenses are situated in the x-y plane, with the
optical axis along the z-axis. The simulated 2D electric fields of four different metalenses in
the x-z plane, along with the 1D electric field along the x axis in the focal plane, are plotted
in Fig. 5. The peak electric field amplitude at the center of the focal spot was improved with
increasing optimization iterations, validating that meta-atom designs with better performance
(corresponding to higher transmission and more precise phase shift) have been identified with the
help of the proposed PNN during the optimization iterations. Importantly, with this data-driven
approach, time taken for the performance evaluation process is dramatically reduced and becomes
comparable with the design generation time, thereby solving the problem of computational
overhead for verification that limits existing DNN-based approaches. A detailed time efficiency
contrast analysis is included in Section VII in Supplement 1.
4. Conclusion
In this paper, a deep learning-based dielectric meta-atom modeling approach is proposed and
demonstrated. Accurate spectrum responses of dielectric meta-atoms were derived using a
novel CNN-based network structure. Compared with previous work, our approach can handle a
significantly larger set of input parameters, including various shapes, thickness, lattice size and
refractive index of the meta-atoms. The proposed deep learning network processes enable strong
generalizability and make accurate predictions for meta-atoms with features that do not exist in
the training dataset. We have further demonstrated that the presented network can be adopted as
an efficient modeling tool in various application scenarios, including rapid meta-atom design
and meta-device optimization. The proposed DNN-based methodology presented herein can be
readily applied to other physics domains where a fast, accurate modeling tool is highly desired to
provide the link between a broad and sophisticated parametric space and corresponding physical
responses.
Funding
Defense Advanced Research Projects Agency (HR00111720029).
Disclosures
The authors declare no conflicts of interest.
Data Availability
The all-dielectric meta-atom dataset used during training is open source and available at
https://github.com/SensongAn/Meta-atoms-data-sharing. This work also participates in the
MetaNet [32] data sharing project and will be available at http://metanet.stanford.edu/ soon.
References
1. D. Z. Zhu, E. B. Whiting, S. D. Campbell, D. B. Burckel, and D. H. Werner, “Optimal high efficiency 3D plasmonic
metasurface elements revealed by lazy ants,” ACS Photonics 6, 2741–2748 (2019).
2. J. Jiang and J. A. Fan, arXiv preprint arXiv:1906.07843 (2019).
3. J. Jiang and J. A. Fan, arXiv preprint arXiv:1906.04157 (2019).
4. Z. Liu, L. Raju, D. Zhu, and W. Cai, arXiv preprint arXiv:1902.02293 (2019).
5. I. Malkiel, M. Mrejen, A. Nagler, U. Arieli, L. Wolf, and H. Suchowski, “Plasmonic nanostructure design and
characterization via Deep Learning,” Light: Sci. Appl. 7(1), 60 (2018).
6. D. Liu, Y. Tan, E. Khoram, and Z. Yu, “Training Deep Neural Networks for the Inverse Design of Nanophotonic
Structures,” ACS Photonics 5(4), 1365–1369 (2018).
7. Z. Liu, D. Zhu, K.-T. Lee, A. S. Kim, L. Raju, and W. Cai, arXiv preprint arXiv:1907.03366 (2019).
8. L. Gao, X. Li, D. Liu, L. Wang, and Z. Yu, “A bidirectional deep neural network for accurate silicon color design,”
Adv. Mater. 31, e1905467 (2019).
9. X. Li, J. Shu, W. Gu, and L. Gao, “Deep neural network for plasmonic sensor modeling,” Opt. Mater. Express 9(9),
3857–3862 (2019).
10. W. Ma, F. Cheng, and Y. Liu, “Deep-Learning-Enabled On-Demand Design of Chiral Metamaterials,” ACS Nano
12(6), 6326–6334 (2018).
11. J. Peurifoy, Y. Shen, L. Jing, Y. Yang, F. Cano-Renteria, B. G. DeLacy, J. D. Joannopoulos, M. Tegmark, and M.
Soljačić, “Nanophotonic particle simulation and inverse design using artificial neural networks,” Sci. Adv. 4(6),
eaar4206 (2018).
12. K.-F. Lin, C.-C. Hsieh, S.-C. Hsin, and W.-F. Hsieh, “Achieving high numerical aperture near-infrared imaging based
on an ultrathin cylinder dielectric metalens,” Appl. Opt. 58(32), 8914–8919 (2019).
13. S. An, C. Fowler, B. Zheng, M. Y. Shalaginov, H. Tang, H. Li, L. Zhou, J. Ding, A. M. Agarwal, C. Rivero-Baleine,
K. A. Richardson, T. Gu, J. Hu, and H. Zhang, “A Deep Learning Approach for Objective-Driven All-Dielectric
Metasurface Design,” ACS Photonics 6(12), 3196–3207 (2019).
14. Z. Liu, D. Zhu, S. P. Rodrigues, K.-T. Lee, and W. Cai, “Generative Model for the Inverse Design of Metasurfaces,”
Nano Lett. 18(10), 6570–6576 (2018).
15. W. Ma, F. Cheng, Y. Xu, Q. Wen, and Y. Liu, “Probabilistic representation and inverse design of metamaterials based
on a deep generative model with semi-supervised learning strategy,” arXiv preprint arXiv:1901.10819 (2019).
16. J. Jiang, D. Sell, S. Hoyer, J. Hickey, J. Yang, and J. A. Fan, “Free-Form Diffractive Metagrating Design Based on
Generative Adversarial Networks,” ACS Nano 13(8), 8872–8878 (2019).
17. S. An, B. Zheng, H. Tang, M. Y. Shalaginov, L. Zhou, H. Li, T. Gu, J. Hu, C. Fowler, and H. Zhang, arXiv preprint
arXiv:1908.04851 (2019).
18. S. So and J. Rho, “Designing nanophotonic structures using conditional deep convolutional generative adversarial
networks,” Nanophotonics 8(7), 1255–1261 (2019).
19. X. Han, Z. Fan, C. Li, Z. Liu, and L. J. Guo, arXiv preprint arXiv:1912.03696 (2019).
20. H. P. Wang, Y. B. Li, H. Li, S. Y. Dong, C. Liu, S. Jin, and T. J. Cui, “Deep learning designs of anisotropic
metasurfaces in ultrawideband based on generative adversarial networks,” Advanced Intelligent Systems, 2000068
(2020).
21. L. Lewin, “The electrical constants of a material loaded with spherical particles,” J. Inst. Electr. Eng. 3 94(27), 65–68
(1947).
22. B. A. Slovick, Z. G. Yu, and S. Krishnamurthy, “Generalized effective-medium theory for metamaterials,” Phys. Rev.
B 89(15), 155118 (2014).
23. P. R. Wiecha and O. L. Muskens, “Deep Learning Meets Nanophotonics: A Generalized Accurate Predictor for Near
Fields and Far Fields of Arbitrary 3D Nanostructures,” Nano Lett. 20(1), 329–338 (2020).
24. R. Lin, Y. Zhai, C. Xiong, and X. Li, “Inverse design of plasmonic metasurfaces by convolutional neural network,”
Opt. Lett. 45(6), 1362–1365 (2020).
25. Y. Kiarashinejad, M. Zandehshahvar, S. Abdollahramezani, O. Hemmatyar, R. Pourabolghasem, and A. Adibi, arXiv
preprint arXiv:1909.07330 (2019).
26. R. Socher, D. Chen, C. D. Manning, and A. Ng, “Reasoning with neural tensor networks for knowledge base
completion,” in Advances in neural information processing systems (2013), pp. 926–934.
27. S. Levine, P. Pastor, A. Krizhevsky, J. Ibarz, and D. Quillen, “Learning hand-eye coordination for robotic grasping
with deep learning and large-scale data collection,” Int. J. Robot. Res. 37(4-5), 421–436 (2018).
28. A. Arbabi, Y. Horie, M. Bagheri, and A. Faraon, “Dielectric metasurfaces for complete control of phase and
polarization with subwavelength spatial resolution and high transmission,” Nat. Nanotechnol. 10(11), 937–943
(2015).
Research Article Vol. 28, No. 21 / 12 October 2020 / Optics Express 31942
29. P. Genevet, F. Capasso, F. Aieta, M. Khorasaninejad, and R. Devlin, “Recent advances in planar optics: from
plasmonic to dielectric metasurfaces,” Optica 4(1), 139–152 (2017).
30. L. Zhang, J. Ding, H. Zheng, S. An, H. Lin, B. Zheng, Q. Du, G. Yin, J. Michon, Y. Zhang, Z. Fang, M. Y. Shalaginov,
L. Deng, T. Gu, H. Zhang, and J. Hu, “Ultra-thin high-efficiency mid-infrared transmissive Huygens meta-optics,”
Nat. Commun. 9(1), 1481 (2018).
31. K. E. Chong, I. Staude, A. James, J. Dominguez, S. Liu, S. Campione, G. S. Subramania, T. S. Luk, M. Decker, and
D. N. Neshev, “Polarization-Independent Silicon Metadevices for Efficient Optical Wavefront Control,” Nano Lett.
15(8), 5369–5374 (2015).
32. J. Jiang, R. Lupoiu, E. W. Wang, D. Sell, J. P. Hugonin, P. Lalanne, and J. A. Fan, “MetaNet: a new paradigm for
data sharing in photonics research,” Opt. Express 28(9), 13670–13681 (2020).