J. Microelectron. Manuf.

3, 20030301 (2020)
doi: 10.33079/jomm.20030301

Study of Inverse Lithography Approaches based on Deep Learning

Xianqiang Zhang1, Xu Ma1,*, Shengen Zhang1, Yihua Pan1, Gonzalo R. Arce2
1
Key Laboratory of Photoelectronic Imaging Technology and System of Ministry of Education
of China, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081,
China
2
Department of Electrical and Computer Engineering, University of Delaware, Newark, DE,
19716, USA

Abstract: Computational lithography (CL) has become an indispensable technology to improve

imaging resolution and fidelity of deep sub-wavelength lithography. The state-of-the-art CL
approaches are capable of optimizing pixel-based mask patterns to effectively improve the degrees
of optimization freedom. However, as the growth of data volume of photomask layouts,
computational complexity has become a challenging problem that prohibits the applications of
advanced CL algorithms. In the past, a number of innovative methods have been developed to
improve the computational efficiency of CL algorithms, such as machine learning and deep
learning methods. Based on the brief introduction of optical lithography, this paper reviews some
recent advances of fast CL approaches based on deep learning. At the end, this paper briefly
discusses some potential developments in future work.

Keywords: Computational lithography, inverse lithography technology (ILT), optical proximity

correction (OPC), deep learning.

1. Introduction the photomask patterns. ILT regards the mask pattern

as a binary pixelated image, where the zero-valued
Optical lithography is a crucial technology to and one-valued pixels represent opaque and
manufacture the integrated circuits (IC) in transparent regions, respectively. Figure 1(b)
semiconductor industry. Figure 1(a) shows the presents an illustration of ILT method [4]. Pixel-based
schematic diagram of a typical deep-ultraviolet ILT greatly improves the degrees of freedom in mask
(DUV) optical lithography system, which is used to optimization, so it can effectively improve the
transfer the IC layouts from photomask onto wafer [1, imaging performance of lithography systems.
2] In the past, researchers have proposed a number
. The illumination of DUV lithography system
emits the light rays with 193nm wavelength, which of gradient-based algorithms to solve the ILT
pass through the optical lens and uniformly problems [6, 7]. For instance, Liu et al. compensated
illuminate the photomask. The transmitted light rays the image distortion for both binary mask and phase-
from mask are collected by the projection optics, and shifting mask using the branch and bound algorithm,
then form the aerial image on the wafer. On the top as well as the simulated annealing algorithm [8].
surface of wafer there is a thin layer of photo- Sherif et al. proposed a binary mask optimization
sensitive material, namely photoresist that is exposed method for incoherent diffraction-limited imaging
and developed. Finally, the IC layout pattern is system, where the problem was formulated as a
replicated on the wafer after etch process. mixed linear integer program (MLIP), and then the
As the critical dimensions (CD) continuously branch and bound method was used to solve the
shrink, computational lithography has been widely problem [9]. Granik et al. formulated ILT as nonlinear,
used to improve the resolution of wafer image and constrained minimization problems over a domain of
extend the life of Moore’s Law [3]. Computational mask pixels, and then applied local variation and
lithography refers to a set of technologies that design gradient descent methods to quickly solve ILT
and optimize lithography systems and processes problems [4]. Poonawala et al. proposed a set of
through mathematical and algorithmic approaches. gradient-based algorithms and regularization
Inverse lithography technology (ILT) is a methods to solve the ILT problem in coherent
representative computational lithography approach lithography imaging system [7]. However, traditional
that compensates image distortion by pre-warping gradient-based ILTs have to face with the great

*
Address all correspondence to Xu Ma, E-mail: [email protected]

1
Zhang et al.: Study of Inverse Lithography Approaches based on Deep Learning

Source (λ)

Condensor Target Layout ILT Mask

Mask

Projector

θmax Print Image Print Image

Photoresist

Wafer

(a) (b)
Figure 1. (a) The sketch of deep ultraviolet optical lithography system, and (b) the illustration of ILT method
(revised from Fig. 1 in Ref. [5]).

challenges of large amount of computation and low learning.

efficiency [10]. The rest of this paper is organized as follows.
In order to overcome the computational Section 2 summarizes the ILT methods based on
complexity, many machine learning techniques have standard deep learning approaches, and Section 3
been applied to accelerate the ILT algorithms [11]. describes and discusses several ILT methods based
Wang et al. used machine learning to efficiently on a radically new learning method, namely model-
generate sub-resolution assist features (SRAF) on driven convolution neural network (MCNN). The
full-chip layout at 20nm technology node, and paper will be concluded in Section 4.
achieved a high imaging accuracy [12]. Guajardo et al.
used machine learning methods to jointly optimize 2. ILT based on Standard Deep
the main features (MF) and SRAFs [13]. Ma et al. Learning
proposed fast mask optimization algorithms based on
non-parametric kernel regression, which can Deep learning has been used to solve a series of
effectively improve the computational efficiency and problems in computational lithography, for example
mask manufacturability [14]. Xu et al. proposed a fast the defect characterization and classification of
SRAF generation method that involved support masks based on convolutional neural networks [19],
vector machines (SVM) and logistic regression and hotspots correction based on the cycle-consistent
models in the complete mask optimization process generative adversarial network [20]. Lan et al.
[15] proposed a new technique to apply deep neural
. K. Luo et al. and R. Luo et al. respectively
proposed fast mask optimization methods based on networks in GPU-accelerated mask optimization
SVMs [16] and multilayer perceptual neural networks platform, which provided a fast and accurate ILT
[17]
. solution for 10nm and below technology nodes [21].
Due to the high nonlinearity of ILT problem, Shi et al. proposed an optimal feature vector
traditional machine learning methods have their automatic design method based on convolution
inherent limitations. For instance, traditional neural network (CNN), which greatly improved the
machine learning methods often require a large computational efficiency of ILT [22]. Chen et al. used
number of training samples to accurately construct Auto Pattern Selection (APS) tool to train the
the nonlinear mapping between the IC layout and the Newron SRAF deep learning network and
corresponding ILT solution [10]. In the latest decade, successfully realized the inverse mask optimization
deep learning has become the forefront of fast ILT on full-chip layout [23]. As examples, this section will
approaches, since it can properly fit any complex detail two ILT methods based on variational
nonlinear function [18]. This paper will describe and autoencoder (VAE) [24] and generative adversarial
discuss in detail several ILT methods based on deep network (GAN) [25].

J. Microelectron. Manuf. 3, 20030301 (2020) 2

Zhang et al.: Study of Inverse Lithography Approaches based on Deep Learning

Mean Vector
Input Encoder Decoder Output
Mask Mask

Print Image Print Image

Sampled
Input Latent Vector

Standard Deviation Vector

(a) Architecture of VAE
Optimized Mask Print Image

(b) Control points (c) Result of VAE

Figure 2. The (a) architecture of VAE, (b) control points, and (c) the optimized mask and print image obtained
by VAE method (revised from Figs. 1, 7 and 10 in Ref. [24]).

to the optimal latent variable Figure 2(c) shows the

2.1. ILT using VAE
optimized mask and the corresponding print image
In 2018, Zhang et al. proposed a mask design obtained by the VAE method. Although VAE
method based on a widely used deep learning improves the efficiency of mask design process, it is
framework, i.e., variational autoencoder [26]. As necessary to collect a large number of training data
shown in Fig. 2(a), VAE is composed of an encoder through other methods.
and a decoder. In the VAE method, a large number 2.2. ILT using GAN
of mask patterns are used as the training data set,
In 2018, Yang et al. proposed a mask
which can be obtained in advance through other
optimization method, also called optical proximity
methods. Here, the mask patterns of the training data
correction (OPC), based on a generative adversarial
are obtained by adding or removing rectangular
network model to improve the imaging performance
features at some control points on the target layout,
of lithography system [27]. This method modifies the
as shown in Fig. 2(b). The network regards the mask
conventional GAN generator by using an auto-
and the corresponding print image as the input data
encoder, which is composed of an encoder and a
pair, which is transferred to the encoder to obtain
decoder, as shown in Figs. 3(a) and 3(b). The
latent variables that satisfy the Gaussian distribution.
modified generator can learn the nonlinear mapping
Then, the latent variables are sampled from the
between the target layout and OPC solution. The
Gaussian distribution, and processed by the decoder
discriminator in Fig. 3(c) is responsible for
to obtain the output data pair. At the beginning, the
distinguishing the true dataset from the OPC
VAE network is trained to learn the relationship
solutions emulated by the generator. In order to
between the mask patterns and their corresponding
further improve the prediction capacity of generator,
print images. This is implemented by minimizing the
the GAN-OPC network is pre-trained by an ILT-
distance between the input of encoder and the output
guided method. After the network is trained, the
of decoder. If a new latent variable is obtained by
OPC solution can be obtained by inputting the target
sampling the Gaussian distribution, the decoder will
layout into the generator, followed by a refinement
generate a new data pair. After that, the optimal
process via a gradient-based ILT method, as shown
latent variable is found to minimize the error
in Fig. 4(a). Figure 4(b) illustrates the simulation
between the print image and the target layout, and
results of the GAN-OPC method.
then the optimized mask will be calculated according

J. Microelectron. Manuf. 3, 20030301 (2020) 3

Zhang et al.: Study of Inverse Lithography Approaches based on Deep Learning

Target Layout Target OPC

1.62 3.83 3.15 Layout Solution

Encoder
Modify

Decoder
0.2 0.8
Bad OPC Good OPC
Solution Solution

OPC Solution
(a) Generator of conventional GAN (b) Generator of GAN-OPC (c) Discriminator of GAN-OPC
Figure 3. The (a) generator of conventional GAN, the (b) generator of the proposed GAN-OPC method, and the
(c) discriminator of the proposed GAN-OPC method (adopted from Figs. 3 and 4 in Ref. [25]).

Quasi-optimal Optimal OPC

Target Layout
OPC Solution Solution

Gradient-
Generator
based ILT

(a) GAN-OPC flow

Target Layout OPC Solution Print Image

(b) The simulation results

Figure 4. The (a) workflow and the (b) simulation results of the proposed GAN-OPC method (adopted from
Figs. 6 and 8 in Ref. [25]).

3. ILT based on Model-Driven Deep inherited from existing deep learning architecture,
Learning but derived from a general inverse optimization
model. In addition, MCNN provides a systematic
The two methods described in Section 2 were initialization method for the network parameters
migrated from the standard deep learning networks, based on the mathematical model of optimization
which were modified slightly to adapt to mask problem. That is where the network’s name came
optimization problem. This section describes the ILT from. In this approach, the network structure of
methods developed recently based on a new kind of MCNN was constructed by unfolding and truncating
deep learning approach called MCNN. the SD-ILT algorithm [28], as shown in Figs. 5(a) and
5(b). The network parameters were systematically
3.1. Model-Driven Convolution Neural Network initialized according to the imaging model of
In 2018, Ma et al. introduced the principles of lithography system. As shown in Fig. 5(c), the
MCNN to computational lithography realm. The lithography imaging model was used as a decoder,
MCNN was used to provide an initial guess of ILT which allows to train the MCNN in an unsupervised
solution for a given layout pattern, and then the manner. The unsupervised training method is
steepest descent (SD) algorithm can be used to refine beneficial to avoid the time-consuming labelling
the mask pattern and further reduce the lithography process that plagues many machine learning
image distortion [5]. The MCNN network is not architectures. The MCNN leads to much faster

J. Microelectron. Manuf. 3, 20030301 (2020) 4

Zhang et al.: Study of Inverse Lithography Approaches based on Deep Learning

S

M D T W M̂

(a) The SD-ILT algorithm

Unfolding Truncating

S1 S K

M1g M b2 M bK M Kg
M1b =Z D1 T1 W1 sig m (•) D K TK WK sig m (•) M̂

(b) The encoder of MCNN

2 Z
E Lithography System

(c) The decoder of MCNN

Figure 5. (a) The flowchart of SD-ILT algorithm, (b) the encoder of MCNN, and (c) the decoder of MCNN
(revised from Figs. 3 and 4 in Ref. [5]).
SD MCNN
1000 10 10
MCNN
20 20
Mask
800 SD
Pattern Error

30 30
600 40 40
50 50
10 20
(b)30 40 50 10 20(c)
30 40 50
400
Print Image

10 10
200 20 20
30 30
0
0 200 400 600 800 1000 40 40
50 50
(a) The convergence curves 10 20 30 40 50 10 20 30 40 50
(d)PE=68 (e)PE=32
Figure 6. Comparison between MCNN and SD approaches (revised from Figs. 10 and 11 in Ref. [5]).

convergence than the SD algorithm, and its pattern imaging model serves as the decoder. In addition, the
error (PE) is smaller as shown in Fig. 6 DMDL can effectively alleviate the gradient
vanishing problem and extend the depth of network,
3.2. Dual-Channel Model-Driven Deep Learning which greatly improves its prediction capacity. It
was proven that for simple layout patterns, DMDL
Recently, Ma et al. generalized the prior MCNN method was capable of obtaining the ILT solution
method to a dual-channel model-driven deep directly, and did not require the subsequent
learning (DMDL) method. The DMDL approach refinement process. As shown in Fig. 8, the DMDL
outperforms traditional ILT algorithms in terms of approach can achieve higher image fidelity
both computational efficiency and image fidelity [29]. compared to the traditional SD method.
Similar to MCNN, the network structure of DMDL
is derived from the SD-ILT model as shown in Figs. 4. Conclusion and Discussion
7(a) and 7(b). However, DMDL approach formulates
the mask pattern as the superposition of MFs and This paper briefly described the concepts of
SRAFs. Thus, the DMDL network divides the data computational lithography, and then reviewed the
flow into two parallel channels as shown in Fig. 7(b), development of some ILT lgorithms. This paper
which are used to predict the optimization results of focused on the description and discussion on the fast
MFs and SRAFs, respectively. Therefore, DMDL ILT methods based on deep learning. Due to the
method can successfully insert SRAFs on mask to length limitation, we selected a set of representative
improve the image fidelity of lithography system. As methods to introduce. Deep learning brings
shown in Fig. 7(c), the DMDL approach uses an opportunities for the advances of novel
unsupervised training strategy, where the lithography computational lithography methodologies. In the

J. Microelectron. Manuf. 3, 20030301 (2020) 5

Zhang et al.: Study of Inverse Lithography Approaches based on Deep Learning

2
Z 2
E
Channel 1 Channel 2

M1sb M1mb
M 0g
sig m ()
Channel 1 Channel 2 M1b
Z
Ms Mm D1s D1m

T1 T1

S1s Ws1 Wm1 S1m

sig m ()
1 1
Mb M b M b

（1− M ） 1
b （1− M1b）
Ds Dm Truncating Q Q

T T M1sg M1g M1mg

sig m (•) sig m (•) sig m (•)
S s Ws Wm S m M 2sb M b2 M 2mb Lithography
System
Mb Mb Unfolding K
M sb M bK K
M mb

DsK DmK
(1 − M b ) (1 − M b )
TK TK

S sK W K
WmK
Q Q s
S mK
K
M b M bK

（1−MbK） （1−MbK）
ˆ
M ˆ
M
s m
Q Q
K
M sg K
M mg
M Kg
sig m (•)

M̂
(a) The SD-ILT algorithm (b) Encoder (c) Decoder

Figure 7. (a) The flowchart of SD-ILT algorithm, (b) the encoder of DMDL, and (c) the decoder of DMDL
(revised from Figs. 3 and 4 in Ref. [29]).

SD DMDL
Optimized Mask

(a) (b) (c) (d) (e) (f)

Print Image

(g) PE=118 (h) PE=73 (i) PE=72 (j) PE=55 (k) PE=18 (l) PE=40

Figure 8. Comparison between DMDL and SD approaches (revised from Figs. 8-10 in Ref. [29]).

future, different deep learning frameworks may be approaches may be an interesting topic to study. In
introduced and applied to solve for computational addition, several other aspects may have
lithography problems, including ILT and source- considerable impacts on the applications of those
mask optimization (SMO). How to exploit the deep learning methods, including generation of
synergy between the existing deep learning reliable training data sets or sample libraries, as well
approaches and model-based deep learning as the effective and efficient training methods.

J. Microelectron. Manuf. 3, 20030301 (2020) 6

Zhang et al.: Study of Inverse Lithography Approaches based on Deep Learning

Acknowledgments correction based on machine learning,” Microelectronic

Engineering 168, 15-26 (2016).
[15] X. Xu, Y. Lin, M. Li, T. Matsunawa, S. Nojima, C.
We thank for the financial support by the Kodama, T. Kotani, and D. Z. Pan, “Sub-resolution assist
feature generation with supervised data learning,” IEEE
National Natural Science Foundation of China Transactions on Computer-Aided Design of Integrated
(NSFC) (61675021), and the Fundamental Research Circuits and System 37(6), 1225-1236 (2018).
Funds for the Central Universities (2020CX02002, [16] K. Luo, Z. Shi, X. Yan, and Z. Geng, “SVM based
2018CX01025) layout retargeting for fast and regularized inverse
lithography,” J. Zhejiang Uni-Sci C 15(5), 390-400 (2014).
[17] R. Luo, “Optical proximity correction using a
multilayer perceptron neural network,” Journal of Optics
References 15(7), 075708 (2013).
[18] Y. LeCun, Y. Bengio, and G. Hinton, “Deep
learning,” Nature 521(7553), 436-444 (2015).
[1] A. K. Wong, Resolution Enhancement Techniques in [19] D. S. Woldeamanual, A. Erdmann, and A. Maier,
Optical Lithography (SPIE, 2001). “Application of deep learning algorithms for lithographic
[2] F. Schellenberg, “A little light magic,” IEEE Spectrum mask characterization,” Proc. SPIE 10694, 1069408
40(9), 34-39 (2003). (2018).
[3] X. Ma and G. R. Arce, Computational Lithography [20] W. Sim, K. Lee, D. Yang, J. Jeong, J. Hong, S. Lee,
(John Wiley and Sons, 2010). and H. Lee, “Automatic correction of lithography hotspots
[4] Y. Granik, “Fast pixel-based mask optimization for with a deep generative model,” Proc. SPIE 10961,
inverse lithography,” J. Micro-Nanolith. Mem 5(4), 1096105 (2019).
043002 (2006). [21] S. Lan, J. Liu, Y. Wang, K. Zhao, and J. Li, “Deep
[5] X. Ma, Q. Zhao, H. Zhang, Z. Wang, and G. R. Arce, learning assisted fast mask optimization,” Proc. SPIE
“Model-driven convolution neural network for inverse 10587, 105870H (2018).
lithography,” Opt. Express 26(25), 32565-32584 (2018). [22] X. Shi, Y. Zhao, S. Cheng, M. Li, W. Yuan, L. Yao,
[6] J. Yu and P. Yu, “Impacts of cost functions on inverse W. Zhao, Y. Xiao, X. Kang, and A. Li, “Optimal feature
lithography patterning,” Opt. Express 18(22), 23331– vector design for computational lithography,” Proc. SPIE
23342 (2010). 10961, 109610O (2019).
[7] A. Poonawala and P. Milanfar, “Mask design for [23] K. Chen, A. Lan, R. Yang, V. Chen, S. Wang, S.
optical microlithography—an inverse imaging problem,” Zhang, X. Xu, A. Yang, S. Liu, X. Shi, A. Li, S. Hsu, S.
IEEE Trans. Image Process 16(3), 774-788 (2007). Baron, G. Zhang, and R. Gupta, “Full-chip application of
[8] Y. Liu, and A. Zakhor, “Binary and phase shifting machine learning SRAFs on DRAM case using auto
mask desigh for optical lithography,” IEEE Trans. pattern selection,” Proc. SPIE 10961, 1096108 (2019).
Semicond. Manuf 5, 138-152 (1992). [24] Y. Zhang and W. Ye, “Deep learning based inverse
[9] S. Sherif, B. Saleh, and R. D. Leone, “Binary image method for layout design,”
synthesis using mixed linear integer programming,” IEEE https://arxiv.org/abs/1806.03182.
Trans. Image Process 4(9), 1252-1257 (1995). [25] H. Yang, S. Li, Y. Ma, B. Yu and E. Young, “GAN-
[10] X. Ma, Z. Wang, J. Zhu, S. Zhang, G. R. Arce, and S. OPC: mask optimization with lithography-guided
Zhao, “Nonlinear compressive inverse lithography aided generative adversarial nets,” in Proceedings of IEEE
by low-rank regularization,” Opt. Express 27(21), 29992- Conference on Design Automation Conference (IEEE,
30008 (2019). 2018), 1-6.
[11] S. Choi, S. Shim, and Y. Shin, “Machine learning [26] D. P. Kingma, and M. Welling, “Auto-encoding
(ML)-guided OPC using basis functions of polar Fourier variational bayes”, arXiv:1312.6114v10 (2013).
transform,” Proc. SPIE 9780, 9780H-1 (2016). [27] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu,
[12] S. Wang, J. Su, Q. Zhang, W. Fong, D. Sun, S. Baron, D.Warde-Farley, S. Ozair, A. Courville, and Y. Bengio,
C. Zhang, C. Lin, B. Chen, R. Howell, S. Hsu, L. Luo, Y. “Generative adversarial nets,” Proc. NIPS, 2672–2680
Zou, Y. Lu, and Y. Cao, “Machine learning assisted SRAF (2014).
placement for full chip,” Proc. SPIE 10451, 104510D [28] X. Ma and G. R. Arce, “Generalized inverse
(2017). lithography methods for phase-shifting mask design,” Opt.
[13] M. Guajardo, Y. Chen, P. Brooker, C. E. Wu, K. Express 15, 15066–15079 (2007).
Hooker, F. Latinwo, and K. Lucas, “Investigation of [29] X. Ma, X. Zheng, and G. R. Arce, “Fast inverse
machine learning for dual OPC and assist feature printing lithography based on dual-channel model-driven deep
optimization,” Proc. SPIE 10962, 109620E (2019). learning,” Opt. Express 28(14), 20404-20421 (2020).
[14] X. Ma, S. Jiang, J. Wang, B. Wu, Z. Song, and Y. Li,
“A fast and manufacture-friendly optical proximity

J. Microelectron. Manuf. 3, 20030301 (2020) 7