0% found this document useful (0 votes)
128 views5 pages

A Comparative Analysis of GAN-Based Methods For SAR-To-Optical Image Translation

This document compares different GAN-based methods for translating SAR images to optical images. It discusses how paired GAN methods require corresponding SAR and optical image pairs for training, while unpaired methods can be trained on separate SAR and optical image datasets without correspondences. The document analyzes state-of-the-art paired and unpaired GAN translation methods and introduces a new higher resolution SAR-to-optical image dataset. It finds GAN methods can achieve satisfactory image translation results but performance depends on scene complexity and data resolution.

Uploaded by

MAHESWARI M
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
128 views5 pages

A Comparative Analysis of GAN-Based Methods For SAR-To-Optical Image Translation

This document compares different GAN-based methods for translating SAR images to optical images. It discusses how paired GAN methods require corresponding SAR and optical image pairs for training, while unpaired methods can be trained on separate SAR and optical image datasets without correspondences. The document analyzes state-of-the-art paired and unpaired GAN translation methods and introduces a new higher resolution SAR-to-optical image dataset. It finds GAN methods can achieve satisfactory image translation results but performance depends on scene complexity and data resolution.

Uploaded by

MAHESWARI M
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL.

19, 2022 3512605

A Comparative Analysis of GAN-Based Methods


for SAR-to-Optical Image Translation
Yitao Zhao , Turgay Celik , Nanqing Liu , Graduate Student Member, IEEE,
and Heng-Chao Li , Senior Member, IEEE

Abstract— Unlike optical sensors, synthetic aperture aperture radar (SAR) sensors are among the most commonly
radar (SAR) sensors acquire images of the Earth’s surface with used imaging sensors with different imaging capabilities. For
all-weather and all-time capabilities, which is vital in a situation instance, in disaster-affected areas covered by clouds, optical
such as a disaster assessment. However, SAR sensors do not offer
as rich visual information as optical sensors. SAR-to-optical sensors lose the ability to acquire images with good area
image-to-image translation generates optical images from SAR coverage. On the other hand, SAR sensors, with all-weather
images to benefit from what both imaging modalities have and all-time capabilities, can acquire images with minimal
to offer. It also enables multisensor image analysis of the effects of clouds or weather conditions, which enables the
same scene for applications such as heterogeneous change assessment of the disaster impact by practitioners. Yet, SAR
detection. Various architectures of generative adversarial
networks (GANs) have achieved remarkable image-to-image sensors do not offer as rich visual information as optical
translation results in different domains. Still, their performances sensors, making it difficult for remote-sensing practitioners to
in SAR-to-optical image translation have not been analyzed interpret the acquired SAR images. Combining all-weather and
in the remote-sensing domain. This letter compares and all-time capabilities of SAR sensors with the high-resolution
analyzes the state-of-the-art GAN-based translation methods detailed visual information of optical sensors can improve the
with open-source implementations for SAR-to-optical image
translation. The results show that GAN-based SAR-to-optical practitioner’s interpretation of SAR images and further guide
image translation methods achieve satisfactory results. However, downstream tasks such as heterogeneous change detection.
their performances depend on the structural complexity of However, the significant diversity of feature expression in SAR
the observed scene and the spatial resolution of the data. and optical images hinders joint data analysis, which remains a
We also introduce a new dataset with a higher resolution prevalent challenge for subsequent image-processing pipelines.
than the existing SAR-to-optical image datasets and release
implementations of GAN-based methods considered in this letter A feasible solution for this challenge is establishing a map-
to support the reproducible research in remote sensing. ping enabling the conversion between the source (i.e., SAR)
and target (i.e., optical) domains. Through cross-domain map-
Index Terms— Generative adversarial network (GAN), image-
to-image translation, multisensor images, optical, remote sens- ping, images in the source domain obtain the target domain’s
ing, SAR-to-optical image translation, synthetic aperture characteristics, reducing the nonlinear difference between het-
radar (SAR). erogeneous images. Since the cross-mapping process is similar
to the translation in natural language processing, this method
I. I NTRODUCTION is named image-to-image translation (I2IT). Several remark-
able I2IT methods have achieved excellent results on natural
E ARTH observation platforms acquire multisensor images
(or heterogeneous images) of the land surface at different
times, revealing complementary multitemporal information on
images, among which the most popular methods are based on
the generative adversarial network (GAN) [1]. According to
the land surface properties. The sensor characteristics deter- the optimization strategy and the setup of the source–target
mine the type of information and all-weather and all-time datasets, the existing GAN-based I2IT (GAN-I2IT) meth-
capabilities of the imaging platforms. Optical and synthetic ods fall under two main categories of paired and unpaired
methods [2].
Manuscript received April 18, 2022; accepted May 18, 2022. Date of The paired methods operate on a dataset of the correspond-
publication May 23, 2022; date of current version June 2, 2022. This work ing source and target images to achieve image translation,
was supported in part by the Sichuan Provincial Science and Technology
Projects under Grant 2019JDJQ0023 and in part by the National Key that is, for each source image (SAR), there must be a target
Research and Development Program of China under Grant 2020YFB0505704. (optical) image. Among the existing paired GAN-I2IT meth-
(Corresponding author: Turgay Celik.) ods, Pix2Pix [3] is a milestone using paired samples as the
Yitao Zhao, Nanqing Liu, and Heng-Chao Li are with the School of
Information Science and Technology, Southwest Jiaotong University, Chengdu input conditions and inherits the convolutional GAN [4] to
610032, China (e-mail: [email protected]; [email protected]; process image data. Many GAN-I2IT methods are inspired by
[email protected]). this method [5], [6]. Building a paired dataset requires careful
Turgay Celik is with the School of Information Science and Technology,
Southwest Jiaotong University, Chengdu 610032, China, also with the School selection of coregistered source and target images. However,
of Electrical and Information Engineering, University of the Witwatersrand, most image data collected are unpaired images, which means
Johannesburg 2000, South Africa, and also with the Faculty of Engineer- a lack of constraining conditions such as label images or
ing and Science, University of Agder, 4604 Kristiansand, Norway (e-mail:
[email protected]). instance images used in paired translation methods. Therefore,
Digital Object Identifier 10.1109/LGRS.2022.3177001 the research interest in recent years has gradually shifted from
1558-0571 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: DMI College of Engineering - CHENNAI. Downloaded on August 18,2022 at 05:20:19 UTC from IEEE Xplore. Restrictions apply.
3512605 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 19, 2022

paired translation to unpaired translation. Contrary to paired


translation, unpaired image translation deals with two unpaired
datasets and establishes the conversion relationship between
the source and target domains. As a well-known method in
unpaired image translation, CycleGAN [7] introduces cycle
consistency loss and identity loss into the original GAN loss
function so that the generated images through loops possess
high similarity with the original images. A variety of unpaired
methods work on the improvement of CycleGAN structure [2],
[8], [9]. For example, no independent component for encoding
GAN (NICE-GAN) [8] reuses the discriminator to encode the
input samples after the training phase. A decoupling training
paradigm is also proposed to stabilize the training. Contrastive
unpaired translation (CUT) [2] is proposed based on the
contrast learning strategy to achieve unpaired translation by
single images.
Because of the success of GAN-I2IT methods on natural
Fig. 1. Paired image translation (top) and unpaired image translation
images, some researchers have employed them in the remote- (bottom).
sensing domain. Dialectical GAN [10] was proposed based on
Spatial Gram matrices and WGAN-GP framework, realizing II. GAN-BASED I MAGE - TO -I MAGE T RANSLATION
the translation from a low-resolution Sentinel-1 image to a
The GAN [1] is a deep learning framework solving the prob-
high-resolution TerraSAR-X image. Atrous cGAN [11] is a
lem of guiding computers to generate high-quality images. The
framework for SAR-to-optical image translation by introduc-
GAN structure consists of two independent subnetworks antag-
ing a spatial pyramid for obtaining multiscale features. Most
onizing mutually. One subnetwork is a “generator,” whose duty
methods derive from the CycleGAN structure, focusing on
is to generate random samples similar to real images. The
the translation from SAR to the optical image or between two
other subnetwork is a “discriminator” that determines whether
different SAR sensor images. Although GAN-based methods
the input is a real or a synthetic sample generated by the
have emanated many applications in the remote-sensing field,
generator. The generator produces fake images with nearly
there is still a lack of comprehensive analysis of their perfor-
real data distribution in the optimization process to deceive
mances in the SAR-to-optical image translation. Furthermore,
the discriminator. Meanwhile, the discriminator improves its
there is a need for new datasets to support this line of research.
ability to distinguish authenticity. Taking the image translation
We aim to address these issues in this letter with the following
as a task, assume that X and Y are two training datasets
contributions.
with samples {x i | x i ∈ X } and {yi | yi ∈ Y}. The goal of
1) We surveyed seven widely used GAN-I2IT methods image translation is to establish a mapping between X and Y.
(both paired and unpaired) and their strengths and draw- For each sample x ∈ X , the distribution of x follows the
backs, exploring the possibilities of adopting the selected distribution x ∼ pd (x). So, the adversarial loss  function is
methods for SAR-to-optical image translation. This let- formulated as LGAN (G, D,
 X , Y) = E y∼ pd (y) log D(y) +

ter is the first comprehensive analysis of GAN-I2IT Ex∼ pd (x) log(1 − D(G(x)) , where the generator G tries to
methods for SAR-to-optical image translation to the best minimize the objective function, while the discriminator D
of our knowledge. tries to maximize it. Benefiting from the adversarial loss
2) We collected SAR-to-optical image pairs from function, the image generated by GAN is increasingly indis-
TerraSAR and other optical satellite image resources tinguishable from the real image.
and introduced a high-resolution dataset named In this study, we select seven state-of-the-art paired and
SAR2Opt comprising more than 4000 pairs of samples. unpaired GAN-I2IT methods for SAR-to-optical image trans-
3) We perform an extensive experimental analysis of lation as demonstrated in Fig. 1. We only considered the
GAN-I2IT methods on the SEN1-2 dataset [12] and the methods with open-source implementations for fair compar-
newly proposed SAR2Opt dataset to establish a baseline isons, only modified their data loading implementations to
for future research. We provide several recommendations accommodate the datasets, and did not make any other changes
for selecting GAN-I2IT methods for SAR-to-optical to the original implementations.
image translation and some possible future research
directions. A. Paired Methods
4) To support the reproducible research on SAR-to-optical 1) Pix2Pix [3]: Pix2Pix is a milestone method for applying
image translation, we released implementations of GAN to image-to-image translation. Pix2Pix pioneers the
GAN-I2IT methods considered in this letter and the utilization of conditional GAN (cGAN), and paired images
newly introduced SAR2Opt dataset at https://github. are considered as input conditions to guide the generation
com/MarsZhaoYT/Sar2Opt-Heterogeneous-Dataset. procedure. cGAN significantly improves on the drawback that

Authorized licensed use limited to: DMI College of Engineering - CHENNAI. Downloaded on August 18,2022 at 05:20:19 UTC from IEEE Xplore. Restrictions apply.
ZHAO et al.: COMPARATIVE ANALYSIS OF GAN-BASED METHODS FOR SAR-TO-OPTICAL IMAGE TRANSLATION 3512605

the original GAN merely generates random images through the TABLE I
input noise vector, in which artificial control is challenging to A MOUNT OF T RAINING AND T ESTING PAIRS
exert.
2) BicycleGAN [5]: Since the pairing translation strategy
of Pix2Pix can only generate a single image based on the
input sample, the proposal of BicycleGAN remedies the short-
coming in the diversity of sample generation. BicycleGAN
adds the coding information of the target domain contained
in the hidden layer and introduces random noise satisfying 1) SEN1-2 Dataset: SEN1-2 [12] is an extensive public
the Gaussian distribution to broaden the diversity of image dataset released by TUM, which contains 2 82 384 pairs of
generation. SAR at a maximum spatial resolution of 5 m and optical
images acquired by Sentinel-1 and Sentinel-2 satellites, respec-
B. Unpaired Methods tively. The dataset contains four subdatasets (SEN1-2-Spring,
1) CycleGAN [7]: Corresponding to Pix2Pix in paired SEN1-2-Summer, SEN1-2-Fall, and SEN1-2-Winter) with
translation, CycleGAN is also a representative method of imaging times of different seasons. The images in the SEN1-2
unpaired translation. CycleGAN uses unpaired datasets and dataset are 256 × 256 pixels.
introduces a cycle consistency loss function. It converts the 2) SAR2Opt Dataset: We used TerraSAR-X to collect SAR
samples in the source domain into the target domain and then images at the spatial resolution of 1 m of ten cities in Asia,
transmits them back to calculate the cycle consistency. North America, Oceania, and Europe from 2007 to 2013.
2) MUNIT [13]: Compared with CycleGAN, which directs The corresponding optical images collected from Google
feature extraction, multimodal unsupervised image-to-image Earth Engine are coregistered to SAR images by manual
translation (MUNIT) decomposes the image into content space selection of control points. We extracted image patches of
and attribute space, encodes the global image content and size 600 × 600 pixels from coregistered SAR-to-optical image
style, and then exchanges the attribute codes to realize trans- pairs to form the SAR2Opt dataset.
lation. Specifically, the content codes from the source domain Note that the SEN1-2 and SAR2Opt datasets contain multi-
are combined with different style codes from the target domain ple paired SAR-to-optical image samples. We used the original
to reconstruct and realize the diversity of generation. paired datasets to train the paired methods. However, we ran-
3) NICE-GAN [8]: NICE-GAN is an improvement over domly shuffled the source and target images of the datasets
the structure of CycleGAN. A typical GAN-based image for unpaired methods to ensure that the datasets no longer
translation method requires alternate training of the generator contained paired samples. After that, we trained and tested
G and the discriminator D. There is no requirement for an paired and unpaired GAN-I2IT methods on the paired and
independent generator in NICE-GAN. After the discriminator unpaired datasets, respectively.
training, part of the early layers in the discriminator are
retained and reused as a generator for generation, which B. Metrics
reduces the complexity of the model structure. We employed full-reference image quality metrics peak
4) CUT [2]: CUT introduces contrastive learning into signal-to-noise ratio (PSNR) and structural similarity index
GAN-based image generation for the first time. Traditional measure (SSIM) [15] to measure the quality of SAR-to-optical
contrast learning uses internal patches and external patches image translation. The PSNR is a simple to compute metric,
during sampling. However, only internal patches are used but it may not align well with the perceived distortion. On the
in CUT. This strategy eliminates the strict restriction of other hand, the SSIM metric measures the perceptual similarity
“bijection” between two image domains required in the cycle based on local image structure, luminance, and contrast and
consistency. In addition, in terms of model structure, CUT aligns well with the perceived distortion. The higher the PSNR
streamlines the structure of CycleGAN, retaining only the and SSIM scores, the better the translation is.
single generator and discriminator structure.
5) Attn-CycleGAN [14]: To deal with the critical problems C. Model Training and Testing
such as mode collapse and diminished gradient in practical
The training and inference processes are implemented on
applications of GAN, Attn-CycleGAN incorporates a spatial
an NVIDIA RTX 2080Ti GPU with 11 GB of memory. Each
attention mechanism to calculate the attention map and guide
of the selected methods is trained for 100 epochs. The initial
the translation process of the generator.
learning rate of each method is 0.001, and the batch size is
set to 1. The learning rate decreases gradually after 50 epochs.
III. E XPERIMENTS
Considering that the SEN1-2 dataset [12] is too large for our
A. Datasets computing resources, we randomly sample datasets from the
We used the SEN1-2 dataset [12] and our newly proposed SEN1-2 dataset comprising 20% of the original dataset. The
SAR2Opt dataset to evaluate image translation performances SAR2Opt dataset was employed as a whole given its moderate
of different GAN-based methods. Fig. 2 shows some examples size. For all experiments, we use 80% of the data for training
from the datasets, and the relevant characteristics of the and the remaining 20% for testing. The detailed amount of
datasets are summarized as follows. samples used for training and testing is listed in Table I.

Authorized licensed use limited to: DMI College of Engineering - CHENNAI. Downloaded on August 18,2022 at 05:20:19 UTC from IEEE Xplore. Restrictions apply.
3512605 IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 19, 2022

TABLE II
E XPERIMENTAL R ESULTS ON THE SEN1-2 AND SAR2O PT D ATASETS

Fig. 2. Sample SAR-to-optical translation results on the SEN1-2 (top row) and SAR2Opt (bottom row) datasets.

Furthermore, we did not use data augmentation during the is better than their performances on the SEN1-2 dataset.
training process of the methods considered in this study. Specifically, Pix2Pix and CycleGAN maintain the best perfor-
mance on SAR2Opt. In addition, we note that Attn-CycleGAN
D. Experimental Results achieves comparable performance results to CycleGAN on the
1) Quantitative Analysis: Table II(a)–(d) shows the perfor- SEN1-2-Spring and SAR2Opt datasets. Since the SAR2Opt
mances of the GAN-I2IT methods on the SEN1-2 dataset, dataset contains higher-resolution SAR-to-optical image pairs
demonstrating that Pix2Pix and CycleGAN, on average, with richer image textures than the image pairs in the SEN1-2
achieve the best performances among paired and unpaired dataset, the GAN-I2IT methods, on average, yield better
methods. Table II(a) and (b) shows that the unpaired performances in this dataset.
methods achieve comparable results to the paired meth- 2) Qualitative Analysis: Fig. 2 shows SAR-to-optical image
ods on the SEN1-2-Spring and SEN1-2-Summer subdatasets. translation results of the GAN-I2IT methods on sample test
However, paired methods outperform unpaired methods on images from the SEN1-2 and SAR2Opt datasets. Among
SEN1-2-Fall and SEN1-2-Winter subdatasets as shown in the paired methods, Pix2Pix generates visually better results
Table II(c) and (d). By analyzing the corresponding datasets, compared to the results from BicycleGAN. BicycleGAN aims
we discovered that the land surface properties of the to improve the diversity in the generation process, which
SEN1-2-Spring and SEN1-2-Summer subdatasets are more resulted in additional surface artifacts and style changes in
complex than that of SEN1-2-Fall and SEN1-2-Winter sub- the translated images. CycleGAN’s results are closest to the
datasets, covering coastal, towns, and farmlands. Therefore, ground-truth images among the unpaired methods for low-
the unpairing strategy in CycleGAN achieves the best results resolution and high-resolution samples from the SEN1-2 and
on the first two subdatasets containing complex scenes. SAR2Opt datasets. The translation results of Attn-CycleGAN,
Table II(c) and (d) indicates that Pix2Pix and CycleGAN CUT, and NICE-GAN are comparable to CycleGAN’s results.
maintain the best performance on the SEN1-2-Fall and Due to its attention mechanism, Attn-CycleGAN produces
SEN1-2-Winter subdatasets. Since the two datasets possess results with more detailed texture information on the translated
relatively simple scenes and weak texture, paired methods images than CUT and NICE-GAN. One can also observe the
result in better performance on these two datasets. patching artifacts on the generated images of NICE-GAN.
Table II(e) shows the results of the GAN-I2IT methods Among all methods, MUNIT produced results with severe
on the proposed SAR2Opt dataset, indicating that the paired visual artifacts.
methods, on average, achieve better performance than the 3) Computational Stability and Efficiency: We test the
unpaired methods, among which CycleGAN outperforms other stability and convergence speed of the GAN-I2IT methods
methods. The metric results on the SAR2Opt dataset follow during the training process on the SAR2Opt dataset based
a similar trend to that of the SEN1-2 dataset, confirming on the Fréchet inception distance (FID) score [16] as shown
the applicability of the proposed dataset. The overall per- in Fig. 3. The FID score measures the distance between
formance of GAN-I2IT methods on the SAR2Opt dataset the feature vectors of synthetic and real images. Lower FID

Authorized licensed use limited to: DMI College of Engineering - CHENNAI. Downloaded on August 18,2022 at 05:20:19 UTC from IEEE Xplore. Restrictions apply.
ZHAO et al.: COMPARATIVE ANALYSIS OF GAN-BASED METHODS FOR SAR-TO-OPTICAL IMAGE TRANSLATION 3512605

that GAN-based methods are sensitive to scene complexity


and textural information. In general, unpaired methods are
more robust in complex scenes. Pix2Pix and CycleGAN
achieve better results than other methods considered in this
letter in restoring the texture and color style in generating
optical images from the SAR images. CycleGAN achieves the
best training stability with remarkable translation results. Its
bidirectional mapping process optimizes the cycle consistency
constraints, resulting in the highest inference times compared
to the other methods. One could achieve further improvements
on SAR-to-optical image translation using CycleGAN, or other
similar methods, by incorporating the parameters of SAR
sensors, such as polarization, as constraints.
Fig. 3. Computational stability and convergence speed of the GAN-I2IT
methods during the training process on the SAR2Opt dataset as FID-Epoch R EFERENCES
curves. [1] I. J. Goodfellow et al., “Generative adversarial nets,” in Proc. NIPS,
TABLE III vol. 27, 2014, pp. 2672–2680.
[2] T. Park, A. A. Efros, R. Zhang, and J.-Y. Zhu, “Contrastive learn-
AVERAGE AND S TANDARD D EVIATION OF I NFERENCE T IMES IN ing for unpaired image-to-image translation,” in Proc. ECCV, 2020,
M ILLISECONDS ( IN ms) OF GAN-I2IT M ETHODS pp. 319–345.
[3] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation
with conditional adversarial networks,” in Proc. CVPR, Jul. 2017,
pp. 1125–1134.
[4] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation
learning with deep convolutional generative adversarial networks,” 2015,
arXiv:1511.06434.
[5] J.-Y. Zhu et al., “Toward multimodal image-to-image translation,” in
Proc. NIPS, 2017, pp. 465–476.
[6] T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, and B. Catanzaro,
“High-resolution image synthesis and semantic manipulation with con-
represents a narrower gap between the real and generated ditional GANs,” in Proc. CVPR, Jun. 2018, pp. 8798–8807.
images. CycleGAN and Pix2Pix possess the best training [7] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image
stability, while CUT is the worst. We also report the average translation using cycle-consistent adversarial networks,” in Proc. ICCV,
2017, pp. 2223–2232.
and standard deviation of inference times in milliseconds [8] R. Chen, W. Huang, B. Huang, F. Sun, and B. Fang, “Reusing discrimi-
(in ms) of GAN-I2IT methods in Table III. MUNIT decreases nators for encoding: Towards unsupervised image-to-image translation,”
the structural complexity and improves the algorithm comput- in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR),
Jun. 2020, pp. 8168–8177.
ing efficiency by compressing the image into a style code, [9] E. Richardson et al., “Encoding in style: A styleGAN encoder for image-
which BicycleGAN also adopts. In addition, MUNIT employs to-image translation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern
one-way translation. Hence, it achieves the minimum inference Recognit. (CVPR), Jun. 2021, pp. 2287–2296.
[10] D. Ao, C. O. Dumitru, G. Schwarz, and M. Datcu, “Dialectical GAN
time at the cost of lower-quality generated images. CycleGAN for SAR image translation: From Sentinel-1 to TerraSAR-X,” Remote
and Attn-CycleGAN use bidirectional mapping to optimize the Sens., vol. 10, no. 10, p. 1597, Oct. 2018.
cycle consistency constraints for an improved fidelity in the [11] J. N. Turnes, J. D. B. Castro, D. L. Torres, P. J. S. Vega, R. Q. Feitosa,
and P. N. Happ, “Atrous cGAN for SAR to optical image translation,”
translation, resulting in the highest inference times compared IEEE Geosci. Remote Sens. Lett., vol. 19, pp. 1–5, 2022.
to the other methods. [12] M. Schmitt, L. H. Hughes, and X. X. Zhu, “The SEN1–2 dataset for
deep learning in SAR-optical data fusion,” ISPRS Ann. Photogramm.
IV. C ONCLUSION Remote Sens. Spatial Inf. Sci., vol. 1, pp. 141–146, Jul. 2018.
[13] X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz, “Multimodal unsu-
We surveyed the existing GAN-based image-to-image pervised image-to-image translation,” in Proc. Eur. Conf. Comput. Vis.,
translation methods mainly designed for natural images and 2018, pp. 172–189.
[14] Y. Lin, Y. Wang, Y. Li, Y. Gao, Z. Wang, and L. Khan, “Attention-based
compared their performances for SAR-to-optical image trans- spatial guidance for image-to-image translation,” in Proc. IEEE Winter
lation in the remote-sensing domain. We introduced a new Conf. Appl. Comput. Vis. (WACV), Jan. 2021, pp. 816–825.
high-resolution dataset, SAR2Opt, for SAR-to-optical image [15] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli,
“Image quality assessment: From error visibility to structural sim-
translation and released the implementations of GAN-based ilarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612,
methods considered in this letter to support the reproducible Apr. 2004.
research in remote sensing. [16] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter,
“Gans trained by a two time-scale update rule converge to a local nash
The experimental results for SAR-to-optical image trans- equilibrium,” in Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017,
lation on the existing and newly proposed datasets reveal pp. 6629–6640.

Authorized licensed use limited to: DMI College of Engineering - CHENNAI. Downloaded on August 18,2022 at 05:20:19 UTC from IEEE Xplore. Restrictions apply.

You might also like