MosaiQ: Quantum Generative Adversarial Networks

for Image Generation on NISQ Computers

Daniel Silver Tirthak Patel William Cutler

Northeastern University Rice University Northeastern University
[email protected] [email protected] [email protected]

Aditya Ranjan Harshitta Gandhi Devesh Tiwari

arXiv:2308.11096v1 [quant-ph] 22 Aug 2023

Northeastern University Northeastern University Northeastern University

[email protected] [email protected] [email protected]

Abstract Opportunity Gap for Quantum GANs.

Quantum machine learning and vision have come to the A recent work by Huang et al. [11], referred to as QG-
fore recently, with hardware advances enabling rapid ad- Patch in this paper, is the state-of-the-art demonstration of
vancement in the capabilities of quantum machines. Re- QuantumGANs on real quantum computers. As our pa-
cently, quantum image generation has been explored with per also demonstrates, QGPatch can learn different shapes
many potential advantages over non-quantum techniques; and produce recognizable images in some cases, but can of-
however, previous techniques have suffered from poor qual- ten yield low-quality images. It suffers from the scalability
ity and robustness. To address these problems, we introduce challenge because it breaks the image into “patches” and
MosaiQ a high-quality quantum image generation GAN performs a pixel-by-pixel learning. Second, it is not effec-
framework that can be executed on today’s Near-term In- tive at generating a variety of images within the same class
termediate Scale Quantum (NISQ) computers.1 – this problem is known as “mode collapse” [7]. It is non-
trivial to achieve high-quality image generation while also
maintaining variety. Motivated by these limitations, Mo-
1. Introduction saiQ’s design pushes the state of the art by achieving higher
Generative Adversarial Networks, or GANs, are a type scalability, image quality, and variety.
of neural network architecture used in machine learning and
computer vision for generative modeling [8, 22, 2, 9]. A
Contributions of MosaiQ
classical GAN consists of two neural networks, a genera-
tor, and a discriminator, that are trained simultaneously in I. MosaiQ introduces the design and implementation of a
a competitive process. The generator generates fake data novel quantum generative adversarial network for image
samples, while the discriminator tries to distinguish them generation on quantum computers. MosaiQ’s approach is a
from real ones found in the training set, hence serving as an hybrid classical-quantum generation network where a net-
“adversarial entity”. Classical GANs have received signif- work of low-circuit-depth variational quantum circuits are
icant attention for generating high-quality images, among leveraged to learn and train the model. Upon acceptance,
other purposes including text generation, data augmenta- MosaiQ will be available as an open-source contribution.
tion, and anomaly detection [20, 1, 5].
Naturally, this has spawned interest in the quantum infor- II. MosaiQ’s design demonstrates how the extraction of
mation science community to develop corresponding quan- principal components of images enables us to learn and gen-
tum GANs that run on quantum computers. While recent erate higher-quality images, compared to the state-of-the-
efforts toward developing quantum GANs have been instru- art approach which is limited in its scalability due to pixel-
mental and early results have been encouraging, we discov- by-pixel learning [11]. However, exploiting information in
ered that existing approaches have severe scalability bottle- principal components to its full potential is non-trivial, and
necks and have significant room for improvement. This is MosaiQ proposes to mitigate those challenges using fea-
especially true in the context of generating high-quality im- ture redistribution. Furthermore, MosaiQ introduces a novel
age generation on real-system quantum computers. adaptive input noise generation technique to improve both
1 Accepted to appear in the proceedings of International Conference on the quality and variety of generated images – mitigating the
Computer Vision (ICCV), 2023. This is authors’ pre-print copy. common risk of mode collapse in generative networks.
Random Noise Images Random Noise Transform Images Digit 0 (MNIST) Boot (Fashion-MNIST)

Z Z Real
Input Classical QGPatch Input Classical QGPatch
Real
Fake Real
Fake Real or
or Fake?
Fake? Quantum
Generator Classical
Generator Discriminator Discriminator
(a) Classical GAN (b) MosaiQ’s GAN
Figure 2. State-of-the-art quantum image generator, QG-
Patch [11], generates low-quality images for different
datasets. However, its quality is still similar to a classi-
Figure 1. Classical generative adversarial network (GAN) cal GAN with 25× more parameters and iterations.
and MosaiQ’s hybrid quantum-classical GAN architecture.
tive optimization objectives of the generator and the dis-
criminator. The use of log provides more numerical stabil-
III. Our evaluation demonstrates that MosaiQ significantly ity because it converts the multiplication of multiple small
outperforms the state-of-the-art methods [11] on both sim- probabilities into addition, and it also allows us to calculate
ulation and real quantum computers with a hardware er- the derivatives more easily during the optimization process.
ror. MosaiQ is evaluated on the MNIST [6] and Fash- Once fully trained, a generator component of the GAN
ion MNIST datasets [21] – widely-used for QML evalua- is capable of converting random noise into new data sam-
tion on Near-term Intermediate Scale Quantum (NISQ) ma- ples (e.g., boots) that conform to the original distribution
chines [15, 11]. MosaiQ outperforms the state-of-the-art of the training data. Essentially, the generator, without the
methods [11] both visually and quantitatively – for example, use of a discriminator, can be inferred to obtain new data
over 100 points improvement in image quality generated on samples. GANs have been shown to be useful in a variety
IBM Jakarta quantum computer using the FID (Fréchet In- of applications such as image and text generation, data aug-
ception Distance) score [10], which is popularly used for mentation, and anomaly detection [20, 1, 5]. Naturally, this
comparing image quality. has spawned interest in the quantum information science
community to develop corresponding quantum GANs.
2. MosaiQ: Challenges and Solution
Quantum GANs follow a similar generator and discrim-
In this section, we present the design and implementation inator structure as classical GANs but use quantum princi-
of MosaiQ, a quantum image generative network. To pro- ples to train and infer from the model. Typically, the dis-
vide better context for understanding, first, we briefly de- criminator functions on classical resources, and the genera-
scribe the classical generative adversarial networks (GAN) tor is trained on quantum resources. This is expected, as this
(depicted in Fig. 1(a)), describe the limitations of the state- structure allows the Quantum GANs to leverage quantum
of-the-art quantum GANs, and then, provide the details of resources for generation tasks – the component that persists
MosaiQ’s quantum GAN design (depicted in Fig. 1(b)). beyond training; recall that the discriminator is essentially
a quality inspector that can be decoupled after training and
2.1. Generative Adversarial Networks (GANs) is not required during inference. While this structure is in-
Generative Adversarial Networks, or GANs, are a type tuitive and has been demonstrated to be somewhat promis-
of neural network architecture used in machine learning ing [11], there are multiple open challenges that need to be
for generative modeling [8]. The basic architecture and overcome to achieve higher quality.
workflow of a classical (non quantum) GAN are shown in
2.2. Limitations of Existing Quantum GANs
Fig. 1 (a). A classical GAN consists of two neural net-
works, a generator, and a discriminator, that are trained A recent work by Huang et al. [11], referred to as QG-
simultaneously in a competitive process. The generator Patch, is the most complete, state-of-the-art demonstration
generates fake data samples, while the discriminator tries of QuantumGANs on real quantum computers. QGPatch
to distinguish them from real ones found in the training follows the Quantum GAN architecture described earlier
set, hence serving as the “adversary”. Through training, but achieves limited effectiveness – QGPatch can learn dif-
the generator learns to create increasingly realistic samples ferent shapes and produce recognizable images in some
that mimic the distribution of the real data in order to bet- cases, but often suffers from low quality. Fig. 2 shows that
ter fool an increasingly effective discriminator, eventually digit ‘0’ is a reasonable approximation of the ground truth,
reaching an attempt to attain equilibrium with the value but the generated boot image is far less recognizable. De-
function expressed as minG maxD Ex∼qdata (x) [log D(x)]+ spite the imperfect generations, we can see in Fig. 2 how
Ez∼p(z) [log(1 − D(G(z)))]. This value function has two a classical GAN with 25× the parameters of QGPatch and
main identifiable components that correspond to the respec- trained for 25× as many iterations is worse or similar to the

2
quantum technique – supporting the potential of quantum Qubit 1 Ry Rx Ry Z
machine-learning for image generation tasks as compared Qubit 2 Ry Rx Ry Z
x6
to the classical approach with much higher resources and Qubit 3 Ry Rx Ry Z
complexity. This independently provides experimental evi- Qubit 4 Ry Rx Ry Z
dence for why the quantum information science community Qubit 5 Ry Rx Ry
is motivated to accelerate the progress of Quantum GANs,
despite its current limitations. Figure 3. Circuit ansatz design of MosaiQ’s generators.
The reasons for the limited effectiveness of Quatum The RY gates in each layer are optimized during training.
GANs are multi-fold. The quantum generator component
runs on the quantum hardware and requires many qubits
cuits where some components of the circuits are tunable pa-
to produce high-quality images from random input noise (a
rameters, which are iteratively optimized. As a background,
qubit is the fundamental unit of computation and informa-
a quantum circuit is essentially a sequence of quantum gates
tion storage on quantum computers). QGPatch addresses
applied to qubits. The quantum gates are fundamental op-
this challenge by breaking the image into “patches” and
erations that manipulate the qubit state, and any gate can
employing a generator for different patches. While this is
be represented as a unitary matrix. One-qubit gates, such
reasonable for smaller resolution images, the “patch-based”
as the Pauli gates (σx , σy , σz ), apply a rotation to just one
approach suffers from scalability due to its fundamental na-
qubit and are categorized by the axis around which the rota-
ture of learning pixel-by-pixel. For example, a total of 245
tion takes place, and by how much. Multi-qubit gates, such
qubits are required for images in the full-resolution 784-
as the CNOT gate, allow the creation of entanglement.
pixel MNIST dataset.
The second challenge is performing efficient learning Variational quantum circuits in MosaiQ are comprised
from random input noise – it is critical for the quantum gen- of two types of sections. The first type, X, has fixed gates
erator to effectively utilize the random input noise to gen- that entangle the qubits. The second type, θ, has tunable
erate a variety of images for the same class. Inability to U3 gates that are optimized via training. The overall cir-
generate a variety of images within the same class – which cuit V is therefore an optimization function built of unitary
even classical GANs suffer from – is popularly known as the transformations such that V = U (X, θ). One key feature of
“mode collapse” problem [7]. Mode collapse is a side-effect variational quantum circuits is the ability to leverage many
of the generator learning to generate only produce one type off-the-shelf classical deep-learning techniques for training
of image for a given class because the generator has learned and optimization. This includes learning through common
to “fool” the discriminator for this image and saturates in loss functions such as L2 loss and leveraging existing opti-
its learning. It is non-trivial to achieve high-quality gen- mizers such as Adam [13] that MosaiQ leverages.
erating while also maintaining variety. Motivated by these All sub-generator circuits in MosaiQ have identical ar-
limitations, MosaiQ’s design pushes the state of the art. chitectures, composed of a five-qubit circuit. Fig. 3 shows
an example of the sub-generator circuit. The circuit consists
2.3. Overview of MosaiQ and Key Ideas of encoding the input noise as angles using Rx gates and Ry
Hybrid Quantum-Classical Architecture. MosaiQ uses a gates. Following the embedding of the noise, the parame-
hybrid quantum-classical architecture for image generation, terized weights are encoded on each quantum layer along-
as summarized in Fig. 1(b). The generator is quantum in na- side CZ gates used to entangle the qubits at each layer.
ture and trained using quantum simulation, and the discrim- These weights contain the portion of the circuit that is op-
inator is classical – similar to the construction in [15, 11]. timized. Following these repeated layers, the P auliX ex-
However, there are two key novel architectural changes: pected value is taken for each qubit in measurement.
(1) to address the scalability and quantum resource bottle- We note the simplistic design of MosaiQ’s generator cir-
necks for generating high-quality images, MosaiQ applies a cuits is intentional; limiting the number of tunable param-
transformation on the input image dataset, and (2) MosaiQ eters and depth of the circuits enables MosaiQ to mitigate
employs quantum-style random noise as input to enhance hardware error impact on real quantum machines and main-
the “variety or mode-collapse” challenge. These two key tain high quality, as confirmed by our evaluation (Sec. 3).
features are described after a brief description of MosaiQ’s
Scalable Learning by Extracting Principal Components.
generator and discriminator.
Unlike previous approaches, which learn to construct the
MosaiQ’s Quantum Generator Network. MosaiQ’s pixels of an image directly and consequently, suffer from
quantum generator component is a network of multiple sub- the scaling bottleneck, MosaiQ demonstrates that extract-
generators. Each sub-generator is a variational quantum ing and learning the rich features is effective. The principal
circuit that is iteratively optimized to train a model. Vari- component analysis (PCA) method enables efficient learn-
ational quantum circuits are specific types of quantum cir- ing by focusing on the important features that compromise

3
 PCA Features Numbered in Order of Explained Variance age generation, when some sub-generators do not contain
1 2 3 4 5 6 7 8 much useful information in any feature, the entire generator
40 33 32 25 24 17 16 9 does not pose much utility.
39 34 31 26 23 18 15 10 Therefore, while effective in achieving scalability, only
38 35 30 27 22 19 14 11 performing PCA may not achieve the full potential of Mo-
37 36 29 28 21 20 13 12 saiQ’s quantum image generation as this leads to unbal-
GAN1 GAN2 GAN3 GAN4 GAN5 GAN6 GAN7 GAN8 anced training, likely to quickly discover gradients that ig-
Figure 4. MosaiQ divides the PCA features among the eight nore many sub-generators and plateaus in training. To mit-
generator circuits in a manner that the total explained vari- igate this challenge, we propose a new PCA feature distri-
ance is close to equal across all generators. bution mechanism to counteract the unbalanced nature of
assigning principal components to sub-generators (Fig. 4).
We begin by assigning the first principal component to
the first sub-generator, the second principal component to
unique images, as opposed to having to learn to distinguish the second, and so on until each sub-generator has one
the important features from the redundant features in an im- top principal component. We follow this up by assign-
age via the pixel-by-pixel method. PCA maximizes the en- ing the last n-1 principal components to the first genera-
tropy for a given number of components, concentrating the tor, where n is the size of each sub-generator. This is fol-
information of a dataset efficiently. lowed by assigning the second to last n-1 features to the
The first step in learning principal components is nor- second sub-generator and repeated on the third generator
malizing the input data. Once the images are normalized, and so on until all features have been assigned. This creates
MosaiQ decomposes the images into principal components a much more balanced distribution, which enables us to en-
and scales these components between [0, 1] so that the quan- sure all sub-generators hold utility during the training pro-
tum sub-generators can generate throughout the entire space cess and the learning is not skewed. As an important side-
of inputs. As quantum machines rely on only unitary trans- note, MosaiQ’s distribution also mitigates the pathologi-
formations, it is not possible to obtain an output with an ab- cal side-effects of hardware errors on NISQ quantum ma-
solute value greater than one in measurement. These scaled chines where the majority of critical principal component
components are fed into the discriminator as the “real la- features could be concentrated on a single sub-generator.
bels” and the generator begins to learn how to mimic the This sub-generator could be mapped to qubits with higher
distribution throughout the training process. After learning, hardware error rates, and this can potentially make the train-
the outputs are scaled back to the original principal compo- ing less effective. Our evaluation confirms the effective-
nent range, then are inverse transformed to form a unique ness of MosaiQ’s principal component feature distribution
image that is within the distribution of the original images. among learners. Finally, we describe how MosaiQ utilizes
This method helps us achieve higher quality in a scalable the adaptive noise during image generation to increase the
way. However, experimentally we found that this alone is variety of images generated to avoid mode collapse.
not sufficient. MosaiQ performs intelligent distribution of
features to make the learning more effective and robust. Adaptive Input Noise to Improve Variety of Generated
Images. Recall that it is critical for the quantum generator
Feature Distribution Among Learners. In the conven- to effectively utilize the random input noise to generate a
tional implementation of the previously proposed idea, variety of images for the same class. The inability to gen-
all generated features are distributed among several sub- erate a variety of images within the same class leads to the
generators before being concatenated. Recall that the Mo- “mode collapse” problem. Therefore, while MosaiQ’s pre-
saiQ’s quantum generator is an ensemble of multiple sub- vious methods help us achieve high quality, it is critical to
generators, where each sub-generator is a variation quantum achieving high-quality, while also maintaining variety.
circuit. In the default case where PCA features are aligned
one after another to the sub-generators, an unbalanced dis- To address this challenge, MosaiQ introduces an adap-
tribution of explained variance may emerge, where some tive noise range based on the ratio of training loss between
sub-generators are responsible for significantly more impor- the quantum generator and the classical discriminator. The
tant generations than others. This is because, by definition, noise range is determined by the current progress of the gen-
the explained variance of PCA features is heavily concen- erator discriminator mini-max game, instead of the tradi-
trated on the first few features, and there is often a signif- tional fixed range of [0, π2 ] as employed by QGPatch. For-
icant drop-off in explained variance from one PCA feature mally, the adaptive noise (Noiseadaptive ) is defined below in
to the next. Recall that PCA features are entangled within terms of the Generator loss GL and Discriminator loss DL
G
sub-generators in training to form rich connections for im- and the Ratio DLL0 is the Discriminator/Generator loss ratio
0

4
 Image Concatenate Quantum Random MosaiQ’s Concatenate PCA Inverse Generated
Dataset Generators Random Noise Generators and Reorder Transform Output Image
and Reorder

Z
Noise

Z
PCA Classical Adaptive Figure 6. MosaiQ’s inference process to generate new im-
Transform Discriminator Noise Range ages via running the generators on quantum machines.
Network
Generator Loss

Discriminator Loss

tent training as the generator can focus on quality when it

Figure 5. MosaiQ’s process for training and optimizing is doing relatively worse compared to the classical discrim-
the quantum generator circuits using PCA transformation, inator and expand to generating more variety in high qual-
feature distribution, and adaptive noise generation. ity as it begins to fool the discriminator more on an easier
objective. Overall, the adaptive noise mechanism helps us
increase the variety of images for a given class while ensur-
ing that the image quality also remains high. This is further
observed after the first epoch during training.
confirmed by our evaluation (Sec. 3).

\text {Noise}_{\text {adaptive}} = \dfrac {\pi }{8} + \dfrac {5\pi }{8} \textit {ReLU}(\textit {Tanh}\big (\dfrac {D_L}{G_L} - \frac {G_{L_{0}}}{D_{L_{0}}}\big )) (1) MosaiQ’s Classical Discriminator. MosaiQ ’s discrimina-
tor is a classical deep-learning network, designed to aid in
As the above formula indicates, this adaptive bound en- the training of the quantum generator. This network is es-
forces a minimum noise range of π8 and a maximum range sentially training wheels to guide a difficult-to-train quan-
of 3π tum GAN, which can be discarded after training. The dis-
4 for the noise. While the bounds are chosen to be a
specific constant, the noise is always distributed within this criminator is much larger than the quantum generator, as
range and the upper bound automatically adjusts itself based there is only a single discriminator that must compete in the
on the relative effectiveness of the generator and the dis- adversarial game with multiple quantum generators. Typi-
criminator. MosaiQ does not require tuning the thresholds cal to its classical counterpart, MosaiQ’s discriminator has
for the range for different classes, it automatically adjusts multiple linear layers with ReLU activation (e.g., 64 lay-
itself to adapt to different conditions during training. ers). MosaiQ has a terminal layer with Sigmoid activa-
MosaiQ embeds the adaptive noise using angle embed- tion which introduces more non-linearity into the network,
ding on the quantum circuit. The T anh function is used which enables it to learn more complex structures. The sin-
to scale the adaptive noise to asymptotically approach the gle value produced at the end of the discriminator network
range [−1, 1]. MosaiQ chose T anh because it is a widely- allows it to act as a classifier to distinguish between real and
used activation function in deep learning applications which generated data.
transforms unbounded data into data bound by [−1, 1]. Putting It All Together. The overall workflow for train-
2x
T anh is defined as T anh(x) = ee2x −1 +1 . In practice, we ing and inference for MosaiQ are visually summarized in
observed that it is not effective to have a noise range lower Fig. 5 and 6, respectively. Fig. 5 depicts the continuous
than π8 , we leverage ReLU to force negative values to be feedback-driven process where the discriminator and gen-
zero. ReLU is also a commonly used activation function, erator participate in a non-cooperative game to improve the
however, ReLU is used to provide a non-linear transform overall quality of MosaiQ. Depending upon the relative loss
that makes all negative values 0 (to ensure that the noise is of the discriminator and generator networks, the random in-
never less than π8 ), and keeps all positive values the same. put noise is automatically adjusted to help MosaiQ genera-
While the non-linearity of T anh and ReLU are impor- tors avoid the problem of mode collapse problem and enable
tant when used typically as activation functions, the non- it to generate variety. During the training process, the input
linearity is not the main function in this application. If images are transformed using PCA to encode critical and
the generator is not able to catch up with the discrimina- more information in a compact manner for resource-limited
tor for some iterations at this small range, it should learn NISQ quantum computers, instead of learning over the in-
this smaller range instead of getting even smaller. We set put images pixel by pixel. Finally, the five-step inference
the minimum as π8 as we discovered that a lower value than procedure shown in Fig. 6 highlights that the PCA transfor-
π
8 results in mode collapse, with almost no distinction be- mation is inverted to generate new images when running the
tween images generated for a given class. fully-trained MosaiQ generators on real quantum computers
Leveraging adaptive noise for input ensures more consis- for inference to generate new images.

5
 Digit 0 Digit 5 Digit 9 Boots T-shirts Trousers
3. Experimental Methodology
Input
Datasets. MosaiQ is evaluated on the MNIST [6] and Fash-
QGPatch
ion MNIST datasets [21] as they have been widely-used for
QML evaluation on NISQ-era quantum machines [15, 11]. MosaiQ

MNIST consists of 28x28 gray-scale images of handwritten

digits from 0 to 9. Fashion MNIST consists of 28x28 gray- Figure 7. MosaiQ produces visually higher-quality images
scale images of clothing and accessories. Fashion MNIST compared to the state-of-the-art technique, QGPatch, for
provides more challenges for image generation and is cho- different classes of MNIST and Fashion-MNIST datasets.
sen to explore more domains of quantum image generation. highest quality upscaling. For QGPatch, similar to MosaiQ,
For both datasets, we split the dataset by image label and all metrics scores shown are calculated based on 500 im-
train individual models for each data type. This technique ages generated at the end of training compared to the entire
has been used in [11] and allows us to interpret the genera- distribution of the respective image category.
tion process and difficulties isolated for each class of data.
We additionally set up other classical experiments to act
Experimental Framework and Training Details. The as ablations for the efficacy of MosaiQ. We introduce two
environment for MosaiQ consists of PyTorch [16] acting techniques: (1) PCAInverse (using random inputs and PCA
as a wrapper for Pennylane [3]. MosaiQ is trained on inverse transformation to generate images), and (2) Classi-
IBM’s quantum simulator for speed, but the inference is calPCA (a purely-classical GAN using the same number of
performed on both the quantum simulator and real quantum parameters as MosaiQ, but uses MosaiQ’s PCA technique
machine. For real quantum machine experiments, Penny- for feature compression). PCAInverse applies the same in-
lane compiles the circuits into a backend compatible with verse PCA transformation as MosaiQ, to random noise of
IBM-Qiskit [18]. All real machine runs were performed on size 40. This technique is designed to explore the isolated
the IBM QX Jakarta machine [18]. Images used in training effects of the inverse PCA transformation procedure since
are selected from the training set and decomposed to prin- it does not perform any learning. ClassicalPCA trains a
cipal components of size 40. These components are divided purely-classical GAN with an identical number of param-
across eight five-qubit sub-generators. The discriminator eters as MosaiQ and apply the same inverse PCA transfor-
learns to differentiate at the level of principal components mation workflow.
and does not need to utilize a full image. Our final FID Figures of Merit. To capture the quality of image gener-
metrics reported are at the end of the training after 500 iter- ation, our evaluation figures of merit are both qualitative
ations (where all methods achieve near-final stability). and quantitative. Our primary quantitative figure of merit is
MosaiQ uses PyTorch’s Standard Gradient Descent Op- FID (Fréchet inception distance) score [10]. FID evaluates
timizer for both the generator and discriminator and Binary the distance between two distributions, as defined below for
Cross Entropy Loss for the shared loss of the generator and a Gaussian with mean (m, C) and a second Gaussian with
discriminator. The Adaptive Noise range is simple to calcu- 2
mean (mw , Cw ): FID = ∥m − mw ∥2 + T r(C + Cw −
late based on the generator and discriminator loss and Mo- 1/2
2(CCw ) ). A lower FID score between two distributions
saiQ automatically guides it to adjust itself every training
indicates higher similarity – and hence, lower FID scores
iteration. The generator learning rate is 0.3 and the discrim-
imply higher quality. The FID score has been shown to pro-
inator learning rate to .05 with a batch size of 8.
vide a more meaningful comparison over alternative metrics
Framework for Competing Techniques. We set up QG- (e.g., Inception Score) for image GANs [10].
Patch, the state-of-the-art technique [11], using the popu- We also evaluate the variance of our images, by evaluat-
lar Pennylane framework and choose the parameters used ing the variance of the pixel values relative to the mean.
in the original paper [11]. Our QGPatch training is per- As defined below, the variance metric gives insight into
formed using the batch size of eight and it trains until the the distinctness of the images generated for each method.
quality stabilizes (500 iterations). We use the same network For each pixel in an image row r and column c in an im-
size that is mentioned in the original paper, using 4 sub- age, we sum the squared difference from the mean value
generators with 5 features each. Evaluating the probabili- for that respective pixel to achieve our variance score V .
ties in measurement yields a 64-pixel (8x8) image. As Mo- We then take the cumulative density function (CDF) of
saiQ’s results are based on the original size of the datasets of these variance scores for each image generated image G(z)
784 pixels, we upscale the results of QGPatch using Bilin- given
P Puniformly distributed noise z from [0, π/2]: V =
2
ear Interpolation to allow for direct comparison. We rigor- r (µ
c rc −G(z) rc ) . A higher variance indicates higher
ously explored multiple interpolation techniques to provide variety in generated images for a given class – which is de-
QGPatch as much as an advantage as possible and experi- sirable, however, it is also critical that a method should at-
mentally determined that Bilinear interpolation yielded the tain a lower FID score before demonstrating higher variety.

6
 QGPatch MosaiQ HQCGAN PCAInverse ClassicalPCA MosaiQ
Fréchet Inception

Fréchet Inception
(lower is better)

(lower is better)
Distance (FID)

Distance (FID)
60
100
40

20 50

109

124

108

110

121

129

125

116
33
19

45
38

47
33

47
40

50
36

51
37

52
41

51
42

57
44

59
32

90
31
22
19

51
43
38

45
37
33

50
45
40

46
40
36

48
43
37
88
54
46
41

54
48
42

56
49
44

42
36
32
0 0
1 3 9 8 4 6 5 0 2 7 1 3 9 8 4 6 5 0 2 7
MNIST Digit Classes MNIST Digit Classes

Figure 8. MosaiQ consistently produces lower FID score

images (higher quality images) compared to QGPatch Figure 11. MosaiQ produces higher quality images on ev-
across different classes of the MNIST dataset. ery class of the MNIST dataset as compared to HQCGAN
[19] in addition to the classical methods tested.

QGPatch MosaiQ
Fréchet Inception

HQCGAN PCAInverse ClassicalPCA MosaiQ

(lower is better)
Distance (FID)

Fréchet Inception

(lower is better)
Distance (FID)
75
150
50
100
25
50
37
31

41
22

45
37

52
37

53
39

55
43

55
22

58
41

75
55

82
37
0

141

134

125

162

119

106

168

133

157
42
34
31

33
26
22

48
41
37
83
46
42
37

49
42
39

53
46
43

33
26
22

55
44
41

68
61
55

47
40
37
0
ss

er

g
rt

ot
al

irt

r
a

ke

ve

Ba
hi

nd
re

us

Co

Bo
Sh

ss

er

g
rt

t
al

irt

ot
r
llo
ea

ke

ve
s
D

Ba
o

Sa

hi

nd
re

us
T-

Co

Bo
Sh
Sn
Tr

Pu

llo
ea
s
D

Sa
T-

Sn
Tr

Pu
Fashion MNIST Classes
Fashion MNIST Classes
Figure 9. MosaiQ consistently produces lower FID score
images (higher quality images) compared to QGPatch Figure 12. MosaiQ produces higher quality images on ev-
across different classes of the Fashion-MNIST dataset. ery class of the Fashion MNIST dataset as compared to
HQCGAN [19] in addition to the classical methods tested.

Input

HQCGAN vs. ‘5’ and different types of shoes). In comparison, Mo-

PCAInverse
saiQ is effective across different class types – this is further
substantiated by lower FID scores of the images generated
ClassicalPCA by MosaiQ (Fig. 8 and 9).
MosaiQ As expected, the FID score obtained by MosaiQ varies
across different classes for both datasets; this is because of
Figure 10. MosaiQ produces higher quality images on the varying degrees of difficulty and shapes of different images.
MNIST and Fashion MNIST dataset as compared to HQC- However, the most notable observation is that MosaiQ con-
GAN [19]. MosaiQ produces higher quality images than sistently outperforms QGPatch across all class types by a
the classical methods: PCAInverse and ClassicalPCA. significant margin (by more than 10 points in many cases).
In fact, MNIST digit ‘7’ is quite challenging for QGPatch
(FID score is 60), and MosaiQ improves the FID score by
approximately 20 points. Similarly, the most challenging
4. Evaluation and Analysis class in the Fashion-MNIST dataset (Boot) receives a 45-
point improvement by MosaiQ. The key reason MosaiQ
In this section, we go over the results of MosaiQ and
outperforms QGPatch is the efficiency of principal compo-
provide an analysis of the key elements of the design.
nents in capturing information. Instead of having to learn
MosaiQ: Key Results and SOTA Comparison. Mo- pixels one by one, MosaiQ scales more easily by learning
saiQ yields significantly better image quality both visually a distribution of features that better organize redundancy.
and quantitatively compared to the state-of-the-art (SOTA) For example, most of the background of MNIST and Fash-
method, QGPatch [11]. As discussed earlier, MosaiQ is ion MNIST images are black, in the case of QGPatch, each
evaluated on two different datasets to cover diversity across pixel must be synthesized, where MosaiQ may be able to
datasets and within a dataset with different shapes and build these features with only a few principal components.
styles (digits and clothing). First, we highlight that MosaiQ We also compare MosaiQ to other methods, including
provides a significant visual enhancement in the images the PCAInverse technique, the ClassicalPCA technique,
produces across different classes compared to the SOTA and a recent work on hybrid quantum GANs which we re-
method. In fact, Fig. 7 shows that the SOTA-produced im- fer to as HQCGAN [19]. We sample generated images in
ages are almost unrecognizable for sophisticated shapes (‘0’ Fig. 10 and find the images produced by all methods tested

7
 MosaiQ without PCA Feature Redistribution Fixed Noise ( π8 ) Fixed Noise ( π2 ) Adaptive Noise
MosaiQ with PCA Feature Redistribution 1.00
0.15 0.75

CDF
0.10 0.50
PDF

0.25
0.05
0.00
0 1 2 3 4 5
0
41 42 43 44 45 Variation Across Images
FID Score
Figure 14. Adaptive noise during generation helps MosaiQ
Figure 13. MosaiQ’s PCA feature distribution among
achieve variety (higher variance across images for the same
learners improves the quality of generated images. The fig-
class to mitigate mode-collapse challenge), while achieving
ure shows a distribution of 200 FID scores (comparing 8
lower FID score at the same time, as shown earlier.
images each to the entire data distribution) for the case with
and without PCA feature redistribution.
Digit 0 Digit 5 Digit 9 Boots T-shirts Trousers

are lower quality than MosaiQ, producing images which QGPatch

are far less human-recognizable. We show the correspond- MosaiQ
ing FID scores for MNIST in Fig 11 and Fashion MNIST
in Fig. 12. We find that MosaiQ produces higher quality
Figure 15. Visual image quality of images produced by Mo-
images and significantly improved FID scores in all cases
saiQ on real IBM quantum machine (Jakarta) for six repre-
tested compared to the classical methods (PCAInverse and
sentative classes for MNIST and Fashion-MNIST datasets.
ClassicalPCA), and HQCGAN. Importantly, MosaiQ out-
performs ClassicalPCA, demonstrating the power of quan-
tum networks in image generation when compared to equal- noise threshold range (i.e., [0, π2 ]) used by the QGPatch
sized classical networks. Our results show that while HQC- method. Recall that GANs often suffer from intra-class
GAN is promising and useful, the final quality may not be mode-collapse challenges where they can produce a high-
as high as MosaiQ – this is because HQCGAN requires sig- quality image for a given class, but there is a possibility that
nificantly more parameters, more training resources, lacks all generated images for a given class appear similar. There-
noise-mitigation, and learns noise in a fixed range ([0, 1) in- fore, it is critical to ensure that the generated images for a
stead of MosaiQ’s adaptive noise range technique). Next, given class have sufficient variety. Fig. 14 confirms that
we dissect the key reasons behind MosaiQ’s effectiveness adaptive noise improves variety (class digit ‘5’ used as an
in more detail. example) compared to fixed noise ranges. This is because
Why Does MosaiQ Work Effectively? Effect of careful adaptive noise enables the generators to learn over differ-
PCA feature distribution among sub-generators. Recall that ent distributions more effectively and generate variations.
MosaiQ employs an intelligent PCA feature distribution Having an adaptive range allows the generator to improve
among sub-generators in the generation phase. The goal variety when it is doing comparatively well and focus on
is to equalize the explainable PCA feature variance across stability when it is performing poorly with a smaller range
learners – in an effort to make learners equally capable in- of inputs. Taking advantage of this allows us to have high
stead of weaker learners not effectively contributing toward stability in training and high variety over time.
the overall quality. To better understand and demonstrate MosaiQ on NISQ Quantum Machines. Our experimental
the effectiveness of this mechanism, Fig. 13 shows the FID campaign on real superconducting-based IBM NISQ quan-
score over multiple training iterations with and without this tum computers confirms MosaiQ produces higher quality
mechanism while keeping all other design elements intact. images across different datasets with diverse classes of im-
For easier interpretation, Fig. 13 shows this for class digit ages. Fig. 15 and Table 1 summarize the image quality
‘5’ for the MNIST dataset. We observe that PCA feature results for QGPatch and MosaiQ. While MosaiQ’s qual-
distribution allows MosaiQ to achieve lower (and hence, ity remains consistent with simulations, QGPatch’s qual-
better) FID score over training compared to without em- ity degrades considerably on real computers. In fact, our
ploying this mechanism. A side benefit of this mechanism results revealed QGPatch produces lower quality images
is that MosaiQ can also mitigate hardware errors on real and sometimes unrecognizable images because QGPatch is
quantum computers, as discussed later. more prone to side-effects of prevalent quantum gate errors
Effect of adaptive noise generation during training. Fig. 14 on real machines, while MosaiQ’s simplistic design suc-
shows the effect of adaptive noise generation that is used cessfully mitigates those effects as discussed next.
by the MosaiQ generators instead of using a constant First, we observe that MosaiQ produces competitive

8
Table 1. QGPatch and MosaiQ’s FID scores on a real ponential superiority of Quantum GANs over their classi-
quantum computer for MNIST & Fashion-MNIST datasets. cal counterparts. This was later followed by the work of
(a) QGPatch Dallaire-Demers et al. [4], which established a way to de-
Environment Digit 0 Digit 5 Digit 9 T Shirt Pants Shoes sign and train variational quantum circuits as the generator
Quantum Simulator 52 52 48 45 42 82 component of a Quantum GAN model. Nakaji et al. [15]
IBM Jakarta Machine 145 134 131 125 144 124 use a hybrid configuration with a classical discriminator and
(b) MosaiQ a quantum generator for classification. Their results demon-
Environment Digit 0 Digit 5 Digit 9 T Shirt Pants Shoes strate the potential for quantum GANs to solve problems
Quantum Simulator 42 42 33 34 22 38 that are intractable with classical GANs.
IBM Jakarta Machine 45 44 35 38 23 38
Enchancing Classical GANs using Quantum Compo-
nents. Approaches such as the work done by Rudolph
quality images despite hardware errors on real quantum et al. [17] use a quantum enhancer for large-scale fully-
computers. Due to high queue wait time and limited avail- classical generators to enhance the quality of image gen-
ability of quantum resources, we are presenting results only eration. While the images generated are high quality, this
for selected classes (three from each dataset) which cover a is mostly attributed to the large classical generators. The
diverse range of intricacies in terms of shape and informa- visual results in these works seem appealing because the
tion. In fact, via visual inspection for different images pro- classical GAN-based baseline is already high quality, with
duced for a given class, we observe that the variety in the a quantum preamble which only improves the image quality
produced images by MosaiQ is also high on the quantum slightly. This is in contrast to MosaiQ which uses a quantum
computer - effectively mitigating the mode-collapse pitfall generator to generate images without a classical generator.
in GANs. This is because of MosaiQ’s adaptive noise mech-
Fully-Quantum Generators. The work of Huang et
anism during the generation phase that improves image va-
al. [12] uses quantum gradients to train quantum GANs to
riety and avoids mode collapse.
replicate logic gates such as XOR with high fidelity and also
Second, we observe that the FID scores of MosaiQ on
generate images from the MNIST dataset. The technique re-
a real quantum computer are similar to error-free simula-
lies on using quantum-based loss functions for their gener-
tion results, across different classes. This result demon-
ator and discriminator. This work also shows that quantum
strates that MosaiQ’s design is effective even when hard-
GANs have similar performance as classical GANs while
ware errors are present. In fact, the IBM Jakarta computer
using 94.98% fewer parameters. A succeeding work with
has a considerably low quantum volume (higher is better)
higher quality MNIST image generation can be found in
with a quantum volume of 16. Quantum volume is a use-
QGPatch [11], with a generator split among several sub-
ful metric in determining the error rate and capabilities of
generators each responsible for generating a part of the full
a quantum computer. For perspective, at the time of writ-
image. QGPatch trains a quantum generator against a clas-
ing, IBM has computers available with quantum volumes of
sical discriminator. The quality of generation is the highest
128. We chose a relatively less capable quantum computer
among pre-MosaiQ quantum GANs with a purely quantum
for our experiments to demonstrate the portability and ro-
generative component, at the time of writing. Therefore, we
bustness of MosaiQ. The reason for MosaiQ’s effectiveness
compare MosaiQ against QGPatch in this paper and show
is twofold: (1) its simplistic design for generator learners,
that MosaiQ outperforms QGPatch in terms of image qual-
which avoids a large number of two-qubit gates, and cir-
ity and variety.
cuits with high depths. which are more prone to hardware
noise, and (2) its PCA feature distribution mapping among
the ensemble of learners in the generator – this careful re- 6. Conclusion
distribution ensures that most critical PCA features are not
concentrated on a few qubits, which in the pathological case MosaiQ is the first quantum GAN to successfully gen-
could be most severely impacted by hardware-errors. Mo- erate high-quality images on real quantum computers. Mo-
saiQ’s PCA redistribution ensures that the learners do not saiQ incorporates PCA feature redistribution and adaptive
need to know the error rate of individual qubits on the com- noise to facilitate improved interaction between the quan-
puter or make assumptions about different error types. tum generator and classical elements of the model. This in-
tegration enhances the quality and variety of the generator’s
5. Related Work output respectively.
Demonstration of the Speedup Achieved by Quantum Acknowledgements We thank the reviewers for construc-
GANs. Quantum GANs were first introduced by Lloyd et tive feedback. This work was supported by NSF Award
al.[14]. This work theoretically proved the potential for ex- 2144540, Northeastern University, and Rice University.

9
References [16] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer,
James Bradbury, Gregory Chanan, Trevor Killeen, Zem-
[1] Samet Akcay, Amir Atapour-Abarghouei, and Toby P. ing Lin, Natalia Gimelshein, Luca Antiga, Alban Desmai-
Breckon. Ganomaly: Semi-supervised anomaly detection son, Andreas Kopf, Edward Yang, Zachary DeVito, Mar-
via adversarial training. In C. V. Jawahar, Hongdong Li, tin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit
Greg Mori, and Konrad Schindler, editors, Computer Vision Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch:
– ACCV 2018, pages 622–637, Cham, 2019. Springer Inter- An imperative style, high-performance deep learning library.
national Publishing. 1, 2 In Advances in Neural Information Processing Systems 32,
[2] Martin Arjovsky, Soumith Chintala, and Léon Bottou. pages 8024–8035. Curran Associates, Inc., 2019. 6
Wasserstein gan, 2017. 1 [17] Manuel S Rudolph, Ntwali Bashige Toussaint, Amara
[3] Ville Bergholm, Josh Izaac, Maria Schuld, et al. Pennylane: Katabarwa, Sonika Johri, Borja Peropadre, and Alejandro
Automatic differentiation of hybrid quantum-classical com- Perdomo-Ortiz. Generation of high-resolution handwritten
putations, 2018. 6 digits with an ion-trap quantum computer. Physical Review
[4] Pierre-Luc Dallaire-Demers and Nathan Killoran. Quantum X, 12(3):031010, 2022. 9
generative adversarial networks. Phys. Rev. A, 98:012324, [18] A tA v, MD Sajis Anis, and Abby-Mitchell others. Qiskit:
Jul 2018. 9 An open-source framework for quantum computing, 2021. 6
[5] Lucas Deecke, Robert Vandermeulen, Lukas Ruff, Stephan [19] Shu Lok Tsang, Maxwell T West, Sarah M Erfani, and
Mandt, and Marius Kloft. Anomaly detection with genera- Muhammad Usman. Hybrid quantum-classical generative
tive adversarial networks, 2018. 1, 2 adversarial network for high resolution image generation.
[6] Li Deng. The mnist database of handwritten digit images for arXiv preprint arXiv:2212.11614, 2022. 7
machine learning research. IEEE Signal Processing Maga- [20] Xuan Xia, Xizhou Pan, Nan Li, Xing He, Lin Ma, Xiaoguang
zine, 29(6):141–142, 2012. 2, 6 Zhang, and Ning Ding. Gan-based anomaly detection: A
[7] Zhitong Ding, Shuqi Jiang, and Jingya Zhao. Take a close review. Neurocomputing, 493:497–535, 2022. 1, 2
look at mode collapse and vanishing gradient in gan. In 2022 [21] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-
IEEE 2nd International Conference on Electronic Technol- mnist: a novel image dataset for benchmarking machine
ogy, Communication and Information (ICETCI), pages 597– learning algorithms, 2017. 2, 6
602, 2022. 1, 3 [22] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A.
[8] Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Efros. Unpaired image-to-image translation using cycle-
Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and consistent adversarial networks, 2017. 1
Yoshua Bengio. Generative adversarial networks, 2014. 1, 2
[9] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent
Dumoulin, and Aaron Courville. Improved training of
wasserstein gans, 2017. 1
[10] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner,
Bernhard Nessler, and Sepp Hochreiter. Gans trained by a
two time-scale update rule converge to a local nash equilib-
rium. Advances in neural information processing systems,
30, 2017. 2, 6
[11] He-Liang Huang, Yuxuan Du, Ming Gong, Youwei Zhao,
Yulin Wu, Chaoyue Wang, Shaowei Li, Futian Liang, Jin
Lin, Yu Xu, et al. Experimental quantum generative adver-
sarial networks for image generation. Physical Review Ap-
plied, 16(2):024051, 2021. 1, 2, 3, 6, 7, 9
[12] Kaixuan Huang, Zheng-An Wang, Chao Song, Kai Xu,
Hekang Li, Zhen Wang, Qiujiang Guo, Zixuan Song, Zhi-Bo
Liu, Dongning Zheng, et al. Quantum generative adversarial
networks with multiple superconducting qubits. npj Quan-
tum Information, 7(1):165, 2021. 9
[13] Diederik P. Kingma and Jimmy Ba. Adam: A method for
stochastic optimization, 2014. 3
[14] Seth Lloyd and Christian Weedbrook. Quantum generative
adversarial learning. Phys. Rev. Lett., 121:040502, Jul 2018.
9
[15] Kouhei Nakaji and Naoki Yamamoto. Quantum semi-
supervised generative adversarial network for enhanced data
classification. Scientific reports, 11(1):19649, 2021. 2, 3, 6,
9

10