Received March 26, 2020, accepted April 30, 2020, date of publication May 11, 2020, date of current

version June 16, 2020.

Digital Object Identifier 10.1109/ACCESS.2020.2993928

FGGAN: Feature-Guiding Generative Adversarial

Networks for Text Generation
YANG YANG , XIAODONG DAN , XUESONG QIU , AND ZHIPENG GAO
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
Corresponding author: Yang Yang ([email protected])
This work was supported in part by the National Key Research and Development Program of China under Grant 2019YFB2103200, in part
by the Open Subject Funds of Science and Technology on Information Transmission and Dissemination in Communication Networks
Laboratory under Grant SKX182010049, in part by the Fundamental Research Funds for the Central Universities under Grant 500419319
2019PTB-019, and in part by the Industrial Internet Innovation and Development Project 2018 of China.

ABSTRACT Text generation is a basic work of natural language processing, which plays an important role
in dialogue system and intelligent translation. As a kind of deep learning framework, Generative Adversarial
Networks (GAN) has been widely used in text generation. In combination with reinforcement learning, GAN
uses the output of discriminator as reward signal of reinforcement learning to guide generator training, but
the reward signal is a scalar and the guidance is weak. This paper proposes a text generation model named
Feature-Guiding Generative Adversarial Networks (FGGAN). To solve the problem of insufficient feedback
guidance from the discriminator network, FGGAN uses a feature guidance module to extract text features
from the discriminator network, convert them into feature guidance vectors and feed them into the generator
network for guidance. In addition, sampling is required to complete the sequence before feeding it into the
discriminator to get feedback signal in text generation. However, the randomness and insufficiency of the
sampling method lead to poor quality of generated text. This paper formulates text semantic rules to restrict
the token of the next time step in the sequence generation process and remove semantically unreasonable
tokens to improve the quality of generated text. Finally, text generation experiments are performed on
different datasets and the results verify the effectiveness and superiority of FGGAN.

INDEX TERMS Generative adversarial networks, text generation, deep learning, reinforcement learning.

I. INTRODUCTION in the field of text generation, but the structure has the fol-
Generative Adversarial Networks (GAN) [1] has gradually lowing drawbacks:
developed into a hot research field in deep learning since it 1) The probability that the discriminator outputs positive
was proposed. As a generative model, its main application and negative samples is used as a reward signal in rein-
field is image generation, but GAN also has a high research forcement learning. The feedback signal is a scalar and
potential in text generation. Poetry writing, dialogue system cannot preserve the high-level semantic information of
and intelligent translation are all based on text generation. the text. The generator lacks a clear training direction.
Although some progress has been made, the quality of gen- 2) Sampling is required to complete the sequence and
erated text is often poor or limited to specific areas and lack get feedback signal through discriminator in the text
of generality. sequence generation process. Due to the limitation of
The combination of GAN and reinforcement learn- sampling times, the sampling process is highly random
ing (RL) for text sequence generation has become one of the and inadequate, and may result in semantically unrea-
hotspots of research. Sequence GAN (SeqGAN) proposed sonable sequences such as subject repetition and verb
by Yu et al. [2] for the first time feeds discriminator output deletion.
as reward signals of reinforcement learning to generators as Based on the current problems of text generation, we pro-
decision guidance for generating sequences. GAN combined pose a text generation algorithm based on the Feature-
with reinforcement learning has achieved remarkable results Guiding Generative Adversarial Networks (FGGAN). The
improved algorithm will effectively solve the shortcomings
The associate editor coordinating the review of this manuscript and of the existing text generation model. The main contributions
approving it for publication was Nizam Uddin Ahamed . of this paper are as follows:

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 8, 2020 105217
 Y. Yang et al.: FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

1) Because the output probability of the discriminator completed the training by adjusting the generator to make
is scalar, the training direction of the generator is the generated samples have the same characteristics as the
not clear. In this paper, a feature guidance module real samples. Su et al. [10] have made some achievements in
is improved to transform the high-order text feature dialogue generation using GAN combined with a hierarchical
extracted from the discriminator into the generator net- recurrent encoder-decoder. Fedus et al. [11] introduced an
work for feedback guidance. The feedback signal is actor-critic conditional GAN and produced more realistic text
transformed into a guidance vector with more guidance samples.
information to improve the generation of sequences by GAN can be combined with reinforcement learning. The
the generator. generator can be seen as a decision maker and the network
2) Because of the randomness and inadequacy in the can be optimized using a policy gradient. Reinforcement
process of sequence sampling, it is possible to gen- learning requires reward signals as feedback. At each time
erate non semantic tokens in the process of sequence step, complete sequences can be sampled using Monte Carlo
generation. This paper proposes a method to create Tree Search (MC Search) [12] and fed into the discriminator
a vocabulary mask based on semantic rules, which to get reward signal. Yu et al. proposed SeqGAN, which is
restricts the tokens generated in the next time step the first time to use discriminator as reward for reinforcement
during the sequence generation. Remove the candidate learning. Nie et al. [13] proposed Relational GAN (RelGAN),
tokens with low correlation with the generated pre- which uses Gumbel-Softmax to train GAN on discrete data
fix sequence to make the generated sequence is more and multiple embedded representations in the discriminator
realistic. to provide a more informative signal. Lin et al. [14] proposed
RankGAN which replaces the original binary classifier with a
II. RELATED WORK sorting model based on cosine similarity to make the feedback
Text generation is a task of simulating and generating of the discriminator more continuous. However, these models
sequence data. Most text generation tasks are based on Recur- still have the disadvantage that the output of the discrimi-
rent Neural Network (RNN). The Long Short-Term Mem- nator as a feedback signal is scalar and has weak guidance.
ory (LSTM) proposed by Hochreiter et al. [3] has been In addition, the randomness of the sampling process may
widely used. Wen et al. [4] used LSTM to build a natural cause the network not to learn the implicit semantic infor-
language dialogue system. In 2014, Goodfellow proposed the mation, resulting in the unrealistic generated data. In this
generative adversarial networks. Compared with the single paper, FGGAN is proposed for the shortcomings of the exist-
generation model, GAN has a more significant effect in ing model and the effectiveness of the improved module is
data generation. GAN uses generator and discriminator to verified by experiments.
train and optimize in the adversarial way and finally reaches
the state of Nash equilibrium to effectively learn the data III. ALGORITHM
distribution. This paper adopts the overall framework of GAN and puts
GAN has been successfully applied in the field of images to forward improvement aiming at the existing defects. Firstly,
generate realistic images. Chang et al. [5] proposed KGGAN the output scalar of the discriminator in the correlation algo-
by setting up multiple generators, one of which is respon- rithm is used as the feedback signal, which leads to the
sible for learning the information in a priori knowledge unclear training direction of the generator. In this paper,
field and directing the learned knowledge to another gen- a feature guidance module is improved to transform the
erator to generate a variety of image data. But the disad- high-order text feature extracted from the discriminator into
vantages are that a priori domain knowledge is needed to the generator network for feedback guidance. In addition,
assist and the learned diversity is not easily accepted by this paper proposes a method to create a vocabulary mask
the discriminator. Lian et al. [6] proposed FG-SRGAN for based on semantic rules, which restricts the tokens generated
high-resolution image generation, mainly by setting up a in the next time step during the sequence generation. The
guidance module to learn the mapping from low-resolution improved model is named Feature-guiding GAN, and the
image to high-resolution image, to improve the quality of the overall network structure is shown in Fig. 1.
generated image by the generator. The overall structure of the GAN is divided into a generator
However, due to the discreteness of text data, the original Gθ and a discriminator Dφ . The objective of the generator is
GAN could not optimize the generator parameters based on to find the parameters of the optimal distribution probability
gradient backpropagation. Martin et al. [7] conducted some of the data. However, the parameter update does not originate
analysis on the training methods of GAN such as using the from the data sample, but from the back-propagation gradient
Wasserstein divergence instead of the traditional JS diver- of the discriminator. The model is trained via the adversarial
gence to train GAN in the field of text generation. In addition, strategy. Also, the generator and the discriminator are alter-
Che et al. [8] proposed the maximum-likelihood augmented nately optimized.
discrete GAN (MailGAN) and designed training techniques The generator on the left activity box in Fig.1 can be
to directly calculate the difference between the generated data regarded as performing a text generation task. The objective
distribution and the real data distribution. Zhang et al. [9] of the generator G is to predict the next token based on the

105218 VOLUME 8, 2020

Y. Yang et al.: FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

a series of action selection. The action of each step is to

select the next token, i.e. ai = yt , and the state of each
step is a prefix sequence composed of generated tokens, i.e.
st = y1 , . . . , yt−1 . After the last step, the complete sequence
is sent to the discriminator and the output probability is used
as the reward of the generator. The goal of the generator is to
maximize the reward expectation. The optimization objective
of the generator is shown in (5), as follows:
max EY1:T ∼Gθ [log Dφ (Y1:T )] (5)
θ
where log Dφ (Y1:T ) is the reward of generator The strat-
egy of reinforcement learning is expressed as Gθ (a|s) =
P(a|s; θ), which specifies the probability of selecting an
FIGURE 1. The overall network structure of Feature-guiding GAN
action under state s. The action reward value function is
denoted as Qθ (s, a), which indicates the total reward expecta-
tion obtained according to the strategy after action a in state s.
generated sequence. Assuming a sequence of tokens, where
The reward expectation of the Tth token is shown in (6),
yi represents a token, y represents the vocabulary. Genera-
as follows:
tor is mainly divided into text generation module marked
blue in Fig.1 and feature guidance module marked purple Qθ (s = Y1:T −1 , a = yT ) = log Dφ (Y1:T ) (6)
in Fig.1, both of which adopt LSTM structure. Combining the The total reward expectation is shown in (7), as follows:
improved feature guiding module (Section III-A for details)
and the mask vector (Section III-B for details), the generation EY1:T ∼Gθ [Qθ (Y1:T −1 , yT )]
Gθ (yT |Y1:T −1 )Qθ (Y1:T −1 , yT ) (7)
X X
equations in time step t can be shown in (1) -(3): = Gθ (y1 |s0 )...
y1 yT
ht = RθR (ht−1 , xt ) (1)
The discriminator can only judge the complete sequence,
P(·|x1 , . . . , xt ) = zt (ht ) = softmax(W (ht wt MaskT ) + c) (2)
which means that the current generated token cannot be evalu-
yt ∼ p(·|x1 , . . . , xt ) (3) ated in the sequence generation. The granularity of evaluating
RθR is the set of generator parameters, ht−1 is the hidden the generation quality is too large when generating a com-
state of the previous step, xt represents the input vector of plete sequence, which is equivalent to treating every token
the current time step. The calculated current step hidden state in the sequence the same. To obtain better results, the Monte
ht is a y-dimensional vector. ht combines the guide vector Carlo search method is used to complete the current gener-
wt generated by feature guidance module and mask vector ated sequence at the intermediate time. In the time step t,
MaskT generated by semantic rule module to pass through a the unknown last T − t tokens can be sampled by the Monte
softmax layer, calculates the probability distribution zt of the Carlo search and a K -time MC search can be shown in (8):
tokens and samples a token yt . 1
{Y1:T , . . . , Y1:T
K
} = MC(Y1:T ; K ) (8)
The discriminator on the right activity box in Fig.1 uses the
The purpose of MC search is to evaluate the corresponding
Convolutional Neural Network (CNN). After vectoring the
reward with the generated prefix sequence for each new
input text, it uses convolutional kernels of different sizes to
generated token. Random sampling is conducted through the
extract the text features and transforms the extracted features
generator to continue to complete the incomplete sequence
into the generator for more detailed sequence generation
and the sequence is sent to the discriminator to judge the
guidance. The discriminator is trained by the negative sam-
score. The scores that are obtained via multiple sampling are
ples generated by the generator and the positive samples in the
averaged to obtain the reward expectation of the current token
real data set, which is essentially a binary classification task.
and to reduce the granularity of reward guidance to the level
For each sample x, the discriminator outputs the probability
of tokens. The calculation method can be shown in (9):
Dφ (x) of whether x is the real data and sends it back to
the generator as the reward in reinforcement learning. The Qθ (s = Y1:t−1 , a = yt )
 X 
optimization objective of the discriminator is shown in (4), 1 K k k
log Dφ (Y1:T ), Y1:T ∈ MC(Y1:T ; K ) for t<T

as follows: = K k=1
log D (Y ) for t=T
φ 1:t
max EY ∼Pdata [log Dφ (Y )] + EY ∼Gθ [log(1 − Dφ (Y ))] (4)
φ (9)
Because of the discretization of the text sequence, the gen- where K is the times of sampling when calculating the reward
erator cannot solve the gradient of the objective function expectation for each time step. We finally set K = 4 in the
to Dφ . For this problem, the reinforcement learning theory experiment. The increase of sampling times will increase the
can be combined with the sequence generation process as time complexity of the whole network.

VOLUME 8, 2020 105219

 Y. Yang et al.: FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

A. FEATURE GUIDING NETWORK

In the process of generating sequence token by token, the gen-
erator needs the output of the discriminator as the reward
signal of reinforcement learning. However, the comparison
algorithm such as SeqGAN directly transmits the probability
value as the reward to the generator. The generator only uses
this scalar to optimize the network training. The sequence
can only be optimized the training direction through a large
number of samplings and the limited number of sampling
is also very small for a complete spatial distribution. The
gradient of the discriminator Dφ back to the generator is also
very small during training. These problems make the gener-
ator unable to optimize the network parameters effectively. FIGURE 2. The feature vector extracted from the convolutional layer.
In view of the weak guidance of the discriminator feedback
signal, we add a feature guidance module in the generator
network. The purpose of the feature guidance module is to to uncoordinated network training, so a conversion module is
obtain more abundant text features from the discriminator to needed for further processing. In this paper, the network is
guide the generation. named the feature-guiding module.
The reward signal that is fed back to the generator by the The feature vector passed in by the discriminator is
discriminator is a scalar and does not contain much seman- recorded as ft and continues to use an LSTM network for
tic information. The discriminator is a trained convolutional transformation, keeping synchronization with the text gener-
neural network. The input data to be discriminated is first ation module.
processed by word embedding layer, then the semantic fea- The network structure can be shown in (11), as follows:
tures of text vectors are extracted by convolution operations
gt , hC
t = CθC (ft , ht−1 )
C
(11)
of different sizes in convolution layer. Convolutional neural
networks typically connect to the pooling layer to reduce the where C denotes a feature-guiding network, θC denotes the
dimension after the convolutional layer. However, the dif- parameter set of C, ht denotes a hidden state vector of the
ference between positive and negative samples of the text is current time step. The feature-guiding vector of the current
often not on the macro level and it is necessary to consider the time step gt is jointly determined by the feature vector ft of
differences in detail. Therefore, we do not adopt the pooling the current time step and the hidden state vector hCt−1 of the
method. Instead, it is replaced by a convolutional layer with previous time step. To maintain the stability of the network
the step size equal to 1. It is subsequently sent to the output training, gt can be normalized into a unit vector.
layer through the fully connected layer. The output layer uses Because the dimension of the feature-guiding vector is
the sigmoid function. The overall network structure of the inconsistent with the vector that is generated by each time
discriminator can be shown in (10): step of the text generation module, linear conversion is also
Dφ (x) = sigmoid(φoT F(x)) (10) necessary. Considering the stability of the network, the trans-
formation can be combined with the feature vector of the
where φ is the parameter of the discriminator network, φo latest k-step. In this paper, k = 4 is finally selected. The
represents the parameter of the output layer which is the last process can be shown in (12), as follows:
layer of the discriminator. F represents all parameters of the
k
network except the output layer and this part can be regarded X
wt = W gt−i (12)
as a feature extraction network. This output result can be used
i=1
as the intermediate output to the output layer of the discrimi-
nator and can also be sent to the feature guidance module as where W represents the dimensional transformation matrix
the input. Since F is the output vector of the feature extraction and wt represents the dimensionally transformed vector. wt
by the CNN, relative to the scalar of the discriminator output, and the vector generated by text generation module jointly
F contains more detailed semantic information. This vector determine the generation of the next token. The equation of
can represent the mapping of input samples to higher order text generation module is can be shown in (13).
feature spaces. It can provide more detailed guidance for hrt = RθR (Xt , hrt−1 ) (13)
generator optimization. The feature vector is shown in Fig. 2.
The feature vectors extracted by the convolution layer where RθR represents the text generation module and Xt rep-
cannot be directly connected to the input of each time step in resents the sequence representation vector of the current time
the text generation network. The feature vector is extracted step. One dimension of Xt is vocabulary size and the other
for the purpose of discriminator distinguishing positive and dimension is the same as wt . Through matrix multiplication,
negative samples, but each time step of text generation net- we can get a vector of vocabulary size, which represents the
work is to generate residual sequence. Direct access may lead probability that each token in vocabulary is selected. Then,

105220 VOLUME 8, 2020

Y. Yang et al.: FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

the next token of the sequence is selected by the softmax layer

until the end. The probability distribution of the next word zt
can be shown in (14), as follows:

zt = softmax(Xt wt ) (14)

B. TEXT SEMANTIC RULES FOR RESTRICTING FIGURE 3. The mask vector that corresponds to each token.
GENERATION
Sequence generation is the process of token by token gen-
eration. The intermediate time sequence is not generated the word vector. The model is a shallow two-layer neural
completely, but it still needs to evaluate the reward of the network that is used to train to reconstruct the token. After
current generated token in this time step. Because the dis- the training has been completed, the word2vec model can be
criminator can only accept the input of the complete sequence used to map each token to a vector, which can be used to
and the Monte Carlo sampling can supplement the incomplete represent the relationships between tokens. This paper uses
sequence, a large number of sampling operations are needed the cosine distance to characterize the similarity between two
to fill the incomplete sequence in the intermediate time. tokens. Each token in vocabulary is traversed to find out the
Then the complete sequence is fed into the discriminator to tokens with high subsequent relevance in real dataset, so as
determine the reward of the current token and the subsequent to restrict in the subsequent generation process. The current
generation is carried out according to the feedback guidance. token is wordi and the cosine distances are calculated between
Due to the limitation of sampling times, the process of Monte wordnext , which denotes the k tokens that appear next in the
Carlo sampling cannot be fully traversed in the complete real data sequence, and all tokens in the vocabulary. For every
vocabulary space, which has a high randomness. There may token in the vocabulary, If k-times calculation similarity is
be generating sequences with unreasonable semantics, such less than threshold Thsim , set the value to 0 in the correspond-
as subject repetition and verb deletion. ing mask vector, otherwise, set it to 1. Thus, the vector Maski
This paper proposes a method to create a vocabulary mask that corresponds to each token is obtained. The vector can be
based on semantic rules, which restricts the tokens generated shown in Fig. 3 and equations (15)-(15).
in the next time step during the sequence generation. The
specific method is to preprocess the real dataset according Wksimn = 0 if Sim(Wk , Vocan ) < Thsim else 1 (15)
k
to the semantic rules to get the corresponding mask vector of [
each token in the vocabulary. The mask vector represents the Maski = Masknextj (16)
relationship between tokens, such as part of speech, similar- j=1

ity. It represents the tokens that should be restricted in the sub- Through the experimental test, the parameter k is set to 5,
sequent generation when the token appears in the current step. Thsim is set to 0.6. The vector similarity formula can be shown
The dimension of the mask vector is equal to the dimension in (17), as follows:
of the vocabulary. If the parameter value on the dimension Pn
Ai ∗ Bi
is 0, it means that the token corresponding to the dimension similarity = qP i=1 qP (17)
has a very low correlation with the generated prefix sequence n 2 n 2
i=1 Ai ∗ i=1 Bi
and subsequent generation should mask the token. The pur-
pose of mask vector is to eliminate the candidate tokens that where n is the word vector dimension. When generation at
do not conform to the objective context in the generation, each time step, consider the m tokens that are generated by
so that the generation process can learn the implicit semantic the prefix sequence. The masks are Maski , i ∈ [1, m]. The
structure and improve the quality of the final generation final mask is ORed by the mask vectors of the latest m tokens
sequence. at the current time step, which can be shown in (18).
Assuming that the vocabulary size is n and the sequence m
[
length is m, the sampling space is of size nm . In the current MaskT = MaskT −i (18)
time step, if the prefix sequence has been generated, the sam- i=1
pling space size is still the vocabulary size n. However, there Through the experimental test, the parameter m is set
are many alternatives that do not conform to the current to 4. Finally, MaskT is used to mask the token selection
semantics. These tokens should be restricted to subsequent probability of the current time step. Combined with the mask
generation. Although the text sequence has a complex seman- vector MaskT and the feature-guiding vector wt in the feature
tic structure, in the case of the generated prefix sequence, guidance module, the current time step vector Xt in the text
the frequency of occurrence of subsequent tokens has a non- generation module is sent to the softmax layer to determine
random probability distribution. We define rules in terms of the next token generation. The probability of the next token
the word similarity. Each token in the text sequence can be can be shown in equation (19) and Fig. 4.
encoded into a word vector in the word embedding layer.
Word2vec [15] is a related model that is used to generate zt = softmax(Xt wt MaskT ) (19)

VOLUME 8, 2020 105221

 Y. Yang et al.: FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

TABLE 1. Convolution layer parameters.

sequence is judged by the negative log-likelihood (NLL) of

Goracle . The second part of the experiment uses the COCO
FIGURE 4. Sampling through the mask vector. dataset [16] as the real text data for the experiment. The
COCO dataset is a dataset that the Microsoft team can use
for image recognition, which has a manual text description of
C. NETWORK TRAINING each image, and it is used as a real dataset. In the third part,
In the early stage of the GAN, the generator will encounter the experiment was conducted using a Chinese poetry dataset
suboptimal model convergence. Because the underlying in which 8,000 Chinese Tang poems were selected as datasets.
parameters are too random, it is impossible to generate higher Each poem contains five or seven Chinese words per sen-
quality text, which causes the feedback to be a large penalty tence. The evaluation of real data experiment is bilingual
value, which leads to slow convergence of the model. There- evaluation understudy (BLEU) [17]. The BLEU score is used
fore, the maximum likelihood pre-training method is used to compare and count the number of commonly occurring
to reduce the risk. In the pre-training process, the generator n-ary words for the quality evaluation of the generated text.
network uses the cross-entropy as the loss function to quickly Using the above datasets for comparison experiments,
optimize the generator parameters. MLE trained LSTM, SeqGAN, RankGAN, RelGAN and
The improved model is still composed of a generator and FGGAN proposed in this paper are used for text generation
a discriminator, where in the generator includes a feature- comparison tests. The text generation experiments that are
guiding network and a text generation network, both of which conducted in this paper all use free generation by inputting a
are composed of LSTM. The discriminator is composed of a random character as the starting character. Finally, the quality
CNN. Except for the last layer of the output layer classifi- of the generated text is judged by calculating the correspond-
cation, all layers are extracted as feature extraction network. ing evaluation.
The text generation network can use supervised pre-training Generator in FGGAN mainly includes text generation
to accelerate the convergence of the model before the whole module and feature guidance module, both of which adopt
system begins training to avoid the model collapse. LSTM structure. In text generation module, it is specified
In the overall training, the method of adversarial training is that the sequence generation length is 20, the dimension of
still adopted. When training the discriminator, the supervised the input embedding layer is 128 and the dimension of the
training method is used. The real data are regarded as the hidden layer is 128. The feature guidance module uses the text
positive sample and the sequence that is generated by the feature vector extracted from the discriminator as the input
generator is regarded as the negative sample. Also, the cross- and generates the feature guidance vector with dimension 16.
entropy loss function is utilized. When training the generator, The guidance vector and the output of text generation module
the parameters of the discriminator are fixed and the gener- are transformed to determine the generation of the next token.
ator network adopts the token-by-token generation method. The discriminator uses CNN structure for feature extraction
The prefix sequence is oversampled to obtain the complete and classification output. The dimension of the input embed-
sequence and sent to the discriminator to obtain the reward ding layer is 256. The subsequent convolutional layer param-
value. We optimize generator parameters in a gradient descent eters are shown in the following Table 1 and the convolution
way to get the highest reward. step is set to 1. The dimension of text feature vector extracted
by convolutional layer is 1720, which is transferred to feature
IV. EXPERIMENTS guidance module for processing.
To evaluate the performance of the design, an experiment1 L2 regularization with weight parameter of 0.2 is added
with three parts was designed: The first part of the experiment on the basis of cross entropy loss function and dropout is
uses synthetic data for the text generation test. The train- set to 0.75 to avoid over fitting. In the adversarial training
ing data comes from the well-trained LSTM model Goracle 2 of generator and discriminator, the ratio of generator and
disclosed in the paper of SeqGAN, which randomly initial- discriminator training times is 1:3, that is, generator training
izes the LSTM network parameters and randomly generates once and discriminator training three times.
sequences with length of 20 as the real dataset. The effect In the synthetic data experiment, initialize the LSTM
of generator Gθ learning data distribution and generating Model Goracle and generate 10000 sentences as the real data.
Both FGGAN and comparison algorithm use this dataset for
1 the code is available on the https://github.com/danxiaodong/FGGAN text generation. The NLL function of Goracle is used as the
2 the code is available on the https://github.com/LantaoYu/SeqGAN evaluation of data generation effect after each batch of data is

105222 VOLUME 8, 2020

Y. Yang et al.: FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

TABLE 2. Experimental results of NLL on synthetic data. TABLE 3. COCO dataset BLEU score comparison.

FIGURE 5. Synthetic data experimental training curves.

FIGURE 6. BLEU on the COCO dataset.

generated. NLL objectively represents the ability of learning

data distribution of the model. The equation is shown in (20),
as follows:
XT
NLL = EY1:T ∼Gθ [ log Goracle (yt |Y1:t−1 )] (20)
t=1
Each model uses random noise to generate data. The effect
comparison is shown in the Fig 5. In the pretraining,
the FGGAN model has shown a more advantageous effect
on the evaluation compared with other models. After entering
the adversarial stage, the FGGAN convergence speed is better
than other models and the final NLL value is closer to the real
data, as shown in the Table 2.
In the COCO real data experiment, using the image
description sequence of dataset as the target of genera-
tion. The dataset includes 19,383 words and 198,751 sen-
tences. This experiment preprocesses the dataset by deleting
words with frequencies of less than 10 and sentences con-
taining them. The processed dataset includes 5126 words. FIGURE 7. FGGAN training loss on BLEU.
45000 sentences were randomly selected as the training set
and 5000 sentences were used as the test set. The BLEU
scores are presented as following Table 3 and Fig. 6. FGGAN effect after adding the improved module is compared and the
training loss on BLEU is shown in Fig. 7. The results demon- effect scores of each model are shown in the Fig. 8. It can be
strate that the FGGAN model outperforms the compared seen that the improved module in this paper has improved the
models. effect of the baseline model.
In order to verify the enhancement effect of the improved The comparison of real data and generated data on the
module proposed in this paper, we use the COCO dataset COCO dataset is shown in table 4.
for comparative experiments and the network settings are To evaluate the improved model that is proposed in this
consistent with the above.The feature guidance moule and paper, text generation of Chinese poetry was also conducted
the semantic rule module are deleted as the baseline. The and 8000 Chinese Tang poems were selected as data sets.

VOLUME 8, 2020 105223

 Y. Yang et al.: FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

TABLE 5. Chinese poetry dataset BLEU score comparison.

FIGURE 8. The improvement of feature guidance module and semantic

rule module.

TABLE 4. Comparison of real data and generated data on the COCO

dataset.
FIGURE 9. BLEU on the Chinese poetry dataset.

judgement standard. The scores are shown in Table 5 and

Fig. 9. The improved model outperforms other models in
simple poetry generation.

V. CONCLUSION
For text sequence generation, this paper proposes an
improved framework FGGAN. In order to solve the problem
that the feedback signal of the discriminator is not very
instructive, this paper proposes a feature guidance module
which obtains the text semantic feature feedback to the gen-
erator with more guidance. In addition, this paper proposes
a method to create a vocabulary mask based on semantic
rules which restricts the tokens during the generation to make
the sequence more realistic. The superiority of the improved
module is evaluated experimentally. In the synthetic experi-
ment, the negative log-likelihood is used for the evaluation.
FGGAN proposed in this paper has higher ability of fit-
ting data distribution. In the experiment on real data, BLEU
is used for the evaluation. Compared with other models,
FGGAN has a higher evaluation score and generates more
realistic text data.

VI. DISCUSS
Compared with other comparison algorithms, FGGAN in this
paper has some improvements in the relevant dataset, but
there are still some problems that need further improvement.
Firstly, the feature guidance module extracts the text features
from the discriminator and sends them to the text generation
module for guidance after transformation. However, because
Each poem contains five or seven Chinese words per sen- the linear transformation may not be able to adapt to the
tence. Among them, 4000 poems are used as training sets, high-speed changing feature space of the discriminator CNN,
4000 are used as test sets and BLEU-2 is used as the result the text generation module may not learn advanced semantic

105224 VOLUME 8, 2020

Y. Yang et al.: FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

features. Further research on extraction and transformation YANG YANG was born in 1981. She received the
of text feature is needed in the future. Secondly, in the pro- Ph.D. degree from the Beijing University of Posts
and Telecommunications, in 2011. She is currently
cess of using semantic rules to restrict sampling, the mask an Associate Professor. More than 30 articles have
vector is obtained by preprocessing the dataset according been published in SCI/EI journals and one inter-
to the semantic rule. However, too complex rule restriction national standard have been applied. Meanwhile,
will cause the neural network to mode collapse, resulting in she is the paper review experts of journals of
Sensors, the International Journal of Distributed
some duplicate text sequences. Subsequent work related to Sensor Networks, and ICC conference. Her current
semantic rule optimization needs to be carried out. research interests are big data analysis and trend
depth prediction. She is the Session Chair of CENET2018 International
REFERENCES Conference Internet of Things and big data analysis.
[1] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,
S. Ozair, A. Courville, and Y. Bengio, ‘‘Generative adversarial nets,’’
in Proc. Adv. Neural Inf. Process. Syst., New York, NY, USA, 2014,
pp. 2672–2680.
[2] L. Yu, W. Zhang, J. Wang, and Y. Yu, ‘‘Seqgan: Sequence generative
adversarial nets with policy gradient,’’ in Proc. AAAI, San Francisco, CA,
XIAODONG DAN was born in 1994. He received
USA, 2017, pp. 2852–2858.
[3] S. Hochreiter and J. Schmidhuber, ‘‘Long short-term memory,’’ Neural the bachelor’s degree from Xidian University,
Comput., vol. 9, no. 8, pp. 1735–1780, 1997. in 2017. He is currently pursuing the master’s
[4] T. H. Wen, M. Gasic, and N. Mrksic, ‘‘Semantically conditioned degree in computer science wit the Institute of Net-
LSTM-based natural language generation for spoken dialogue systems,’’ work Technology, Beijing University of Posts and
Comput. Sci., vol. 3, no. 17, pp. 144–152, 2015. Telecommunications. His main research interests
[5] C.-H. Chang, C.-H. Yu, S.-Y. Chen, and E. Y. Chang, ‘‘KG-GAN: include big data analysis and machine learning.
Knowledge-guided generative adversarial networks,’’ 2019,
arXiv:1905.12261. [Online]. Available: http://arxiv.org/abs/1905.12261
[6] S. Lian, H. Zhou, and Y. Sun, ‘‘FG-SRGAN: A feature-guided super-
resolution generative adversarial network for unpaired image super-
resolution,’’ in Proc. ISNN, Moscow, Russia, 2019, pp. 151–161.
[7] M. Arjovsky and L. Bottou, ‘‘Towards principled methods for training
generative adversarial networks,’’ in Proc. ICLR, Toulon, France, 2017,
pp. 124–131.
[8] T. Che, Y. Li, R. Zhang, R. Devon Hjelm, W. Li, Y. Song, and XUESONG QIU was born in 1973. He received
Y. Bengio, ‘‘Maximum-likelihood augmented discrete generative adver-
the Ph.D. degree from the Beijing University of
sarial networks,’’ 2017, arXiv:1702.07983. [Online]. Available: http://
Posts and Telecommunications, Beijing, China,
arxiv.org/abs/1702.07983
[9] Y. Zhang, Z. Gan, K. Fan, Z. Chen, R. Henao, D. Shen, and L. Carin, in 2000. He is currently a Professor and the Ph.D.
‘‘Adversarial feature matching for text generation,’’ in Proc. ICML, Supervisor. He has authored about 100 SCI/EI
Sydney, NSW, Australia, 2017, pp. 4006–4015. index articles. He presides over a series of key
[10] H. Su, X. Shen, P. Hu, W. Li, and Y. Chen, ‘‘Dialogue generation with research projects on network and service man-
GAN,’’ in Proc. AAAI, New Orleans, LA, USA, 2018, pp. 8163–8164. agement, including the projects supported by the
[11] W. Fedus, I. Goodfellow, and A. M. Dai, ‘‘MaskGAN: Better text gen- National Natural Science Foundation and the
eration via filling in the,’’ in Proc. ICLR, Vancouver, BC, Canada, 2018, National High-Tech Research and Development
pp. 1–17. Program of China. He received 13 national and provincial scientific and
[12] C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, technical awards, including the national scientific and technical awards
P. Rohlfshagen, S. Tavener, D. Perez, S. Samothrakis, and S. Colton, (second-class) twice.
‘‘A survey of Monte Carlo tree search methods,’’ IEEE Trans. Comput.
Intell. AI Games, vol. 4, no. 1, pp. 1–43, Mar. 2012.
[13] C. Zhang, C. Xiong, and L. Wang, ‘‘A research on generative adversarial
networks applied to text generation,’’ in Proc. 14th Int. Conf. Comput. Sci.
Edu. (ICCSE), New Orleans, LA, USA, Aug. 2019, pp. 1268–1288.
[14] K. Lin, D. Li, X. He, Z. Zhang, and M.-T. Sun, ‘‘Adversarial ranking
for language generation,’’ in Proc. NIPS, Long Beach, CA, USA, 2017, ZHIPENG GAO was born in 1980. He received
pp. 3155–3165. the Ph.D. degree from the Beijing University of
[15] T. Mikolov, K. Chen, G. Corrado, and J. Dean, ‘‘Efficient estimation of
Posts and Telecommunications, Beijing, China,
word representations in vector space,’’ 2013, arXiv:1301.3781. [Online].
in 2007. He is currently a Professor and the
Available: http://arxiv.org/abs/1301.3781
[16] X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Dollar, and
Ph.D. Supervisor. He presides over a series of
C. L. Zitnick, ‘‘Microsoft COCO captions: Data collection and evalua- key research projects on network and service
tion server,’’ 2015, arXiv:1504.00325. [Online]. Available: http://arxiv. management, including the projects supported by
org/abs/1504.00325 the National Natural Science Foundation and the
[17] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, ‘‘BLEU: A method National High-Tech Research and Development
for automatic evaluation of machine translation,’’ in Proc. 40th Annu. Program of China. He received eight provincial
Meeting Assoc. Comput. Linguistics (ACL), Philadelphia, PA, USA, 2002, scientific and technical awards.
pp. 311–318.

VOLUME 8, 2020 105225