ISA Transactions xxx (xxxx) xxx

Contents lists available at ScienceDirect

ISA Transactions
journal homepage: www.elsevier.com/locate/isatrans

Practice article

Intelligent fault identification for industrial automation system via

multi-scale convolutional generative adversarial network with
partially labeled samples
∗
Tongyang Pan a , Jinglong Chen a , , Jinsong Xie b , Yuanhong Chang a , Zitong Zhou a
a
State Key Laboratory for Manufacturing and Systems Engineering, Xi’an Jiaotong University, Xi’an 710049, PR China
b
School of Traffic & Transportation Engineering, Central South University, Changsha 410075, PR China

article info a b s t r a c t

Article history: Rolling bearings are the widely used parts in most of the industrial automation systems. As a result,
Received 24 September 2019 intelligent fault identification of rolling bearing is important to ensure the stable operation of the
Received in revised form 8 January 2020 industrial automation systems. However, a major problem in intelligent fault identification is that it
Accepted 8 January 2020
needs a large number of labeled samples to obtain a well-trained model. Aiming at this problem,
Available online xxxx
the paper proposes a semi-supervised multi-scale convolutional generative adversarial network for
Keywords: bearing fault identification which uses partially labeled samples and sufficient unlabeled samples
Fault diagnosis for training. The network adopts a one-dimensional multi-scale convolutional neural network as the
Rolling bearing discriminator and a multi-scale deconvolutional neural network as the generator and the model is
Deep learning trained through an adversarial process. Because of the full use of unlabeled samples, the proposed
Intelligent fault identification
semi-supervised model can detect the faults in bearings with limited labeled samples. The proposed
method is tested on three datasets and the average classification accuracy arrived at of 100%, 99.28%
and 96.58% respectively Results indicate that the proposed semi-supervised convolutional generative
adversarial network achieves satisfactory performance in bearing fault identification when the labeled
data are insufficient.
© 2020 ISA. Published by Elsevier Ltd. All rights reserved.

1. Introduction In mechanical fault detection, vibration signals are considered

to contain more useful fault information since they are collected
Industrial automation is the core of intelligent manufacturing by the acceleration sensors which are directly attached on the
equipment. In the last few years, industrial automation is suc- equipment. In order to analyze these signals and extract the
cessfully implemented in intelligent manufacturing industries [1]. useful information hidden in the noise, many signal processing
During the operation of the industrial automation systems, faults methods are proposed [3–6]. However, these methods are inca-
inevitably occur in these intelligent equipment, which may cause pable of extracting deep features automatically and also shows
serious economic losses. Consequently, effective fault identifica- the weakness of low efficiency and low accuracy.
tion and detection for these equipment is important to avoid Recently, deep learning shows great potential in fault detec-
accidents and ensure the operation reliability of the automation
tion as it is able to process vibration signals efficiently and does
systems.
not require too much prior knowledge [7]. As a consequence,
Rolling bearing is the most basic and commonly used com-
intelligent fault identification is widely used to solve the prob-
ponent in industrial automation systems. However, bearings are
lems that traditional methods failed. Shao et al. [8] adopted the
also the fragile part and may easily fail during operation because
of the extremely complex and tough working conditions such as Gaussian wavelet to replace the original activation function of the
high temperature and high pressure. According to the statistics, deep neural network to construct the deep wavelet auto-encoder
faults caused by rolling bearings takes up approximately 30% of (AE) for electric locomotive fault identification. Dong et al. [9]
the mechanical faults [2]. As a result, fault identification and de- employed the time–frequency analysis to transform the one di-
tection of rolling bearings are of vital importance for automation mensional signals into two dimensional figures. These figures are
systems. then put into a hybrid model which consists of an unsupervised
convolutional neural network (CNN) and a deep belief network
∗ Corresponding author. (DBN) Wu et al. [10] proposed an optimized CNN to learn features
E-mail addresses: [email protected], [email protected] from vibration signals for rotating machinery fault detection.
(J. Chen). Pan et al. [11] proposed an intelligent diagnosis model with the

https://doi.org/10.1016/j.isatra.2020.01.014
0019-0578/© 2020 ISA. Published by Elsevier Ltd. All rights reserved.

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
2 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx

similar structure of lifting scheme which successfully classifies uses unlabeled data and limited labeled data for training and
the different fault conditions even with different rotating speeds. multi-scale analysis is also employed for deep feature extraction.
From the mentioned methods above, it can be found that The contributions of the paper can be summarized as follows:
intelligent fault detection shows great potential in fault feature
extraction to support decision making for maintenance man- (1) A semi-supervised fault detection method based on GAN
agement. In existing methods, an assumption is that sufficient is proposed to solve the problem that labeled data are
labeled data are always available to train an intelligent detection insufficient in real industrial manufacturing. In this paper,
model well [12]. However, in real industry applications, most of both the labeled data and unlabeled data are used to train
the monitoring data are unlabeled and labeling all the data is the model well.
a waste of money and labor [13]. Consequently, the mentioned (2) One-dimensional multi-scale convolution neural network
supervised models may be trained poorly and perform bad in (MSCNN) is adopted in the discriminator to make the
fault detection without sufficient labeled data. For this purpose, it model more effective in feature extraction and fault identi-
is considered to be a stable and robust implementation to exploit fication. Because of the multi-scale characteristics of vibra-
unlabeled data with few labeled data to train a deep learning tion signals, MSCNN is capable of extracting more useful
model [14]. fault information compared with traditional models.
Nowadays, plenty of semi-supervised learning methods have (3) Fractionally strided convolution (FSC) is employed to con-
been implemented to solve the problem that labeled samples struct the multi-scale deconvolutional neural network (MS-
are insufficient. Tao et al. [15] proposed a semi-supervised sup- DCNN) which is designed as the generator. The applica-
port vector machine (SVM) based on Laplacian regularization to tion of FSC makes it possible to generate high-dimensional
identify the common bearing faults. Jiang et al. [16] presented a signals from low-dimensional noise via convolution opera-
semi-supervised fault detection model mainly based on the dy- tions.
namic sparse stacked AEs. Luo et al. [17] adopted the orthogonal (4) Three case studies are carried out to show the effectiveness
semi-supervised space alignment method to extract features and of this semi-supervised fault detection method. Further-
a transductive SVM was employed for fault classification. Razavi- more, the proposed detection is also compared with several
Far et al. [18] adopted semi-supervised deep ladder network for related intelligent fault detection methods. The result indi-
gearbox fault diagnosis and several traditional signal processing cates that semi-supervised learning based multi-scale con-
methods were used for feature extraction. Dai et al. [14] com- volutional generative adversarial network shows a better
bined ensemble learning and semi-supervised learning to fully performance than the traditional methods.
exploit the information in unlabeled data. Zhao et al. [19] pre-
The rest of the paper are organized as follows: Section 2 will
sented a new sparse coding method for semi-supervised bearing
give a brief introduction of GAN and then present the proposed
fault detection. As can be seen from these mentioned meth- method in detail. In Section 3, data from three different exper-
ods, although these methods reach high classification accuracy iments will be analyzed for testing. In Section 4, the influence
of bearing faults, a significant problem is that they still rely of some key parameters is discussed. And in the last section, a
too much on the signal processing algorithms, which may limit conclusion will be drawn from the above analysis.
their applications in modern industry [20,21]. In this regard, the
paper proposes a semi-supervised fault detection method using 2. Proposed method
multi-scale convolutional generative adversarial network which
extracts deep features automatically and uses limited labeled 2.1. Generative adversarial network
samples to train a model well.
Generative adversarial network (GAN) is a popular unsuper- Inspired by binomial zero-sum game theory, GAN is trained via
vised deep neural network proposed by Goodfellow in 2014 [22]. an adversarial process [30]. Generally, GAN is a generative model
As a generative model, GAN can generate data which have the which intends to learn the distribution of the given data and
similar distribution with the given data and has been widely generate similar data with the same distribution [31]. The classic
used in various areas [23–26]. The great success of GAN attracts GAN model consists of a generator G that captures the potential
attention from many researchers and has been gradually used distribution and a discriminator D that determines whether the
in fault detection. Liu et al. [27] adopted an improved GAN to input data are original data or the synthetic. Finally, the objective
generate realistic fault data and the synthetic data were then function of GANs can be written as follows:
used to establish a fault identification model. Wang et al. [28]
combined GAN and stacked denoising auto-encoders to generate min max (D, G) = Ex∼pdata (x) [log D (x)] + Ex∼pz (z)
G D
samples which has the similar distribution with original vibration
signals. This method showed a better fault diagnosis performance × [log (1 − D (G (z )))] (1)
with small samples. Liu et al. [29] proposed categorical adver- What is more, both of these two parts are usually deep neu-
sarial auto-encoder to classify different working conditions and ral networks which can be convolutional neural network, auto-
achieved high clustering indicators even load and noise changed. encoder or other models. Recently, many techniques are adopted
It can be seen that as a generative model, GAN not only generate to improve the performance of GANs such as the semi-supervised
samples with similar distribution but also can be improved for GAN [32]. The main difference between the standard GAN and the
fault classification. However, related studies focus on generating semi-supervised GAN is that the semi-supervised GAN employs a
labeled samples for training, which ignores the much useful in- softmax classifier to replace the original binary classifier in the
formation in the unlabeled samples. As is mentioned, labeled data discriminator. More specially, a standard softmax classifier takes
are expensive and insufficient while unlabeled data are relatively in x as input and outputs a n-dimensional vector as follows:
easier to obtain. Thus, it is necessary to carry out researches to
eθi
Tx
explore the information in the unlabeled data.
pmodel (y = i |x, θ ) = ∑ (2)
Aiming to solve the serious problem of lacking labeled data n θjT x
j=1 e
and make full use of unlabeled data, the paper combines the
GAN with semi-supervised learning for bearing fault identifica- where θ is the weight matrix of softmax classifier and n repre-
tion and fault detection. The proposed method simultaneously sents number of different categories.

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx 3

Fig. 1. Architecture of proposed fault identification model.

In order to realize semi-supervised learning with GANs, a new

class y = n + 1 is added to the softmax classifier which represents
the generated samples. That is, when an original sample is put
into softmax classifier, the classifier outputs the probability that
which category it belongs to. When the input is a synthetic
sample, the classifier outputs whether it is fake. As a result, the
original dimension of the classifier outputs increases from n to Fig. 2. Illustration of fractionally strided convolution.
n + 1.

2.2. Fault detection based on proposed method

8 and 16 respectively are then employed for multi-scale analysis
as follows:
Aiming at the problem that traditional intelligent fault de- ( )
(2)
tection models may be trained poorly with insufficient labeled di,j = g (2) fj (si ) , i, j = 1, 2, 3, 4 (5)
samples, the paper proposes a semi-supervised multi-scale con-
(2)
volutional generative adversarial network to identify the faults in where fj is the convolution operation from jth scale in the
industrial automation systems. As shown in Fig. 1, the proposed second convolution layer.
fault detection model consists of two neural networks, a one- Consequently, features from sixteen scales can be obtained
dimensional MSCNN as the discriminator and a one-dimensional after two convolution and max-pooling blocks. These extracted
MSDCNN as the generator. features are then concatenated together for further analysis as
As is mentioned, the discriminator in this paper is a one- follows:
dimensional MSCNN which mainly contains two convolution and
D = d1,1 , d1,2 , . . . , d4,3 , d4,4
[ ]
pooling blocks. After that, the extracted features are put into a (6)
flatten layer and a softmax layer for further analysis. In order As the generator, the one-dimensional MSDCNN shares nearly
to realize multi-scale feature extraction, the signals are analyzed the same structure with the MSCNN. And the only difference is
by convolutional kernels with different sizes. The first convo- that the convolution layers in the MSCNN are replaced by the de-
lutional layer employed four convolution kernels with different convolution layers. In order to generate the high-dimensional sig-
sizes which are 2, 4, 8 and 16 to realize multi-scale analysis as nals from low-dimensional random noise, the fractionally
follows: strided convolution is adopted in each deconvolutional layer to
u = [f1 (x), f2 (x), f3 (x), f4 (x)] (3) realize deconvolution operations. The illustration of FSC is shown
in Fig. 2. It can be found that the FSC puts zeros into the vibration
where fi represents the convolution operation under ith scale. x signals and then carries out convolution operations. The major
is the input signal while u is the extracted features. [·] is the difference between the FSC and the commonly used convolution
concatenation operation. is that the stride of FSC is less than one.
For each scale, only one convolution kernel is used for fea- Except for the FSC, the overall structure of the generator is
ture extraction. Thus, features extracted from four scales can be similar to the discriminator. In the first deconvolutional layer,
obtained after the first convolution and pooling operations as four convolution kernels with different sizes which are 2, 4, 8 and
follows: 16 are employed to realize multi-scale analysis as follows:
( )
(1)
si = g (1) fi (x) , i = 1, 2, 3, 4
( )
(1)
(4) hi = g (1) fˆi (z ) , i = 1, 2, 3, 4. (7)
(1)
where fi represents the convolution operation from ith scale where z is the random noise and hi is the generated data from
while g (1) is the max-pooling operation. s is the extracted features ith scale. fˆ (1) represents the deconvolution operation in the first
after the first convolution and pooling block. deconvolution layer.
Similarly, in the second convolutional layer, for features ob- Similarly, in the second deconvolutional layer, four convolu-
tained from each scale, four convolution kernels with sizes of 2, 4, tion kernels with sizes of 2, 4, 8 and 16 are then adopted to

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
4 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx

analyze features from each scale. Therefore, features from totally to compare with the proposed semi-supervised adversarial net-
sixteen scales are obtained as follows: work. First, a traditional intelligent fault detection method which
(
(2)
) uses manually selected features and SVM for fault classification
ei,j = g (2) fˆj (hi ) , i, j = 1, 2, 3, 4 (8) is adopted. Twelve commonly used time-domain feature indexes
are used for feature extraction which are square root amplitude,
It can be found that the main difference between the generator
mean value, absolute mean value, standard deviation, kurtosis,
and the discriminator is that the generator adopts FSC to generate
variance, skewness, skewness index, kurtosis index, waveform
high dimensional features via convolution operations. There are
index, peak index, pulse index respectively. After extracting these
four kernels in the first layer and sixteen kernels in the second in
features, a SVM is used for classification.
both the discriminator and the generator.
During the training process, the generator is used to generate
3.1.2. Graph based semi-supervised method
a fake sample to fool the discriminator. Thus, the input of the
In this paper, a semi-supervised method based on graph based
discriminator may be a fake sample, a labeled sample or an
label propagation is also employed to compare with the proposed
unlabeled sample. The responsibility of the discriminator is to
semi-supervised method. The label propagation algorithm (LPA)
distinguish these three kinds of samples as far as possible. In each
propagates the label from labeled samples to unlabeled samples
iteration, a real sample (labeled or unlabeled) and a synthetic
according to the similarity between different samples. Similarly,
sample will be put into the discriminator simultaneously. When
the 12 time-domain features mentioned in Section 3.1 are firstly
the input sample is real, the first task of the discriminator in this
extracted. And then, a certain percentage of the features are used
model is to tell the real sample from the fake one. After that, the
as labeled data and the rest are considered as unlabeled. Finally,
real sample will be further analyzed by the discriminator. If the
the label propagation algorithm is trained using both labeled and
real sample is also labeled, the discriminator will carry out classi-
unlabeled data.
fication operations. Thus, the loss function of the discriminator is
composed of two parts when the input is a real sample as follows:
3.1.3. MSCNN based method
In order to show whether unlabeled samples influence the
Lr = Ldr + α Lcr (9) training process, another comparison method is to construct a
one-dimensional MSCNN which has the same structure with
where Ldr
represents the loss of the first task while shows the Lcr
the discriminator in the proposed method. As it is a super-
cross entropy of the classification task. Recall that the input may
vised model, the MSCNN is trained only with the labeled sam-
be an unlabeled sample. Therefore, α is set to zero when the input
ples. By training the MSCNN with the same number of samples,
sample is unlabeled while it is one when the input is labeled.
the connection between the MSCNN and the proposed method
For a fake sample, it is generated by the generator and then
will be shown clearly while superiority of the semi-supervised
analyzed by the discriminator. Thus, it has influence on both the
generative adversarial network can be also found.
generator and the discriminator. Therefore, the final loss of the
discriminator and the generator can be written as follows:
3.1.4. Other related studies
Ld = Ldr + α Lcr + λLf Finally, two related researches about intelligent fault identifi-
(10)
Lg = Lf cation are also are investigated for comparison. The first one is a
LeNet-5 CNN based method [33] while the second one is a deep
where Lf is the loss of the input fake sample. Ld and Lg are
belief network based method [34].
the total loss of the discriminator and the generator. λ is a
hyper-parameter to tune the trade-off between these two losses.
3.2. Case 1: Case western reverse university bearing dataset
Through an adversarial process, the discriminator and gener-
ator are trained alternately. In this paper, the back propagation
The rolling bearing dataset provided by Bearing Data Center of
algorithm is used to train the whole semi-supervised adversarial
Case Western Reverse University (CWRU) will be first analyzed in
network. Therefore, the parameters of the generator and the
this section to test the effectiveness of the proposed method. In
discriminator can be automatically updated as follows:
this experiment, bearings are manufactured with a few common
∂ Ld ∂ Ld + α∂ Lcr + λ∂ Lf but different faults using electro-discharge machining. The fault
θd : = θd − µ = θd − r (11) diameters are designed to be 0.007, 0.014, 0.028 and 0.04 in. The
∂θd ∂θd
∂ Lg experiment rig is composed of three parts mainly which are a
θg : = θg − µ (12) motor, a torque transducer and a dynamometer. In this paper, the
∂θg
monitoring data are collected under a 12 kHz sampling frequency
where µ is the learning rate. θd and θg are the parameters in the and four different working conditions are analyzed. More detailed
discriminator and the generator. information is also available on the website of the mentioned data
In the testing process, only the trained discriminator will be center [35] and a few closely related literatures [36,37].
used as a classifier to detect the faults. More specifically, the dis- For further study, the different working conditions are labeled
criminator will be taken out from the multi-scale convolutional as: (1) normal condition (NC), (2) roller fault (RF), (3) inner race
generative adversarial network as a classifier while the trained fault (IRF) and (4) outer race fault (ORF). Totally, 80 samples are
parameters remain the same. collected and there are 2048 data points in each sample. Assum-
ing that 40 samples are labeled, the rest 40 samples are used as
3. Case study unlabeled samples in the training process. That is, each working
condition contains only 10 labeled samples for training. What is
3.1. Comparison methods more, there are 160 samples with the same sample length in the
testing data. Vibration signals from different working conditions
3.1.1. SVM based method are shown in Fig. 3.
In this section, several popular fault identification models are The proposed multi-scale convolutional generative adversarial
established for comparison. Meanwhile, a few closely related re- network is trained for 200 loops using both the labeled and unla-
searches about intelligent fault identification are also introduced beled samples. After training, the discriminator will be taken out

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx 5

Table 1
Classification result in Case 1.
Method Number of Number of unlabeled Accuracy
labeled samples samples (%)
SVM based 40 0 100
MSCNN based 40 0 98.12
LPA based 40 40 100
Proposed method 40 40 100

Table 2
Classification result in Case 1.
Method Number of labeled Sample length Accuracy
Fig. 3. Waveforms of the collected vibration signals in Case 1. Top-left: NC,
samples (%)
top-right: ORF, bottom-left: IRF, bottom-right: RF.
LeNet-5 CNN based 6400 4096 99.79
DBN based 500 4096 99.03
Proposed method 40 2048 100

Table 3
Classification result in Case 2.
Method Number of Number of unlabeled Accuracy
labeled samples samples (%)
SVM based 245 0 81.31
MSCNN based 245 0 83.53
LPA based 245 245 82.86
Proposed method 245 245 99.28

labeled and unlabeled samples. The number of labeled samples

and unlabeled samples are the same with that in the proposed
method. Results show that the SVM based method can reach
an accuracy of 100% as shown in Table 1. Moreover, the aver-
age fault classification accuracy of the MSCNN based method is
98.12%, which implies that the MSCNN can identify the faults in
automation systems effectively.
As mentioned, two related researches are also used to ana-
lyze the CWRU dataset. According to the results in these two
papers [33,34], the classification accuracy is 99.79% for the LeNet-
5 CNN based method and 99.03% for the DBN based method.
However, although these two methods are also satisfactory, too
much labeled data are required to train the model well. Table 2
shows the detailed information about the training data and the
identification result.
It can be found from the table that the LeNet-5 CNN based
model used 6400 labeled samples while the DBN based method
collected 500 labeled samples. Practically, it is difficult to collect
too much labeled data for training. Compared with these meth-
ods, only a small number of labeled samples are used to train the
proposed model, which is much more suitable for real industry
applications. In conclusion, the proposed method can detect the
faults in rolling bearings effectively even with limited labeled
Fig. 4. Experiment results in Case 1. samples, which is meaningful for engineering application.

3.3. Case 2: Spectra quest dataset

for fault identification. It can found from the experimental results
that the average fault identification accuracy of the proposed The analysis of CWRU dataset proves that the proposed semi-
semi-supervised adversarial method arrives at 100% during test.
supervised adversarial method can identify the faults in industrial
What is more, in order to show the experimental results clearly,
systems even with few labeled samples. However, in Case 1, the
t-distribution stochastic neighbor embedding (t-SNE) is adopted
fault identification task is relatively easy for the reason that the
to visualize the extracted features in the discriminator. And then,
the confusion matrix of fault identification results is also given as faults manufactured in bearings are heavy. Therefore, in order to
shown in Fig. 4. verify the performance of the semi-supervised generative adver-
For comparison, three methods mentioned above are carried sarial network further, another case about bearing fault detection
out. In the SVM and MSCNN based methods, the training and is conducted.
testing data keep the same with the proposed method. As the The data are collected on the Spectra Quest (SQ) Machinery
SVM and MSCNN are supervised models, none of unlabeled sam- Fault Simulator. As is shown in Fig. 5, the experiment platform
ples are used for training. Besides, the LPA based method which consists of a magnetic brake as load, a motor, a recorder, a rotor
is also a semi-supervised model needs be trained with both the and some acceleration sensors.

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
6 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx

Fig. 5. Figure of experiment platform.

In Case 2, bearings are manufactured with different single

point faults to simulate the common working conditions. In this
section, totally seven different working conditions are analyzed
which are normal condition, inner race fault with different sever-
Fig. 7. Waveforms of vibration signals from each working condition. From top
ities (minor, medium and severe) and outer race fault with dif- to bottom, from left to right are NC, IR-1, IR-2, IR-3, OR-1, OR-2, OR-3.
ferent severities (minor, medium and severe). For convenience,
these conditions are referred to simply as NC, IR-1, IR-2, IR-3,
OR-1, OR-2 and OR-3 as shown in Fig. 6.
The results show that the MSCNN based method achieves the
As is mentioned, experiments are carried out to collect vibra-
highest accuracy among three comparison methods which is ap-
tion signals for further analysis. The rotating frequency of the proximately 83.53%, implying that these two supervised models
motor is set to 40 Hz while the sampling frequency is set to may be trained poorly without sufficient labeled data. Meanwhile,
25.6 kHz. The training data contains 490 samples and there are the LPA based method reaches 82.86%, indicating that the LPA
also 2048 data points in each one. In order to verify the semi- based semi-supervised method performs much worse than the
supervised learning methods, 245 samples are used as labeled proposed semi-supervised method. What is more, although the
samples and there are 35 samples from each working condition. MSCNN shares the same structure with the discriminator in the
What is more, the rest samples are used as unlabeled ones. proposed method, it still performs bad. It implies that in the
Similarly, another 280 samples are collected and used for testing. proposed method, information in both unlabeled samples and
The vibration signals which will be analyzed in this section are labeled samples can be fully used for training, which leads to the
shown in Fig. 7. better performance than other methods.
Both the labeled and unlabeled samples are put into the net-
work for semi-supervised learning. After 200 loops, repeated 3.4. Case 3: Shipborne antenna dataset
experiment results indicate that the proposed semi-supervised
fault identification model can achieve an average accuracy of In this section, a more complex problem about rolling bearing
99.28% during test. Similarly, the visualization of the features in fault detection is taken into consideration. For further analysis,
the deep neural network and the confusion matrix are both given a high-precision full-size experiment platform is assembled to
to show the classification results of the proposed method in Fig. 8. simulate the common bearing faults in the transmission system
As is mentioned, the SVM and MSCNN based models are also of shipborne antenna. In a real transmission system of shipborne
established out for comparison with 245 labeled samples. Besides, antenna, there are two main transmission chains which are az-
the LPA based method is trained with 245 labeled samples and imuth axis and pitch axis respectively. As can be seen in Fig. 9,
245 unlabeled samples. After training and testing, the results are the experiment platform is also composed of two transmission
shown in Table 3. chains. Besides, there is a reducer and a motor in each chain. All

Fig. 6. Pictures of manufactured fault bearings.

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx 7

Fig. 8. Experiment results in Case 2.

the motors, bearings and reducers are the same with that in a
real ship. As the collected vibration signals contain more noise
and less useful fault information, fault detection in Case 3 is much
more difficult.
As can be seen in Fig. 10, bearings are similarly manufactured Fig. 9. Experiment platform of shipborne antenna transmission system.
with different defects. In this section, six different bearing faults
are considered which are roller fault, retainer fault, outer race
pitting fault and outer race fault with different severities (mi-
nor, medium and severe). Similarly, the different conditions are
abbreviated as ROF, REF, OR-P, OR-1, OR-2 and OR-3.
Based on the operation way of the real shipborne antenna,
the rotating speed is set to 5 Hz while the sampling frequency
is 5 kHz. For each bearing faults, totally 40 samples are obtained
from the experiments. As a results, there are 280 samples in
the training data of which the sample length is 1024. To test
the proposed semi-supervised method, half of these samples are
considered as labeled data while the others are used as unlabeled
data. Besides, another 210 samples from the experiment as used
as testing data. Fig. 11 shows the waveforms of vibration signals
collected from the experiment platform.
The proposed network is trained using both the unlabeled and
labeled data for 100 loops. Finally, the accuracy of the proposed
semi-supervised method arrives at 96.58% during testing. The
visualization of the extracted features in the discriminator and
the confusion matrix of the fault identification results are shown
in Fig. 12.
Three comparison methods are also carried out and are trained Fig. 10. Pictures of fault bearings.
with the same samples. The SVM and MSCNN based models are

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
8 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx

Table 4
Classification results in Case 3.
Method Number of Number of unlabeled Accuracy
labeled samples samples (%)
SVM based 140 0 66.34
MSCNN based 140 0 84.29
LPA based 140 140 81.43
Proposed method 140 140 96.58

data for training. The information about training samples and

fault identification results are shown in Table 4.
As can be seen in the table, the SVM achieves the lowest
accuracy in Case 3. On one hand, insufficient training samples
may cause poor training of the SVM. On the other hand, these
manually selected features may be also insensitive to the faults.
The table also shows that the LPA based method which uses
the same features with the SVM reaches an accuracy of 81.43%,
up by almost 15%. What is more, compared with the MSCNN
based method whose accuracy is the highest in three comparison
Fig. 11. Waveforms of vibration signals from each working condition. From top
to bottom, from left to right are NC, ORF-1, ORF-2, ORF-3, OR-P, ROF, REF.
methods (84.29%), the proposed method reaches an accuracy of
96.58%, which clearly shows that the semi-supervised GAN can
improve the fault diagnosis ability.

4. Discussion

4.1. Number of labeled samples

In Section 3, the proposed semi-supervised adversarial method

is benchmarked on three datasets and shows great potential
in fault diagnosis. Different from popular intelligent fault diag-
nosis, this semi-supervised multi-scale convolutional generative
adversarial network takes a small number of labeled samples for
training and stills performs well. In this paper, the number of
labeled samples is selected manually, which may influence the
fault identification results. As a result, discussion about the num-
ber of labeled samples will be given. During the training process,
the number of labeled samples changes while that of unlabeled
samples remains the same which is 40 in Case 1 and 245 in Case
2. Similarly, the MSCNN based method used as comparison in
Section 3 is also adopted to compare with the proposed method.
As MSCNN is a supervised deep learning model, none of unlabeled
samples are used for training. The experiment result is shown in
Fig. 13.
It can be found that when the number of unlabeled samples
keeps constant in two cases, with more labeled samples, the
identification accuracy of the proposed method also increases.
Owing to space constrains, the similar phenomenon in Case 3
is not shown in this figure. Over all, the proposed convolutional
generative adversarial network method performs better than the
MSCNN based method when these two methods use same num-
ber of labeled samples. It implies that labeled data are necessary
to obtain a well-trained model. With more labeled data, more
fault information may be captured and the classification accuracy
also increases. Meanwhile, much fault information can be also
captured in unlabeled samples. By using unlabeled samples for
training, the proposed semi-supervised method achieves better
performance than the supervised method.
Fig. 12. Experiment results in Case 3. Consequently, when the labeled samples are insufficient, the
unlabeled samples play an irreplaceable role in fault diagnosis.
The proposed method which fully uses both the labeled and
trained with the labeled data as they are supervised models. And unlabeled samples shows great potential to solve the problem in
the LPA based method takes the labeled data as well as unlabeled practical industrial manufacturing.

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx 9

and the discriminator are improved with multi-scale analysis, the

classification accuracy achieves the highest in all three cases.

4.3. Influence of noise

In order to explore the influence of noise on the classifica-

tion results, further researches are carried out in this section.
In this section, additional white Gaussian noise are added into
the vibration signals to simulate the complex background noise
during test. By changing the standard deviation of the Gaus-
sian noise, signals with different signal-to-noise ratios can be
obtained. Moreover, the LPA based semi-supervised adversarial
model is used for comparison. Fig. 14 shows the results of these
two methods in three cases.
From the figures, it can be found when noise gets stronger,
the classification accuracy of the two methods decreases si-
multaneously. Compared with the LPA based method, the pro-
posed method performs better in all three cases when noise gets
stronger, indicating that the robustness of the semi-supervised
GAN is stronger.

5. Conclusion

The paper proposed a semi-supervised multi-scale convolu-

tional generative adversarial network for bearing fault identifi-
cation. In this method, GAN is improved to solve the practical
problem in bearing fault detection that labeled data are expensive
and insufficient. The proposed method utilizes partially labeled
samples and sufficient unlabeled samples to training the fault
detection model. By adopting a one-dimensional MSCNN as the
discriminator, the proposed semi-supervised multi-scale convo-
lutional GAN can automatically extract features from signals for
Fig. 13. Fault identification results of two methods in Case 1 and Case 2.
fault identification. Based on that, a multi-scale convolutional
GAN is constructed to make full use of both labeled and unla-
Table 5
beled data for network training and fault identification. Totally
Performance of different GANs.
three cases are carried out to test the proposed GAN model.
Discriminator–generator Average classification accuracy (%)
Experiment result shows that the proposed semi-supervised fault
Case 1 Case 2 Case 3
detection method can identify the faults effectively even with
CNN–MSDCNN 93.75 92.68 86.43 limited labeled samples.
MSCNN–CNN 94.89 89.82 84.43
CNN–CNN 82.54 88.03 80.93
In further study, considering the characteristics of vibration
Proposed method 100 99.28 96.58 signals, more powerful deep neural networks will be designed as
the generator and the discriminator. What is more, we will study
how the GAN works to achieve a higher classification accuracy
with fewer labeled samples.
4.2. The discriminator and the generator
Declaration of competing interest
In this paper, the discriminator is a MSCNN while the gen-
erator is a MSDCNN. In order to investigate the superiority of The authors declare that they have no known competing finan-
these two networks, other commonly used deep neural networks cial interests or personal relationships that could have appeared
are employed to construct the discriminator and the generator. to influence the work reported in this paper.
First, a standard CNN model is used to replace the MSCNN in
the discriminator while the structure of the generator stays un- Acknowledgments
changed. Second, the generator is replaced by a standard CNN
model while the discriminator remains unchanged. Third, both The authors would like to sincerely thank all the anonymous
the generator and the discriminator are replaced by standard CNN reviewers for the valuable comments that greatly helped to im-
models which share the similar structure with the discriminator. prove the manuscript.
There are also 4 kernels in the first convolution layer and 16 This research is supported financially by the National Key Re-
kernels in the second convolution layer. The kernel sizes are all search and Development Program of China (No.
set to 4. These three models are trained with the same number 2019YFF0302204), National Natural Science Foundation of China
of labeled and unlabeled samples for comparison. After several — Civil Aviation Joint Research Fund of China Civil Aviation
repeated experiments, results are shown in Table 5. Administration (No. U1933101), National Natural Science Foun-
Compared with the CNN–CNN based method, the other three dation of China (No. 51875436), Major Research Plan of the
methods always perform better, which implies that both the National Natural Science Foundation of China (No. 91960106),
MSCNN and the MSDCNN can improve the fault diagnosis ability China Postdoctoral Science Foundation (No. 2018M631145) and
of the semi-supervised GAN models. When both the generator Shaanxi Natural Science Foundation (No. 2019JM-041).

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
10 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx

Fig. 14. Classification results of two methods with additional noise.

References [5] Cao H, Fan F, Zhou K, He Z. Wheel-bearing fault diagnosis of trains using
empirical wavelet transform. Measurement 2016;82:439–49.
[1] Acharya V, Sharma SK, Gupta SK. Analyzing the factors in indus- [6] Ren Z, et al. Crack fault diagnosis of rotor systems sing wavelet transforms.
trial automation using analytic hierarchy process. Comput Electr Eng Comput Electr Eng 2015;45:33–41.
2018;71:877–86.
[7] Li X, et al. Cross-domain fault diagnosis of rolling element bear-
[2] Xi W, Bai L, Hui M, Wu Q. A novel rolling bearing fault detect method ings using deep generative neural networks. IEEE Trans. Ind. Electron.
based on empirical wavelet transform. In: 2018 13th IEEE conference on 2019;66(7):5525–34.
industrial electronics and applications (ICIEA). 2019, p. 2764–8.
[3] Zhao L, Chu X, Huang D. Fault diagnosis for gearbox based on EMD and [8] Shao H, et al. A novel tracking deep wavelet auto-encoder method for
multifractal. In: Control and decision conference. 2014, p. 3792–6. intelligent fault diagnosis of electric locomotive bearings. Mech. Syst.
[4] Sapena-Bano A, et al. Fault diagnosis of rotating electrical machines in Signal Process. 2018;110:193–209.
transient regime using a single stator current’s FFT. IEEE Trans. Instrum. [9] Dong S, et al. Design and application of unsupervised convolutional
Meas. 2015;64(11):3137–46. neural networks integrated with deep belief networks for mechanical

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.
 T. Pan, J. Chen, J. Xie et al. / ISA Transactions xxx (xxxx) xxx 11

fault diagnosis. In: Prognostics and system health management conference [24] Yang G, et al. Dagan: Deep de-aliasing generative adversarial networks
(PHM-Harbin). Harbin, China, 2017, pp. 1–7. for fast compressed sensing mri reconstruction. IEEE Trans Med Imaging
[10] Wu C, et al. Intelligent fault diagnosis of rotating machinery based on one- 2018;37(6):1310–21.
dimensional convolutional neural network. Comput Ind 2019;118:53–61. [25] Radford A, Metz L, Chintala S. Unsupervised representation learning with
[11] Pan J, et al. Liftingnet: a novel deep learning network with layerwise deep convolutional generative adversarial networks. 2015, arXiv preprint
feature learning from noisy mechanical data for fault classification. IEEE arXiv:1511.06434.
Trans. Ind. Electron. 2018;65(6):4973–82. [26] Mao X, et al. Least squares generative adversarial networks. In: In-
[12] Li X, et al. Diagnosing rotating machines with weakly supervised data using ternational conference on computer vision. Venice, Italy, 2017, pp.
deep transfer learning. IEEE Trans. Ind. Inf. 2019. http://dx.doi.org/10.1109/ 2813–2821.
TII.2019.2927590, (in press). [27] Liu J, Qu F, Hong X, Zhang H. A small-sample wind turbine fault detection
[13] Yang B, Lei Y, Jia F, Xing S. An intelligent fault diagnosis approach based on method with synthetic fault data using generative adversarial nets. IEEE
transfer learning from laboratory bearings to locomotive bearings. Mech. Trans. Ind. Inf. 2019;15(7):3877–88.
Syst. Signal Process. 2019;122:692–706. [28] Wang Z, Wang J, Wang Y. An intelligent diagnosis scheme based on
[14] Dai J, Wang Z. Incremental fault diagnosis: Exploiting unlabelled data generative adversarial learning deep neural networks and its applica-
with semi-supervised ensemble learning. In: International conference on tion to planetary gearbox fault pattern recognition. Neurocomputing
electronic measurement & instruments. Yangzhou, China, 2017, pp. 7–14. 2018;310:213–22.
[15] Tao X, Song S, Liu F, Cao P. Bearings fault detection based on [29] Liu H, et al. Unsupervised fault diagnosis of rolling bearings using a deep
semi-supervised SVM Laplacian regularization. Spectrosc Spectr Anal neural network based on generative adversarial networks. Neurocomputing
2011;31(6):4270–4. 2018;315:412–24.
[16] Jiang L, Ge Z, Song Z. Semi-supervised fault classification based on [30] Berthelot D, Schumm T, Metz L. BEGAN: boundary equilibrium generative
dynamic sparse stacked auto-encoders mode. Chemometr Intell Lab Syst adversarial networks. 2017, arXiv preprint arXiv:1703.10717.
2017;168:72–83. [31] Fiore U, et al. Using generative adversarial networks for improving
[17] Luo J, Xu H, Su Z. Fault diagnosis method based semi-supervised manifold classification effectiveness in credit card fraud detection. Inform. Sci.
learning and transductive SVM. In: International conference on sensing, 2017;479:448–55.
diagnostics, prognostics, and control. Shanghai, China, 2017, pp. 710–717. [32] Salimans T, et al. Improved techniques for training gans. In: Advances in
[18] Razavi-Far R, et al. Information fusion and semi-supervised deep learning neural information processing systems. 2016, p. 2234–42.
scheme for diagnosing gear faults in induction machine systems. IEEE [33] Wen L, et al. A new data-driven intelligent fault diagnosis by using
Trans. Ind. Electron. 2018;66(8):6331–42. convolutional neural network. In: International conference on industrial
[19] Zhao M, Li B, Qi J, Ding Y. Semi-supervised classification for rolling fault di- engineering and engineering management. Singapore, Singapore, 2017, pp.
agnosis via robust sparse and low-rank model. In: International conference 813–817.
on industrial informatics. Emden, Germany, 2017, pp. 1062–1067. [34] Gan M, Wang C. Construction of hierarchical diagnosis network based on
[20] Han Y, Tang B, Deng L. An enhanced convolutional neural network with deep learning and its application in the fault pattern recognition of rolling
enlarged receptive fields for fault diagnosis of planetary gearboxes. Comput element bearings. Mech. Syst. Signal Process. 2016;72:92–104.
Ind 2019;107:50–8. [35] Case Western reserve university bearing data center website. Available:
[21] Jia F, et al. A neural network constructed by deep learning technique and http://csegroups.case.edu/bearingdatacenter/home.
its application to intelligent fault diagnosis of machines. Neurocomputing [36] Li X, Zhang W, Ding Q. A robust intelligent fault diagnosis method
2017;272:619–28. for rolling element bearings based on deep distance metric learning.
[22] Goodfellow I, et al. Generative adversarial nets. In: Advances in neural Neurocomputing 2018;310:77–95.
information processing systems. 2014, p. 2672–80. [37] Zhuang Z, Wei Q. Intelligent fault diagnosis of rolling bearing using one-
[23] Wang C, Xu C, Wang C, Tao D. Perceptual adversarial networks for image- dimensional multi-scale deep convolutional neural network based health
to-image transformation. IEEE Trans Image Process 2018;27(8):4066–79. state classification. In: International conference on networking sensing and
control. Zhuhai, China, 2018.

Please cite this article as: T. Pan, J. Chen, J. Xie et al., Intelligent fault identification for industrial automation system via multi-scale convolutional generative adversarial
network with partially labeled samples. ISA Transactions (2020), https://doi.org/10.1016/j.isatra.2020.01.014.