This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

A New Convolutional Neural Network Based

Data-Driven Fault Diagnosis Method
Long Wen, Xinyu Li, Liang Gao, Member, IEEE, and Yuyan Zhang

named as data-driven methods, require a large volume of

Abstract—Fault diagnosis is vital in manufacturing historic data to establish the fault modes about the systems
system, since early detections on the emerging problem without priori known models or signal patterns. They are very
can save invaluable time and cost. With the development of suitable for the complex systems which are difficult to establish
smart manufacturing, the data-driven fault diagnosis
becomes a hot topic. However, the traditional data-driven
the explicit models or signal symptoms. Recently the smart
fault diagnosis methods rely on the features extracted by manufacturing is boosting, and the data in enterprise can be
experts. The feature extraction process is an exhausted collected much faster [4] and more widely than ever before.
work and greatly impacts the final result. Deep learning (DL) This provides new opportunities for the data-driven fault
provides an effective way to extract the features of raw diagnosis methods to make full use of the massive mechanical
data automatically. Convolutional neural network (CNN) is data [5], and has received more and more attentions from the
an effective DL method. In this research, a new CNN based
on LeNet-5 is proposed for fault diagnosis. Through a
researchers and engineers.
conversion method converting signals to 2-D images, the Machine learning is one of the main methods to handle the
proposed method can extract the features of converted 2-D data in data-driven fault diagnosis. The initial attempts of
images and eliminate the effect of handcrafted features. applying support vector machines (SVMs) to fault diagnosis
The proposed method which is tested on three famous were in the late 1990s [6]. Fuzzy logic (FL) can partition the
datasets, including motor bearing dataset, self-priming feature space into fuzzy sets and utilize fuzzy rules for
centrifugal pump dataset, axial piston hydraulic pump
dataset, has achieved prediction accuracy of 99.79%,
reasoning. A novel fuzzy-neural data fusion engine was
99.481% and 100% respectively. The results have been proposed to the on-line monitoring and diagnosis [7]. Artificial
compared with other DL and traditional methods, including neural networks (ANN) is one of the most well-established
adaptive deep CNN, sparse filter, deep belief network, and data-driven fault diagnosis methods. An extension neural
support vector machine. The comparisons show that the network was applied on the fault diagnosis in internal
proposed CNN based data-driven fault diagnosis method combustion engines [8]. A feedforward neural network was
has achieved significant improvements.
applied on the laser welding process monitoring and defects
Index Terms—Fault Diagnosis, Convolutional Neural diagnosis [9]. However, machine learning methods are unable
Network, Image Classification, Data-Driven. to generate discriminative features of raw data and are always
combined with the signal features extraction process. This
features extraction process is an exhausted work and greatly
I. INTRODUCTION impacts the final result.
With the rapid development of machine learning, deep
F AULT diagnosis has attracted many studies in the recent
years [1][2]. Early detections on the emerging problem are
vital for the complex systems, and it can save invaluable time
learning (DL) has emerged as an effective way to overcome the
above drawback. DL can learn the abstract representation
and cost to take remedy measure to avoid dangerous situations. features [10] of the raw data automatically [11], which could
In general, the fault diagnosis methods can be classified into avoid the handcrafted features designed by engineers. Several
model-based, signal-based, knowledge-based and hybrid/active DL methods had already been applied on the fault diagnosis,
methods [3]. The knowledge-based methods, which are also such as deep belief network [12], sparse auto-encoder [13],
stacked denoising auto-encoder [14] [15] and sparse filtering
[16]. DL has achieved the good results compared with the
Manuscript received June 22, 2017; revised September 26, 2017;
accepted October 28, 2017. This work was supported in part by the traditional shadow machine learning methods, but the
Natural Science Foundation of China (NSFC) under Grants 51435009, application of DL on fault diagnosis is still developing.
51775216 and 51711530038, and in part by the 111 Project under Grant As one of the most effective DLs, convolutional neural
B16019.
L. Wen, X.Y. Li, L. Gao, and Y.Y. Zhang are with the State Key
network (CNN) has also been applied on the fault diagnosis.
Laboratory of Digital Manufacturing Equipment & Technology, School of Since the most common data type is the time-domain signals,
Mechanical Science and Engineering, Huazhong University of Science the 1-D CNN has been investigated on the real-time motor fault
and Technology, Wuhan, 430074, China (e-mail: [email protected],
[email protected], [email protected],
diagnosis [17]. In some scenarios, the machinery data can be
[email protected]). (corresponding author to provide phone: presented in 2-D format (such as time-frequency spectrum),
+86-027-87559419; fax: +86-027-87559419; e-mail: and then an image processing method is combined to classify
[email protected]).

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

these images [18]. However, these presentations rely on the The key aspect of DL is that these features are not designed by
expert’s knowledge as well. In this research, a new data human engineers, which can reduce the impacts of different
pre-processing method is introduced to convert the raw experiences coming from different engineers. Jia et al [25]
time-domain signal data to 2-D gray images without any investigated a deep neural network for intelligent fault
predefined parameters, which can eliminate the experts’ diagnosis based on auto-encoder. Liao and Pavel [26] studied
experiences as more as possible. Then a new improved CNN is the enhanced restricted Boltzmann machine for prognostics and
proposed to extract the features of these 2-D images. The health assessment. Gan et al [27] applied a deep belief network
results show that it is very promoting in fault diagnosis. based hierarchical diagnosis network on the fault diagnosis of
The main contributions of this paper are summarized as the rolling element bearings. Cho et al [28] investigated recurrent
neural networks and dynamic Bayesian modeling on the
following three points. Firstly, a new data pre-processing
induction motors fault detection.
method named as a signal-to-image conversion method is
As one of the most effective DL method, CNN has also been
developed to extract the 2-D features of raw data without
applied on the fault diagnosis. Ince et al [17] applied 1-D CNN
predefined parameters. Secondly, an improved CNN is to real-time motor fault diagnosis. Abdeljaber et al [29] studied
proposed by using a deeper structure and the zero-padding 1-D CNN to real-time damage detection. Guo et al [30]
method to increase the nonlinearity of the features. Thirdly, as a investigated a hierarchical adaptive deep CNN to bearing fault
result, the proposed CNN based fault diagnosis method diagnosis.
achieves a significant improvement by comparing with other In some scenarios, the machinery data can be presented in
DL and traditional methods. 2-D format. The conversion from 1-D raw signals to 2-D
The rest of this paper is organized as follows. Section II images is various. Chong [31] proposed an approach to
discusses the related works. Section III presents the effectively extract features of the faults in an induction motor.
methodologies, including the signal-to-image conversion He converted 1-D vibration signals into 2-D gray-level images.
method and the proposed CNN method. Section IV presents the Kang and Kim [32] proposed a 2-D grey images representation
testing results of the proposed method on three datasets. The based on Shannon wavelets and applied multi-class support
conclusion and future researches are presented in the Section V. vector machines to identifying faults in the induction machine.
Li et al [33] investigated a feature-extraction model based on
II. RELATED W ORKS 2D non-negative matrix factorization and generalized S
transform proposed. Lu et al [18] proposed a conversion from
The related works described in this section, contain the signals to images using bi-spectrum, and then used a
data-driven fault diagnosis method and LeNet-5 CNN. probabilistic neural network to classify the images. However,
A. Data-Driven fault diagnosis methods most of these conversions rely on expert’s knowledge as well.
In this research, a new signal-to-image conversion method is
Since the data-driven fault diagnosis methods can discover
presented without any predefined parameters, which can
the underlying knowledge to represent the information among
eliminate the experts’ experiences as more as possible.
the system variables only using historic data, it is very suitable
for the complex systems where they are difficult to establish the B. LeNet-5 CNN
explicit models or signal symptoms. The first data-driven fault CNN is a special structure of artificial neural network (ANN).
diagnosis was published in 1980s using expert system [19]. Different from the fully connected ANN, each neuron of the
With the rapid development of machine learning and smart feature map in each layer is only sparsely connected to a small
manufacturing, the data-driven fault diagnosis method has set of neurons in previous layer in CNN. This was inspired by
become a hot research topic recently. the concept of simple and complex cells in visual cortex in
The intelligent learning from the large volume of historic brain [34], and the visual cortex contains some cells that are
data is essential for the data-driven fault diagnosis method. The only sensitive to the local receptive field [35].
statistical analysis methods, including principal component There are three major layers in CNN: 1) convolutional layer;
analysis (PCA), partial least squares (PLS), and independent 2) pooling layer; 3) fully-connected layer. The convolutional
Component Correlation (ICA), have received considerably layer applies a set number of filters to obtain the feature maps
increasing attentions for industrial process monitoring [20]. of input images. The pooling layer is the down-sampling layer
Yin et al [21] investigated the data-driven process monitoring to reduce the feature dimensions of the input. After several
based on a modified partial least squares. Yin et al [22] studied alternating convolutional and pooling layers, the
a data-driven real-time implementation of fault-tolerant control fully-connected layers are followed to compute the class scores.
system. Machine learning is also one of the dominant tools for CNN has achieved widely successful applications in image
data-driven fault diagnosis, such as SVM[6], FL[7], ANN[8] et recognition tasks. And there are some famous CNN models,
al. Furtherly, The statistical and machine learning methods are such as LeNet-5 [36], AlexNet [37], VGGNet [38], GoogLeNet
often utilized jointly. Grbovic et al [23] investigated cold start [39] and so on.
fault detection framework, in which only normal operating data LeNet-5 which is a classical release of CNN, has been
was available at the beginning and the fault can be observed by applied to handwritten and machine-printed character
PCA with squared prediction error (SPE) statistics. recognition with minimal data preprocessing. There are two
Compared with traditional artificial intelligent techniques, alternating convolutional and pooling layers in this model, with
deep learning holds the potential of feature-representation and a two-layer fully connected ANN. Hong et al [40] employed a
has been applied in the machine health monitoring field [24]. LeNet-5 to face recognition. Ren et al [41] used principal

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

component analysis based CNN to the image classification task. segment signal. P( j= =
, k ) , j 1... M , k 1...M denotes the pixel
Guo et al [30] investigated the LeNet-5 based adaptive deep strength of the image, as shown in equation(1).
CNN on the bearing fault diagnosis.
 L ( ( j − 1) × M + k ) − Min( L) 
In this research, the CNN models based= on LeNet-5 are P( j , k ) round  × 255 (1)
designed to solve the image classification tasks of the fault  Max( L) − Min( L) 
diagnosis. The function round (⋅) is the rounding function and the
whole pixel value has been normalized from 0 to 255, which is
III. PROPOSED CNN FOR FAULT DIAGNOSIS
just the pixel strength of the gray image. The 2×2 filters are
This section presents the proposed CNN based fault commonly used in this paper, and the size of image features on
diagnosis method. Firstly, the signal-to-image conversion each layer would be reduced by half, so the recommendation of
method is presented to handle the raw signal. Then, the n
modified CNN is presented. Moreover, the zero padding M is 2 , such as 16, 32, 64, 128 et al. In this paper, the
method is also introduced. selecting of 64×64 and 16×16 is depended on the volume of
signal data.
A. Signal-to-Image Conversion Method The advantage of this data processing method is that it
In traditional data-driven fault diagnosis methods, the data provides a way to explore 2-D features of the raw signals [31].
pre-processing method is vital since most data-driven methods What’s more, this data pre-processing method can be calculated
cannot handle the raw signals directly. One of the main without any predefined parameters and can reduce the experts’
functions of the data pre-processing method is to extract the experiences as more as possible.
features of the raw signals from the large volume of historic
B. Proposed CNN Structure
data. However, to extract the proper features is an exhausted
work, and these features have great effects on the final results. Once the raw signals have been converted to images, a CNN
In this research, an effective data pre-processing method is can be trained to classify these images. LeNet-5 which is a
developed. The idea of this method is to convert the classical CNN is effective and promoting in the image pattern
recognition. In this research, the CNN models based on
LeNet-5 are designed to solve the image classification tasks of
the fault diagnosis.
The sizes of images in LeNet-5 are 32×32. But in order to
improve the classification results, the sizes of images change
according to the volume of signals data in this research. In the
first and second cases, the signals data has the large volumes,
and larger images size can promote the classification result. So
the sizes of images in these cases are 64×64. Oppositely, since
the volume of signals data is relatively small in the third case,
the sizes of images are 16×16.
In the proposed CNN models, they contain four alternating
convolutional and pooling layers with one or two
fully-connected layers for 64×64 images. While the proposed
CNN models only contain two alternating convolutional and
Fig. 1. Signal-to-Image Conversion method pooling layers with one fully-connected layers for 16 × 16
time-domain raw signals to images. images. The padding method is also different from original
As shown in Fig. 1, in this conversion method, the LeNet-5, and the zero-padding is used in our study. Fig. 2
time-domain raw signals fulfill the pixels of the image by shows the basic structure of the proposed CNN models for 64
sequence. In order to obtain a M × M size image, a segment ×64 images.
signal with the length M 2 would be randomly obtained from
C. Zero-padding Method
the raw signal. Let L(i ) , i = 1...M denote the value of the
2
Padding method is an important technique to control the size

Fig. 2. The proposed CNN structure for 64×64 images

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

Fig. 4. Signal-to-Image conversion image on normal condition

Fig. 3. The zero-padding method used in CNN

of feature dimension. The zero-padding method is applied on
the CNN models to prevent the dimension loss, which is shown
in Fig. 3. Let M be the input size, N be the output size, F be
the filter width, S be the stride. The number of padding on left
PL and on right PR can be calculated by equations (2-5).
The ceil (⋅) and floor (⋅) are the ceil and floor functions.
M 
N = ceil   (2)
 S 
PT = ( N − 1) × S + F − M (3)
 PT 
PL = floor   (4)
 2 
= PT − PL
PR (5)
In zero padding method, zeros would be added automatically
to fill the convolution process. Fig. 3 shows an example of
zero-padding method in one dimension. The parameters are Fig. 5. The converted images on nine fault conditions
=M 5,= S 1,=F 3 , then the padding result will be 1) Image Conversion Result
=
PL 1, PR = 1,= N 5 . It should be noted that the strides of the The size of the converted images is set as 64×64. The signal
convolutional and pooling layers are set to 1 in this research. conversion result on normal condition is presented on Fig. 4.
The converted gray image contains 4096 pixels. The
IV. CASE STUDIES AND EXPERIMENTAL RESULTS conversion results on other nine fault conditions are presented
In this section, the proposed CNN based fault diagnosis in Fig. 5. From the converted images, it can be observed that
method is conducted on three famous fault diagnosis datasets, the images of different fault conditions look totally different
which are the motor bearing fault dataset, the self-priming from each other, and this provides an intuitionistic way to
centrifugal pump dataset, and the axial piston hydraulic pump classify them.
dataset. The CNN models are written in Python 3.5 with 2) CNN Structure Testing Result
TensorFlow and run on Ubuntu 16.04 with a GTX 1080 GPU. The CNN structure on this case study contains four
alternating convolutional and pooling layers with one or two
A. Case 1: Motor Bearing Fault Diagnosis
In this subsection, the proposed CNN based method is TABLE I
conducted on the famous motor bearing data provided by the The Layer Configurations of CNN models
Layer name CNN Models
Case Western Reserve University (CWRU) [42]. In this dataset,
L1 Conv(5×5×32)
there are three fault types, and each fault type has three
L2 Maxpool(2×2)
different damage sizes. There are totally ten health conditions L3 Conv(3×3×64)
with nine fault conditions and one normal condition (NO). L4 Maxpool(2×2)
Three fault types are roller fault (RF), outer race fault (OF) and L5 Conv(3×3×128)
inner race fault (IF). The damage sizes are the 0.18mm, L6 Maxpool(2×2)
0.36mm and 0.54mm. L7 Conv(3×3×256)
The driven end vibration signals are collected under four load L8 Maxpool(2×2)
conditions (0, 1, 2, 3 hp) to verify the performance of the
proposed method. There are 2000 samples per load condition in
TABLE II
the training dataset and 400 samples per load condition in the
The Result of CNN Models with One FC Layer in Case 1 (%)
testing dataset. All the samples are randomly selected from the
CNN- CNN- CNN- CNN- CNN- CNN-
dataset. It should be noticed that the sampling without No.
1024 1536 2048 2560 3072 3584
replacement is used, so the samples in the training dataset and Max 99.78 99.83 99.87 99.85 99.74 99.86
testing dataset are totally different. Min 99.11 99.59 99.57 99.67 99.41 98.811
Mean 99.57 99.71 99.75 99.77 99.65 99.63
Std 0.2063 0.0658 0.0779 0.0544 0.0979 0.3548

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

TABLE III
The Result of CNN Models with Two FC Layers in Case 1 (%)
CNN- CNN- CNN- CNN- CNN- CNN- CNN-
No.
2560 2560-64 2560-128 2560-256 2560-512 2560-768 2560-1024
Max 99.85 99.87 99.89 99.88 99.84 99.91 99.88
Min 99.67 99.66 99.55 99.70 99.69 99.70 99.44
Mean 99.77 99.75 99.76 99.78 99.78 99.79 99.77
Std 0.0544 0.0725 0.0984 0.0526 0.0476 0.0759 0.1275

fully connected (FC) layers. The parameters on each layer are [12], and adaptive deep convolution neural network (ADCNN)
presented in TABLE I. FC1 is the first FC layers, and FC2 is the [30]. The mean prediction accuracy is the final measure term
second FC layers. The denotation of Conv(5×5×32) means for this comparison. The comparison results are shown in
that it is a convolution layer, and the filter size is 5×5 with 32 TABLE IV, and CNN stands for the CNN-2560-768.
channels. Maxpool(2×2) means that it is a maxpool layer with From the results, it can be seen that the proposed CNN
method obtains a wonderful result compared with these
2×2 filters. The results of the proposed CNN models with one
methods. The mean prediction accuracy is as high as 99.79%,
FC layer are presented in TABLE II, and those of the proposed
and it is better than all other methods. The prediction results of
CNN models with two FC layers are presented in TABLE III.
sparse filter, DBN, DBN Based HDN, SVM are 99.66%,
CNN-i-j denotes that there are i neurons in FC1 and j neurons
87.45%, 99.03% and 87.45%, which shows the significant
in FC2, such as CNN-2560 means there are 2560 neurons in
performance of the proposed CNN method. The result of
FC1, and the FC2 layer does not exist. All CNN models run 10
traditional ANN is 67.7%, which is obviously inferior to the
times, and the maximum, minimum, mean, and the standard
proposed CNN models, showing the great improvement of the
deviation of the prediction accuracy are the results measure
proposed CNN method.
terms.
Fig. 6 presents the confusion matrix of the best result of
There are six CNN models with one FC layer. From the
CNN-2560-768, which obtains the prediction accuracy of 99.91%
results, CNN-2560 achieves the best result. Its mean accuracy
(the maximum value of CNN-2560-768 in TABLE III). The
is 99.77%, the minimum accuracy is 99.67%, and the standard
rows stand for the actual label, and the columns stand for the
deviation is 0.0544. The best maximum accuracy is CNN-2048,
predicted label for each condition. It shows that OF0.18,
which is 99.87% and slightly superior than CNN-2560, whose
OF0.36, IF0.18, IF0.54 and NO have 100% accuracy. RF0.54 is
maximum is 99.85%. The prediction results of these CNN like
the worst one which has the accuracy of 99.63%. RF0.36
an “inverted U” type, showing that CNN-2560 is the peak of
receives the most misclassification. 0.19% out of RF0.54, 0.06%
this proposed CNN models.
out of OF0.54 and IF0.36 are misclassified to RF0.36. OF0.18
The CNN models with two FC layers are based on
and NO are 100% accuracy, what’s more, no other condition is
CNN-2560. There are six CNN models. From the results, it can
misclassified to them, meaning they are purely separated from
be seen that all these CNN models are very close to each other.
other conditions.
CNN-2560-64, CNN-2560-128 and CNN-2560-1024 are
slightly inferior to CNN-2560. CNN-2560-256, CNN2560-523
and CNN-2560-768 improves the prediction accuracy slightly.
The best among these CNN models is CNN-2560-768, and its
mean accuracy is 99.79%, the maximum accuracy is 99.91%,
and the minimum accuracy is 99.70%.
3) Compared with other methods
In order to evaluate the performances of the proposed CNN
models, other statistical methods and deep learning methods are
selected to compare the prediction accuracy in this case. They
are sparse filter [16], deep belief network(DBN) [12], DBN
based HDN (hierarchical diagnosis network) [27], support
vector machine (SVM) [43], artificial neural network (ANN)
TABLE IV
Comparison Result of CNN Models and Other Methods (%)
Methods Mean Accuracy
CNN 99.79
Sparse filter 99.66 Fig. 6. The confusion matrix of the result on CNN-2056-768

ADCNN 98.1%
B. Case 2: Self-priming Centrifugal Pump Fault
DBN 87.45 diagnosis
DBN Based HDN 99.03 In this subsection, the proposed method is conducted on
SVM 87.45 self-priming centrifugal pump [18]. The acceleration sensor is
installed on a specific pedestal above the motor housing. In the
ANN 67.70
data acquisition system, the rotation speed is 2,900 per minute.

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

The sampling frequency of vibration signals is 10,240Hz. CNN-1536 achieves the best mean accuracy which is 99.445%.
There are four faults conditions and one normal condition (NO). The result of CNN-2048 is slightly inferior to CNN-1536, and
The fault conditions are bearing roller wearing (BR), inner race the mean accuracy is 99.411%.
wearing (IR), outer race wearing (OR), and impeller wearing TABLE VI presents the result of CNN model with two FC
(IW) fault condition. layers. These five CNN models are based on CNN-1536.
1) Image Conversion Result Among these CNN models, CNN-1536-64 and CNN-1536-128
In this case, the sampling time is two seconds for each dataset, are better than CNN-1536 and the accuracy of CNN-1536-128
and each condition is collected five times. The size of the is 99.481% with the standard deviation of 0.1966.
converted images is 64×64, and they are presented in Fig. 7. 3) Compared with other methods
The samples in the training dataset and the testing dataset are The results of the proposed CNN models are compared with
2,000 and 400 per conditions. There are totally 50,000 images the method in literature [18], and they are presented on TABLE
in the training dataset and 10,000 images in the testing dataset. VII. The SURF based PNN employs bi-spectrum technique to
transform the vibration signals to images, and uses SURF based
PNN to classify the fault diagnosis. From the results, it can be
clearly seen that the proposed CNN method outperforms SURF
based PNN in terms of the mean accuracy and the standard
deviation of the accuracy, showing the potential of the
proposed CNN based fault diagnosis method.
The confusion matrix of the best result of CNN-1536-128 is
presented on Fig. 8 (The maximum value of CNN-1536-128 in
TABLE VI). From the result, the normal condition has the
worst accuracy of 99.45%, and IW wearing and OR wearing
have the accuracy of 99.90%. The most misclassification is
0.4%, where the actual label is normal condition but the
prediction label is BR wearing.

Fig. 7. The converted images in case 2

2) CNN Structure Testing Result

The basic structure of CNN in this case study is the same with
case 1 in section A, except that they have different neurons on
FC layers. There are six CNN models with one FC layer and
five CNN models with two FC layers. In the comparison, all
CNN models run 10 times. From TABLE V, it can be seen that

TABLE V
The Result of CNN Models with One FC Layer in Case 2 (%)
CNN- CNN- CNN- CNN- CNN- CNN-
No.
512 1024 1536 2048 2560 3072
Max 99.40 99.48 99.67 99.68 99.65 99.55 Fig. 8. The confusion matrix of the result on C1536-128
Min 97.60 99.18 99.12 98.55 99.03 99.21
Mean 98.915 99.322 99.445 99.411 99.397 99.379 C. Case Study 3: Axial Piston Hydraulic Pump Fault
Std 0.5266 0.1140 0.1850 0.3380 0.2134 0.1034
Diagnosis
In this subsection, the proposed method is tested on the axial
TABLE VI
piston hydraulic pump [18]. In the experiment, the rotation
The Result of CNN Models with Two FC Layers in Case 2 (%)
CNN- CNN- CNN- CNN- CNN- CNN-
speed is 5280 rpm/min, and the corresponding spindle
No. frequency is 88 Hz. The accelerograph is installed at the end
1536 1536-32 1536-64 1536-128 1536-256 1536-512
Max 99.67 99.74 99.69 99.74 99.63 99.56 face of the pump, and the sampling frequency is 1K Hz. There
Min 99.12 98.95 99.24 99.11 99.20 99.25 are two fault conditions in this case. They are the piston shoes
Mean 99.445 99.35 99.472 99.481 99.397 99.401 and swashplate wearing (PS Wearing) and valve plate wearing
Std 0.1850 0.2621 0.1677 0.1966 0.1528 0.1106 (VP Wearing). The normal condition is also collected as
contrast.
TABLE VII 1) Image Conversion Result
Comparison Result of CNN and SURF based PNN in In this case, each running condition contains only 1024 points,
Case 2 (%) which is relatively small compared with the above two datasets.
CNN- SURF based PNN
So the size of images is 16×16, and the converted images are
1536-128 Test 1 Test 2 Test 3 Test 4
Max 99.74 99.11 100 96.00 98.22 presented in Fig. 9. The samples in the training dataset and the
Mean 99.481 98.33 testing dataset are 400 and 100 per condition. The structure of
Std 0.1966 1.7164 CNN contains two convolutional layers and max-pool layers.

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

The channels of these two convolutional layers is 16 and 64. 3) Compared with other methods
The filter size of convolution is 3×3, and that of maxpool is 2 The prediction accuracy on testing dataset of CNN-8 is 100%
×2. in TABLE VIII. This result outperforms all other CNN models.
The mean prediction accuracy decreases obviously with the
increase of the number of neurons in FC layer. What’s more,
the prediction accuracy (a) and its standard deviation (std)
show a strong linear relationship. The multiple correlation
coefficient R2 is 0.7686, and the linear regressive function is:
std=0.4198-0.4195a. The fitted curve is plotted in Fig. 10.
The results are also compared with the literature [18], and the
result is presented on TABLE IX. From the result, it can be
Fig. 9. The converted images in case 3 clearly seen that the proposed CNN method outperforms SURF
based PNN. Even though PNN has achieved a good result on
2) CNN Structure Testing Result
the prediction accuracy of 98.705%. The proposed CNN based
The results of the proposed methods are presented in TABLE
model achieves a stable prediction of 100%. This result
VIII. Seven CNN models are tested and each CNN model runs
validates the performance of the proposed CNN method.
10 times. CNN-0 means that there doesn’t exist the
fully-connected layer. D. Discussions
In the case studies, the potential of the proposed CNN based
TABLE VIII
The Result of Different CNN Models in Case 3 (%) fault diagnosis method is validated on three datasets (the
Run CNN- CNN- CNN- CNN- CNN- CNN- CNN- famous motor bearing dataset, self-priming centrifugal pump
No. 0 8 16 32 64 128 256 dataset and axial piston hydraulic pump dataset). According to
1 99.50 100 100 100 100 99.83 99.92 the volumes of the signals, the sizes of converted images are 64
2 99.50 100 99.92 100 99.83 99.91 99.92 ×64, 64×64 and 16×16. The prediction accuracies of the
3 99.50 100 100 100 100 100 99.58 proposed CNN are 99.79%, 99.481% and 100% on these three
4 99.67 100 100 100 99.75 99.91 99.92
5 99.75 100 100 99.83 99.91 100
datasets respectively. Since the proposed method can extract
100
6 99.50 100 100 100 100 100 99.92 the 2-D features of the converted images automatically, these
7 99.75 100 100 99.92 100 100 99.75 results outperform other DL and traditional methods, such as
8 99.83 100 100 100 100 99.75 99.75 ADCNN, sparse filter, DBN, SVM, ANN and SURF based
9 99.67 100 100 100 100 100 100 PNN, which shows the potential of the proposed method.
10 99.92 100 100 100 99.92 99.92 99.83
Mea
n
99.66 100 99.99 99.99 99.93 99.93 99.86 V. CONCLUSION AND FUTURE RESEARCHES
Std 0.1544 0 0.0263 0.0263 0.0946 0.0829 0.1306 This research presents a new CNN based fault diagnosis
method. The main contributions of this research are developing
a signal-to-image conversion method, proposing a new CNN
based on LeNet-5, and applying the CNN models to the fault
diagnosis filed. The proposed CNN methods are tested on three
datasets, including motor bearing dataset, self-priming
centrifugal pump fault diagnosis dataset, and axial piston
hydraulic pump fault diagnosis dataset, and they achieve the
prediction accuracies of 99.79%, 99.481% and 100%
respectively, which outperform other DL and traditional
methods. These results show the good potential of proposed
CNN methods in the data-driven fault diagnosis field.
The limitations of proposed method include the following
aspects for real applications. Firstly, the most common faults
conditions are needed to be detected and can be represented in
the dictionary list type. Otherwise, the faults which have not
Fig. 10. The fitted curve of prediction accuracies and its
been learned would be misclassified to be the known ones.
standard deviation Secondly, the training process is very time-consumption, and a
GPU hardware is essential. Based on these limitations, the
TABLE IX future researches can be conducted in the following ways.
Comparison Result of CNN and SURF based PNN in Firstly, this method can be modified to find an unknown fault
Case 3 (%) conditions. Secondly, the CNN based transfer learning can be
CNN- SURF based PNN
studied further to reduce the training time.
8 1 2 3 4
Max 100 97.04 100 100 97.78
Mean 100 98.705
Std 0 1.5255

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

REFERENCES [23] M. Grbovic, W. Li, N. A. Subrahmanya, A. K. Usadi, and S. Vucetic.

“Cold start approach for data-driven fault detection.” IEEE Transactions
[1] X. Dai and Z. Gao, “From model, signal to knowledge: A data-driven on Industrial Informatics, vol. 9, no. 4, 2264-2273, 2013.
perspective of fault detection and diagnosis,” IEEE Transactions on [24] R. Zhao, R. Yan, Z. Chen, K. Ma3o, Peng Wang and R. X. Gao, “Deep
Industrial Informatics, vol. 9, no. 4, pp. 2226-2238, Nov 2013. learning and its applications to machine health monitoring: A survey.”
[2] Z. Gao, C. Cecati and S. X. Ding, “A survey of fault diagnosis and arXiv preprint arXiv:1612.07640, Dec, 2016.
fault-tolerant techniques—Part II: Fault diagnosis with knowledge-based [25] F. Jia, Y. Lei, J. Lin, X. Zhou and N. Lu, “Deep neural networks: A
and hybrid/active approaches,” IEEE Transactions on Industrial promising tool for fault characteristic mining and intelligent diagnosis of
Electronics, vol. 62, no. 6, pp. 3768-3774, Jun, 2015. rotating machinery with massive data,” Mechanical Systems and Signal
[3] Z. Gao, C. Cecati and S. X. Ding, “A survey of fault diagnosis and Processing, vol. 72, pp. 303-315, May, 2016.
fault-tolerant techniques—Part I: Fault diagnosis with model-based and [26] L. Liao, W. Jin and R. Pavel, “Enhanced restricted boltzmann machine
signal-based approaches,” IEEE Transactions on Industrial Electronics, with prognosability regularization for prognostics and health assessment,”
vol. 62, no. 6, pp. 3757-3767, Jun, 2015. IEEE Transactions on Industrial Electronics, vol. 63, no. 11, pp.
[4] D. Wang and W. T. Peter, “Prognostics of slurry pumps based on a 7076-7083, Nov, 2016.
moving-wear degradation index and a general sequential monte carlo [27] M. Gan and C. Wang, “Construction of hierarchical diagnosis network
method,” Mechanical Systems and Signal Processing, vol. 56, pp. based on deep learning and its application in the fault pattern recognition
213-229, May, 2015. of rolling element bearings,” Mechanical Systems and Signal Processing,
[5] A. Gandomi and M. Haider, “Beyond the hype: Big data concepts, vol. 72, pp. 92-104, May, 2016.
methods, and analytics,” International Journal of Information [28] H. C. Cho, J. Knowles, M. S. Fadali and K. S. Lee, “Fault detection and
Management, vol. 35, no. 2, pp. 137-144, Apr, 2015. isolation of induction motors using recurrent neural networks and
[6] T. AYDMJ and R. P. W. Duin, “Pump failure determination using support dynamic Bayesian modeling,” IEEE Transactions on Control Systems
vector data description,” Lecture Notes in Computer Science, pp. 415-425, Technology, vol. 18, no. 2, pp. 430-437, Mar, 2010.
1999. [29] O. Abdeljaber, O. Avci, S. Kiranyaz, M. Gabbouj and D. J. Inman,
[7] D. Wijayasekara, O. Linda, M. Manic and C. Rieger, “FN-DFE: “Real-time vibration-based structural damage detection using
Fuzzy-neural data fusion engine for enhanced resilient state-awareness of one-dimensional convolutional neural networks,” Journal of Sound and
hybrid energy systems,” IEEE Transactions on Cybernetics, vol. 44, no. Vibration, vol. 388, pp. 154-170, Feb, 2017.
11, pp. 2065-2075, Nov, 2014. [30] X. Guo, L. Chen and C. Shen, “Hierarchical adaptive deep convolution
[8] Y. Shatnawi and M. Al-Khassaweneh, “Fault diagnosis in internal neural network and its application to bearing fault diagnosis,”
combustion engines using extension neural network,” IEEE Transactions Measurement, vol. 93, pp. 490-502, Nov, 2016.
on Industrial Electronics, vol. 61, no. 3, pp. 1434-1443, Mar, 2014. [31] U. P. Chong, “Signal model-based fault detection and diagnosis for
[9] D. You, X. Gao and S. Katayama, “WPD-PCA-based laser welding induction motors using features of vibration signal in two-dimension
process monitoring and defects diagnosis by using FNN and SVM,” IEEE domain,” Strojniski vestnik-Journal of Mechanical Engineering, vol. 57,
Transactions on Industrial Electronics, vol. 62, no. 1, pp. 628-636, Jan, no. 9, pp. 655-666, Sep, 2011.
2015. [32] M. Kang and J. M. Kim, “Reliable fault diagnosis of multiple induction
[10] J. Schmidhuber, “Deep learning in neural networks: An overview,” motor defects using a 2-d representation of Shannon wavelets,” IEEE
Neural Networks, vol. 61, pp. 85-117, Jan 2015. Transactions on Magnetics, vol. 50, no. 10, pp. 1-13, Oct, 2014.
[11] Y. LeCun, Y. Bengio and G. Hinton, “Deep learning,” Nature, vol. 521, [33] B. Li, P. L. Zhang, D. S. Liu, S. S. Mi, G. Q. Ren and H Tian, “Feature
no. 7553, pp. 436-444, May 2015. extraction for rolling element bearing fault diagnosis utilizing generalized
[12] H. Shao, H. Jiang, X. Zhang and M. Niu, “Rolling bearing fault diagnosis S transform and two-dimensional non-negative matrix factorization,”
using an optimization deep belief network,” Measurement Science and Journal of Sound and Vibration, vol. 330, no. 10, pp. 2388-2399, May,
Technology, vol. 26, no. 11, pp. 115002, Sep, 2015. 2011.
[13] L. Wen, L. Gao and X. Y. Li, “A New Deep Transfer Learning Based on [34] B. A. Olshausen and D. J. Field, “Emergence of simple-cell receptive
Sparse Auto-Encoder for Fault Diagnosis,” IEEE Transactions on field properties by learning a sparse code for natural images,” Nature, vol.
Systems, Man, and Cybernetics: Systems, vol. pp, no. 99, pp. 1-9. 381, no. 6583, pp. 607, Jun, 1996.
[14] C. Lu, Z. Wang, W. Qin and J. Ma, “Fault diagnosis of rotary machinery [35] Y. Bengio, A. Courville and P. Vincent, “Representation learning: A
components using a stacked denoising autoencoder-based health state review and new perspectives,” IEEE transactions on pattern analysis and
identification,” Signal Processing, vol. 130, pp. 377-388, Jan, 2017. machine intelligence, vol. 35, no. 8, pp. 1798-1828, Aug, 2013.
[15] H. Shao, H. Jiang, F. Wang and H. Zhao, “An enhancement deep feature [36] Y. LeCun, “LeNet-5, convolutional neural networks,” URL: http://yann.
fusion method for rotating machinery fault diagnosis,” Knowledge-Based lecun. com/exdb/lenet. 2015.
Systems, vol. 119, pp. 200-220, Mar, 2017. [37] A. Krizhevsky, I. Sutskever and G. E. Hinton, “Imagenet classification
[16] Y. Lei, F. Jia, J. Lin, S. Xing and S. X. Ding, “An intelligent fault with deep convolutional neural networks,” In Advances in neural
diagnosis method using unsupervised feature learning towards information processing systems, pp. 1097-1105. 2012.
mechanical big data,” IEEE Transactions on Industrial Electronics, vol. [38] K. Simonyan and A. Zisserman, “Very deep convolutional networks for
63, no. 5, pp. 3137-3147, May, 2016. large-scale image recognition,” arXiv preprint arXiv:1409.1556, Sep,
[17] T. Ince, S. Kiranyaz, L. Eren, M. Askar and M. Gabbouj, “Real-time 2014.
motor fault detection by 1-d convolutional neural networks,” IEEE [39] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan,
Transactions on Industrial Electronics, vol. 63, no. 11, pp. 7067-7075, V. Vanhoucke and A Rabinovich, “Going deeper with convolutions,” In
Nov, 2016. Proceedings of the IEEE Conference on Computer Vision and Pattern
[18] C. Lu, Y. Wang, M. Ragulskis and Y. Cheng, “Fault diagnosis for rotating Recognition, pp. 1-9, 2015.
machinery: A method based on image processing,” PloS one, vol. 11, no. [40] S. Hong, Q. Wu, H. Xie, Y. Chen and Y. Kou, “A novel coupled template
10, pp. e0164111, Oct, 2016. for face recognition based on a convolutional neutral network,” In
[19] E. J. Henley, “Application of expert systems to fault diagnosis,” In AIChE Intelligent Systems Design and Engineering Applications (ISDEA), 2015
annual meeting, San Francisco, CA. 1984. Sixth International Conference on, IEEE, pp. 52-56, Aug, 2015.
[20] S. Yin, S. X. Ding, X. C. Xie and H Luo. “A review on basic data-driven [41] X. D. Ren, H. N. Guo, G. C. He, X. Xu, C. Di and S. H. Li,
approaches for industrial process monitoring.” IEEE Transactions on “Convolutional neural network based on principal component analysis
Industrial Electronics, vol. 61, no. 11, pp. 6418-6428, 2014. initialization for image classification,” In Data Science in Cyberspace
[21] S. Yin, H. Luo and S. X. Ding, “Real-time implementation of (DSC), IEEE International Conference on, IEEE, pp. 329-334, Jun, 2016.
fault-tolerant control systems with performance optimization,” IEEE [42] W. A. Smith and R. B. Randall, “Rolling element bearing diagnostics
Transactions on Industrial Electronics, vol. 61, no. 5, pp. 2402-2411, using the Case Western Reserve University data: A benchmark study,”
2014. Mechanical Systems and Signal Processing, vol. 64, pp. 100-131, Dec,
[22] S. Yin, G. Wang and H. Gao, “Data-driven process monitoring based on 2015.
modified orthogonal projections to latent structures,” IEEE Transactions [43] X. Zhang, Y. Liang and J. Zhou, “A novel bearing fault diagnosis model
on Control Systems Technology, vol. 24, no. 4, pp. 1480-1487, 2016. integrated permutation entropy, ensemble empirical mode decomposition
and optimized SVM,” Measurement, vol. 69, pp. 164-179, Jun, 2015.

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TIE.2017.2774777, IEEE
Transactions on Industrial Electronics
IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS

Long Wen was born in Hubei, China, in

December, 1988. He received his B.E degree in
2010 and Ph.D. in 2014 in Industrial Engineering
from the Huazhong University of Science and
Technology (HUST), Wuhan, China.
He is currently a Post-doctor in the Department
of Industrial & Manufacturing Systems
Engineering, State Key Laboratory of Digital
Manufacturing Equipment & Technology, School
of Mechanical Science & Engineering, HUST. His
research interests include deep learning, machine learning, intelligent
algorithm, etc.

Xinyu Li received his Ph.D. degree in industrial

engineering from Huazhong University of
Science and Technology (HUST), China, 2009.
He is an Associate Professor of the
Department of Industrial & Manufacturing
Systems Engineering, State Key Laboratory of
Digital Manufacturing Equipment & Technology,
School of Mechanical Science & Engineering,
HUST. He had published more than 60 refereed
papers. His research interests include intelligent
algorithm, big data and machine learning etc.

Liang Gao (M’08) received Ph.D. degree in

mechatronic engineering from Huazhong
University of Science and Technology (HUST),
China, 2002.
He is a Professor of the Department of
Industrial & Manufacturing System Engineering,
State Key Laboratory of Digital Manufacturing
Equipment & Technology, School of Mechanical
Science & Engineering, HUST. He had published
more than 140 refereed papers. His research
interests include operations research and
optimization, big data and machine learning etc.

Yuyan Zhang was born in Henan, China, in

January, 1989. He received his B.E degree in
2012 and M. degree in 2015 from Xiangtan
University, Xiangtan, China.
He is currently pursuing his Ph.D. degree in the
Department of Industrial & Manufacturing
Systems Engineering, State Key Laboratory of
Digital Manufacturing Equipment & technology,
School of Mechanical Science and Engineering
form Huazhong University of Science and
Technology (HUST). His research interests include deep learning and
its applications in condition-based maintenance for rotating machinery.

0278-0046 (c) 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.