Hindawi

Mathematical Problems in Engineering

Volume 2021, Article ID 7321394, 13 pages
https://doi.org/10.1155/2021/7321394

Research Article
Automatic Fabric Defect Detection Based on an
Improved YOLOv5

1,2 1
Rui Jin and Qiang Niu
1
School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, China
2
Changzhou Vocational Institute of Textile and Garment, Changzhou, China

Correspondence should be addressed to Qiang Niu; [email protected]

Received 7 August 2021; Revised 6 September 2021; Accepted 9 September 2021; Published 30 September 2021

Academic Editor: Paolo Spagnolo

Copyright © 2021 Rui Jin and Qiang Niu. This is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is
properly cited.
Fabric defect detection is particularly remarkable because of the large textile production demand in China. Traditional manual
detection method is inefficient, time-consuming, laborious, and costly. A deep learning technique is proposed in this work to
perform automatic fabric defect detection by improving a YOLOv5 object detection algorithm. A teacher-student architecture is
used to handle the shortage of fabric defect images. Specifically, a deep teacher network could precisely recognize fabric defects.
After information distillation, a shallow student network could do the same thing in real-time with minimal performance
degeneration. Moreover, multitask learning is introduced by simultaneously detecting ubiquitous and specific defects. Focal loss
function and central constraints are introduced to improve the recognition performance. Evaluations are performed on the
publicly available Tianchi AI and TILDA databases. Results indicate that the proposed method performs well compared with other
methods and has excellent defect detection ability in the collected textile images.

1. Introduction which are sensitive to illumination changes. Ye [5] presented

the fuzzy inference based on image histogram statistical
The textile industry is the traditional advantageous industry in variables, which is robust to defects’ rotation and translation.
China’s economic development and is an important liveli- However, handling complex image texture is difficult. For
hood industry. The quality of textiles has a great influence on complex texture images, researchers proposed methods based
the textile industry. Fabric defects would reduce the cost and on edges [6], local binary patterns [7, 8], contour waves [9],
profit by 45%–65% [1]. Therefore, defect detection plays an and gray co-occurrence matrix [10, 11]. These methods
important role in the control of textile quality. Traditional perform well in identifying defective images but have diffi-
textile defect testing is usually achieved by training skilled culty recognizing specific fabric defects. Moreover, several
operators with high training costs, and the manual detection researchers used the characteristics of the high-frequency
efficiency is low (the detection speed is less than 20 m/min). parts, such as Fourier transform methods [12, 13], Gabor filter
The error and leakage rates are high due to personnel fatigue methods [7, 14], and wavelet transform methods [15, 16].
or other subjective factors. Hence, how to detect fabric defects Compared with fabric defect detection in the spatial domain,
by automatic means has become an engaging, difficult re- it has more space-time overheads in the frequency domain.
search spot in the field of textile industry and machine vision. Deep learning has been widely used in the fields of
The core of machine-vision-based fabric defect detection computer vision [17–19]. Researchers designed deep
is extracting the characteristics related to defects from the neural networks to realize fabric defect detection in a
textile images. A detailed review of the machine-vision-based data-driven manner. Liu et al. [20] proposed using
fabric defect detection methods could be found in References multistage GAN the detection of fabric defects through
[2, 3]. Thomas and Cattoen [4] used the gray-scale means of unsupervised data reconstruction. Hence, it could over-
image rows and columns as defect-related characteristics, come the challenges of diversified fabric defects. Mei et al.
2 Mathematical Problems in Engineering

[21] introduced a multiscale convolutional denoising its ability to detect and recognize specific fabric defects. To
autoencoder to learn the reconstruction of textile images. verify the generalization capability of the proposed algo-
The reconstruction errors are utilized to realize automatic rithm, it is tested on self-collected fabric defect images and
defect detection. Xian et al. [22] studied the problem of achieves good results.
metallic surface defect detection that is similar to fabric
defect detection. Convolutional neural network-based 2. Related Technologies
segmentation is used to detect and recognize defect re-
gions. Wei et al. [23] used faster-RCNN to detect fabric 2.1. Convolutional Neural Networks. Convolutional neural
defects automatically. It achieves satisfied detection networks (CNNs) are widely used in computer vision tasks
performance benefiting from faster-RCNN’s strong fea- [31]. CNN is a kind of feed-forward neural network that
ture engineering ability. However, faster-RCNN has large contains convolutional computation and deep structure. It
space-time complexity due to its two-stage object de- has the representation learning ability to learn structured
tection scheme. Jing et al. [24] improved YOLOv3, which and translation-invariant information from input images.
is a single-stage object detection method with real-time Compared with fully connected operations, CNN has the
detection performance. Then, it could better detect fabric advantage of small computational overhead. A common
defects. CNN-based computer vision system consists of the fol-
In addition, several researchers studied the model-driven lowing parts:
fabric defect detection methods, such as Markov random Input layer: it performs gray processing, normalization,
field [25], autoregression [26], and sparse dictionary [27, 28]. and data augmentation on the input images.
After effective training, these methods could identify small- Convolutional layer: it performs convolutional opera-
region defects. However, they are vulnerable to external tions in each layer to ensure the forward and backward
signals such as noise and light. transmission of the information. The feature map of the lth
In conclusion, many researchers have proposed different layer is derived from that of the l−1th layer using the
methods to study how to detect fabric defects. However, convolutional operation, as follows:
detecting fabric defects is still challenging owing to many
yl(i,j) � Kli ∗ xl(r ) ,
j
(1)
kinds of defects with large differences and uneven distri-
butions. These problems lead to the difficulty of designing an where Kli is the weight of ith convolutional kernel in the lth
effective system to detect and localize the fabric defects j
layer, and xl(r ) represents the jth local region being cal-
automatically. Moreover, the proposed system is required to culated in the lth layer.
operate faster and could be realized in an intelligent edge Activation layer: it always follows the convolutional layer
device platform. to introduce nonlinearity. Hence, the network could have
According to the above requirements, a lightweight better representation learning ability. Commonly used ac-
fabric defect detection method is proposed by improving tivation functions contain Sigmoid, Tanh, ReLu, and their
YOLOv5 [29] based on the special needs of the defect de- variants. Figure 1 shows the curves of three different acti-
tection system. It could detect and recognize special fabric vation functions.
defects in real time. The main contributions of this article are
as follows. (1) Pooling layer: it is used to subsample the feature
A teacher-student architecture is introduced to detect maps to decrease computational overheads. It could
fabric defects. The deep teacher network could precisely also mitigate the overfitting phenomenon. Com-
recognize fabric defects. After information distillation, a monly used pooling functions consist of the average
shallow student network could do the same thing in real time and maximum pooling strategies.
with minimal performance degeneration. The student net- (2) Output layer: it presents various structures according
work could be deployed in the edge equipment because of its to different computer vision applications. For clas-
low space-time overheads. sification tasks, the SoftMax function is often used in
To solve the problems of many kinds of fabric defects the output layer to calculate the probability that the
that are difficult to be distinguished, a multitask learning input belongs to each category, thus obtaining the
strategy is proposed to detect ubiquitous and specific defects classification results.
simultaneously. Such a strategy could fully utilize the
The five components above are used in the improved
complementary between ubiquitous and specific defects.
YOLOv5. They would not be introduced in detail in the
Moreover, an attention mechanism is used to enhance the
following sections.
defect-related features.
To handle data imbalance and small-region defects
better, the focal loss function [30] is employed to mitigate 2.2. Object Detection Algorithm. Object detection is one of
data imbalance. The center loss is introduced as a constraint the essential issues in the field of computer vision. It enables
to increase the interclass distance while reducing the the computer to discover and locate targets of interest from
intraclass distance, hence improving the recognition per- images automatically, such as flaws in the fabric. Deep
formance of specific defects. learning-based object detection algorithms have achieved
The proposed method is evaluated on the publicly great successes recently. Commonly used methods include
available Tianchi AI and TILDA databases. The results reveal RCNN [32], fast-RCNN [33], faster-RCNN [34], SDD [35],
Mathematical Problems in Engineering 3

1 1 10

6
0
0.5
-10 -5 0 5 10
4

0 0
-1
-10 -5 0 5 10 -10 -5 0 5 10
Sigmoid tanh ReLU
Figure 1: Curves of the Sigmoid, Tanh, and ReLu activation functions.

and YOLO [36]. However, the above methods have difficulty defects may occupy a small proportion in the overall image,
meeting the real-time requirements of the fabric defect spatial attention can be used to pay more attention to small-
detection system because they have high computational region defects. Moreover, channel attention is used to refine
overheads. To balance precision and speed, a lightweight features and improve the algorithm performance. Figure 3
object detection network, named YOLOv5, is used in this shows that input feature F is initially processed by max-
work. The traditional YOLOv5 is improved based on the pooling (MaxPool) and average-pooling (AvgPool). Then,
characteristics of the fabric defects, such that it can be channel and spatial attention realize feature transformation
applied to the fabric defect detection system. with the shared three-layer MLP and convolutional oper-
Figure 2 demonstrates the structure of the traditional ation, respectively. Finally, the sigmoid activation function is
YOLOv5, which mainly includes Bakbone, PANet, and used to calculate different attention weights.
Output. Bakbone is used to perform feature engineering
from input images. PANet could obtain visual features 3. Proposed Method
robust to scale changes due to the used pyramid structure.
The positions are output, and the regions of interest are Figure 4 illustrates the flow chart of the overall algorithm: (1)
classified simultaneously. Assuming the input image size as training stage: fabric images after data augmentation are sent
608 ∗ 608 ∗ 3 (height ∗ width ∗ channels), the Output to the teacher network to detect specific fabric defects. Then,
part could output three different scales of features with the defect-related knowledge is distilled from the teacher
dimensions of 76 ∗ 76 ∗ 255, 38 ∗ 38 ∗ 255, and network to the lightweight student network. (2) Testing
19 ∗ 19 ∗ 255. Specific details of the YOLOv5 network stage: the student network is used to detect specific fabric
could be found in [29]. defects in real-time performance and with minimal per-
formance degradation. The testing stage requires deploying
the student network on the NVIDIA JETSON TX2 platform
2.3. Attention Mechanism. The attention mechanism draws based on TensorRT, which is used to accelerate the student
on human’s selective attention characteristic. Specifically, a network.
human being could quickly scan the global image and
concentrate on the regions of interest. Then, detail infor-
mation of these regions are obtained, and useless infor- 3.1. Teacher Network Structure. The structure of the pro-
mation is suppressed. Based on different applications, posed teacher network is shown in Figure 5. The feature
attention mechanism could be divided into temporal at- extraction part and multiscale information extraction part of
tention, spatial attention, and channel attention. Temporal the teacher network are implemented using Backbone and
attention [37] could assign different weights to sequence PANet of the YOLOv5 network. Their specific structures
features. Then, the model could automatically focus on have been introduced in Section 2.2 and are not repeated
important sequence features, thus enhancing the ability to here. Two improvements are presented to perform better
process sequence data without increasing the computational fabric defect detection.
costs. Spatial attention [38] transforms the spatial infor- (1) Attention enhancement mechanism: the defect areas
mation in the original image into another space and retains may occupy small regions in the overall textile image.
the key information, thereby identifying the substantial areas Extracting defect-related features from these small
and increasing the attention on these areas. Channel at- regions is still a problem, even if PANet could extract
tention [39] excavates effective features from the channel the context information. Hence, the attention en-
dimension and suppresses task-independent features, thus hancement mechanism is introduced to mitigate the
improving network performance. problem. First, spatial attention is used to enhance
For fabric defect detection, temporal attention cannot be the network’s sensitivity to small defect areas. Then,
used because the input is a static image. Considering that the channel attention is used to suppress the
4 Mathematical Problems in Engineering

Input Backbone PANet Output

Focus CBL CSP1_1 CBL CSP1_3 CBL CSP1_3 CBL CSP1_3 CBL SPP CSP2_1 CBL

608*608*3 upSam
Concate CSP2_1 CBL
pling
Concate CSP2_1 conv 76*76*255

Leaky CBL
CBL = conv BN
relu 38*38*255
Concate CSP2_1 conv

Res
= CBL CBL add CBL
unit
19*19*255
Concate CSP2_1 conv

CSP1_x = CBL Res slice Maxpo

conv
unit ol
Leaky
Concate BN CBL slice Maxpo
relu SPP = CBL
conv Focus = Concate CBL Concate CBL
ol
slice
Maxpo
ol
slice
CSP2_x = CBL CBL conv

Leaky
Concate BN CBL
relu
conv

Figure 2: Pipeline of YOLOv5.

Channel Attention Module

MaxPool

Activation
σ
AvgPool
Channel Attention
Input feature F Shared MLP

Spatial Attention Module

Convolution

Activation
σ

[MaxPool; AvgPool] Spatial

Input feature F
Attention

Figure 3: Illustration of spatial and channel attention mechanisms.

Teacher Network

Distillation

Student Network

Figure 4: Flow chart of the overall algorithm.

Mathematical Problems in Engineering 5

Input

Attention
Backbone PANet
Enhancement

Normal/ Fabric Defect

Abnormal Classification
Output

Output
Fusion module
Probability

Figure 5: Structure of the proposed teacher network.

nondefective features, thus highlighting the defective (3) PN and all Pj are concatenated, and the concatenated
features. Assuming that the output of PANet is F, vector is then sent into the SoftMax activation
spatial attention weight As (F) and channel attention function for normalization. Then, the probability
weights Ac (F) could be calculated as follows: that the given fabric image belongs to a normal
sample or a certain defect could be obtained.
Ac (F) � σ 􏼐MLP􏼐Fsavg 􏼑 + MLP Fsmax 􏼁􏼑,
(2)
As (F) � σ 􏼐Conv􏼐Fsavg ; Fsmax 􏼑􏼑,
3.2. Student Network Structure. Figure 6 exhibits the
where MLP () represents a shared multilayer per- structure of the proposed student network. Different from
ceptron (three layers, the number of neurons is m, the teacher network, the student network performs the
m/4, m, respectively; m represents the channel di- following lightweight processing:
mension of F.) and Conv () represents a convolution (1) The backbone part is thin. Specifically, only two sets
operation with the kernel size 7 ∗ 7. The attention of BottleNeckCSP modules are preserved in the new
enhancement mechanisms used in this work are backbone part. Details of the BottleNeckCSP module
defined as follows: could be found in [29].
F′ � Ac (F) ⊗ F, (2) The PANet is removed to reduce the space-time
(3) complexity. The student network relies on the
F″ � As F′ 􏼁 ⊗ F′ .
knowledge distilled from the teacher network to
(2) Multitask learning strategy: the fabric defect detec- extract multiscale features.
tion task is usually divided into ubiquitous defect The rest of the student network, including the attention
detection and specific defect recognition. Comple- enhancement, multitask learning strategy, and information
mentarity exists between these two tasks. Hence, the fusion, are the same with the teacher network.
multitask learning strategy is introduced to utilize
the complementarity fully. Specifically, two detection
heads are designed to detect ubiquitous defects and 3.3. Loss Functions. The network is trained in a multitask
recognize specific defects. A fusion model is then learning manner, and a weighted combined loss function is
proposed to fuse the outputs of two detection heads presented to optimize the network. The loss functions used
to predict a more accurate defect recognition consist of the following sections:
probability. Details are as follows:
(1) The ubiquitous defect detection is termed as a binary
(1) For the detection head to detect ubiquitous defects,
classification problem. A cross-entropy loss function
the defective probability of the ROI with the largest
LT is used and defined as follows:
defective probability is defined as PA. Then, the
normal probability of the given fabric image is de- 1 N
fined as PN � 1 − PA. LT � − 􏽘 􏼂yi ln pi + 1 − yi 􏼁ln 1 − pi 􏼁􏼃, (4)
N i�1
(2) For the detection head to recognize specific defects,
the defective probability of each ROI is defined as Pj where yi represents the sample label, pi represents the
(j � 1, . . ., M), where M indicates the number of output probability of the ubiquitous defects detec-
ROIs. tion head, and N represents the number of samples.
6 Mathematical Problems in Engineering

Input Backbone Enhancement Normal/

Abnormal
BottleNeckCSP Spatial_Att

BottleNeckCSP Channel_Att

Fabric Defect
Classification

Output

Output
Fusion module
Probability

Figure 6: Structure of the proposed student network.

(2) The specific defect detection is termed as a multiclass L � w1 LT + w2 LS + w3 LF + w4 Lc , (8)

problem. A SoftMax loss function Ls is used and
defined as follows: where the weights w1 , w2 , w3 , and w4 are set to 0.4, 0.4, 0.1,
K and 0.1, respectively. Settings of different weights are ob-
􏽥 i log si ,
LS � − 􏽘 y (5) tained based on the crossvalidation on the publicly available
i�1 databases.
where K represents the kinds of specific defects, y 􏽥i
represents the one-hot encoding of the ground truth 4. Experimental Results
label, and si indicates the probability that the sample
4.1. Databases. One public database comes from the
belongs to the ith defect.
Xuelang Tianchi AI Challenge. It contains 3,331 labeled
(3) Considering the sample imbalance in the ubiquitous images with the rectangular locations to label the defects.
defect detection head, focal loss function LF is used to The number of normal pictures is 2,163, and the number
mitigate the problem. LF is defined as follows: of defective pictures is 1,168. It has 22 kinds of defects,
c including jumps, knots, stains, puncture holes, and
LF � −α 1 − pi 􏼁 log pi 􏼁, (6)
lacking warp. The data distribution on the database shows
where the hyperparameters α and c are used to al- an unbalanced state in which the number of normal
leviate the imbalance problem of positive and neg- pictures is much higher than the number of defective
ative samples and difficult samples, respectively. pictures. Using the same experimental protocol as [19],
(4) To improve the feature discriminability in the spe- the specific defect category is reintegrated into puncture
cific defect detection head, central loss function LC is hole, knots, rubbing hole, thin spinning, jumps, hanging
employed to increase the interclass distances while warp, lacking warp, brushed hole, stains, and others. In
reducing the innerclass distances of learned features. experiments, 70% of the entire database is taken as the
LC is defined as follows: training set, and the remaining 30% are the test set.
Several training samples and their labels are shown in
Figure 7.
1 N ��� ��2
LC � 􏽘 ��xi − cyi ���2 , (7) Another used public database is TILDA, a well-known
2 i�1 fabric texture database containing eight kinds of repre-
sentative fabric categories. Seven error classes and a
where xi represents the sample encoding, and cyi is
correct class are defined according to the textile atlas
the center of the corresponding category, which xi
analysis. Similar to [40], 300 fabric images are chosen and
belongs to.
are divided into six categories, such as holes, scratch,
The final loss function of the proposed method is cal- knots, stain, carrying, and normal. Each class consists of
culated in a weighted manner as follows: 50 fabric images, and each image is resized to 256 × 256
Mathematical Problems in Engineering 7

jumps puncture hole lacking warp hanging warp

thin spinning rubbing hole knots stains

Figure 7: Illustrations of some training samples and their labels on the Tianchi AI database.

pixels. In experiments, 70% of the entire database is taken 4.3. Qualitative Analysis. A qualitative analysis of the pro-
as the training set, and the remaining 30% are the test set. posed method is performed from three aspects: (1) the ability of
Figure 8 demonstrates several samples and their labels. the proposed teacher network to detect specific defects on
public databases is evaluated, and OurNet is used for com-
parison; (2) the accuracy of the proposed teacher network to
4.2. Evaluation Metrics. The defect detection algorithm locate the defect areas is evaluated, and the improved YOLOv3
proposed in this work could distinguish between normal and proposed by Jing et al. [24] is used for comparison; and (3)
defect images and identify specific fabric defects. Therefore, comparisons between the teacher and student networks are
area under the ROC curve (AUC) and mean average pre- performed on self-collected fabric images to verify the gen-
cision (mAP) are used as metrics for evaluation. The former eralization performance of the proposed method. Quantitative
reflects the algorithm’s ability to distinguish between normal comparisons between the teacher and student networks will be
and defective fabric images, whereas the latter reflects the introduced in the following section.
algorithm’s ability to recognize specific fabric defects. To Figure 9 demonstrates the comparisons between the
calculate AUC and mAP, precision (P) and recall (R) are proposed teacher network and OurNet in detecting specific
calculated initially, as follows: defects on the Tianchi AI database. The results show that our
method successfully recognizes different defect types
TP benefiting from the used multitask learning, focal loss
P� ,
(TP + FP) function, and the center loss constraint. By contrast, OurNet
(9) fails to identify the puncture hole defects. It also mistakes the
TP
R� , brushed hole and thin spinning defects for others and jumps
(TP + FN) defects, respectively.
where TP (true positive) represents the number of samples Figure 10 shows the location results between the pro-
whose labels are positive, and the actual forecasts are pos- posed teacher network and the improved YOLOv3 proposed
itive. FP (false positive) indicates the number of samples by Jing et al. [24] on the Tianchi AI database. Types of
whose labels are negative, and the actual forecasts are specific defects are labeled under each subfigure for a clearer
positive. FN (false negative) represents the number of view. In the subfigure, the green box represents the real
samples whose labels are positive, and the actual forecasts defect area, the red box is the positioning result of the
are negative. Based on the calculated P and R, the P-R curve proposed teacher network, and the yellow box is the posi-
could be obtained. Then, the ROC curve could be obtained. tioning result of the improved YOLOv3. Figure 10 shows
The cover area of the ROC curve is AUC. that the defect regions predicted by the proposed method are
mAP represents the mean of different APs, where AP more accurate than those predicted by the improved
represents the area under the P-R curve. mAP is calculated as YOLOv3. Such superiority may be a benefit from the strong
follows: YOLOv5 and our improvements. The improved YOLOv3
suffers from positioning small defect areas, although it could
1 K detect most defects. For example, it fails to detect the
mAP � 􏽘 R − Ri−1 􏼁Pi , (10)
K i�1 i hanging warp and jump defects.
Figure 11 compares the teacher and student networks
where k represents the number of categories. on self-collected fabric images, specifically, their
8 Mathematical Problems in Engineering

Normal Knots Holes

Scratch Stain Carrying

Figure 8: Illustrations of several samples and their labels on the TILDA database.

performance in positioning defect areas. In each sub- A quantitative comparison between the teacher and
figure, the green box represents the real defect area, the student networks is presented in Table 2. The identification
red box is the positioning result of the teacher network, times are tested on an Nvidia JETSON TX2. The table shows
and the yellow box is the positioning result of the student that the student network could still meet the needs of fabric
network. The teacher network could more accurately defect detection, despite the performance degradation ob-
identify the defect areas. The defect detection performance served compared with the teacher network. More impor-
of the student network is slightly weaker than that of the tantly, the identification time of the student network is
teacher network. However, the student network has lower approximately half of the teacher network. Its identification
space-time overheads; thus, it is more suitable to be time guarantees the real-time performance on embedded
arranged for embedded systems. devices.
Finally, comparisons with other mainstream methods
are performed to verify the effectiveness of the proposed
4.4. Quantitative Analysis Results. An ablation study is method. The improved YOLOv3 [24] and the pretrained
performed on the Tianchi AI database to verify the effects of deep CNN [40] are selected as the fabric defect detection
different improvement methods, including multitask algorithms. Faster-RCNN [34] and YOLOv5 [29] are se-
learning, focal loss, and central loss constraints. The results lected as the universal object detection methods. The
are presented in Table 1. The ablation study of the teacher comparison results are presented in Table 3.
network shows that the student network has similar results. The above table shows that the original OurNet based on
Table 1 shows that the teacher network is degraded into AlexNet has poor defect detection performance because it
traditional YOLOv5 when none of the improvements is fails to handle small defect areas well. Two variants of
used. Compared with the YOLOv5-based detection method, OurNet, namely, OurNet-VGG16 and OurNet-ResNet,
the introduced attention module could lead to an improved obtain better performance benefit from extracting better
performance with increased AUC and mAP. Then, AUC and features with deeper structures. Jing et al. [24] achieves
mAP are further improved by simultaneously detecting better defect detection performance using improved
ubiquitous and specific defects with the proposed multitask YOLOv3 networks. A pretrained CNN is also beneficial in
learning strategy because of the complementarity between boosting the defect detection performance as proposed by
different tasks. Based on the multitask learning strategy, the Jing et al. [40]. YOLOv5 and faster-RCNN achieve similar
introduction of the focal loss function and central loss defect detection performance benefiting from their strong
constraint could further improve the defect detection results. power in object detection. Both methods are superior to the
Simultaneously using all improvements achieves the best student network proposed in this work, but the time
performance on the Tianchi AI database, which verifies the overhead is relatively large. The proposed teacher network
effects of different improvement methods. achieves the best fabric defect detection performance,
Mathematical Problems in Engineering 9

Ground Truth: rubbing hole Ground Truth: hanging Ground Truth: knots
OurNet: rubbing hole OurNet: hanging warp OurNet: knots
Ours: rubbing hole Ours: hanging warp Ours: knots

Ground Truth: lacking warp Ground Truth: jumps Ground Truth: stains
OurNet: lacking warp OurNet: jumps OurNet: stains
Ours: lacking warp Ours: jumps Ours: stains

Ground Truth: brushed hole Ground Truth: puncture hole Ground Truth: thin spinning
OurNet: others OurNet: normal OurNet: jumps
Ours: brushed hole Ours: puncture hole Ours: thin spinning
Figure 9: Comparisons between the teacher network and OurNet in detecting specific defects.

whereas the student network provides an alternative to [24] proposed by Jing et al. [40] achieve similar performance
detect fabric defects with acceptable accuracy on embedded due to the reason discussed above. Similar to the compar-
devices. isons on the Tianchi AI database, two state-of-the-art de-
Table 4 presents the comparisons between different tectors, YOLOv5 [29] and faster-RCNN [34], obtained
methods on the TILDA database. OurNet [41] and its higher AUC and mAP compared with that of the proposed
variants perform much better than on the Tianchi AI da- student network. The proposed teacher network still ach-
tabase because the TILDA database contains fewer cate- ieves the best defect detection performance, which verifies
gories and equal samples per category. Improved YOLOv3 the accuracy of the proposed method.
10 Mathematical Problems in Engineering

knots brushed hole

others puncture hole

hanging warp jumps

Figure 10: Comparisons between the teacher network and the improved YOLOv3 in positioning defect areas.
Mathematical Problems in Engineering 11

Figure 11: Comparisons between the teacher and student networks on self-collected fabric images.

Table 1: Ablation study of the teacher network on the Tianchi AI database.

Attention module Multitask learning Focal loss Central loss constraints AUC mAP
0.938 0.403
√ 0.957 0.412
√ √ 0.965 0.431
√ √ √ 0.971 0.441
√ √ √ 0.973 0.442
√ √ √ √ 0.981 0.447

Table 2: Quantitative comparisons between the teacher and student networks on the Tianchi AI database.
AUC mAP Identification times (ms)
YOLOv5 0.957 0.412 32
Teacher network 0.981 0.447 35
Student network 0.952 0.406 16

Table 3: Comparisons of different fabric defect detection algorithms on the Tianchi AI database.
AUC mAP
OurNet [41] 0.787 0.104
OurNet-VGG16 0.848 0.288
OurNet-ResNet 0.882 0.311
Improved YOLOv3 [24] 0.927 0.372
Jing et al. [40] 0.932 0.382
YOLOv5 [29] 0.957 0.412
Faster-RCNN [34] 0.956 0.413
Student network 0.952 0.406
Teacher network 0.981 0.447
12 Mathematical Problems in Engineering

Table 4: Comparisons of different fabric defect detection algorithms on the TILDA database.
AUC mAP
OurNet [41] 0.866 0.301
OurNet-VGG16 0.912 0.346
OurNet-ResNet 0.926 0.382
Jing et al. [40] 0.958 0.411
YOLOv5 [29] 0.970 0.442
Faster-RCNN [34] 0.972 0.443
Student network 0.965 0.428
Teacher network 0.988 0.451

5. Discussion and Conclusion Mathematical Problems in Engineering, vol. 2020, Article ID

8189403, 24 pages, 2020.
An automatic fabric defect detection method based on [3] A. Latif, A. Rasheed, and U. Sajid, “Content-based image
YOLOv5 is proposed because of the considerable role of retrieval and feature extraction: a comprehensive review,”
fabric defect detection in the textile industry. A teacher- Mathematical Problems in Engineering, vol. 2019, Article ID
student architecture is used in considering the real-time 9658350, 21 pages, 2019.
requirements of the fabric defect detection. The deep teacher [4] T. Thomas and M. Cattoen, “Automatic inspection of simply
network could precisely detect specific fabric defects. After patterned material in the textile industry,” in Proceedings of
SPIE: Society of Photo-Optical Instrumentation Engineers,
knowledge distillation, the shallow student network could
pp. 2–12, Bellingham, WA, USA, 1994.
perform fabric defects in real time with an acceptable ac- [5] Y. Ye, “Fabric defect detection using fuzzy inductive rea-
curacy. A multitask learning strategy is introduced to detect soning based on image histogram statistic variables,” in
ubiquitous and specific defects simultaneously, and better Proceedings of the 6th International Conference on Fuzzy
utilize the complementarity between different tasks. Focal Systems and Knowledge Discovery, pp. 191–194, Tianjin,
loss and center loss constraints are introduced for better China, August 2009.
defect detection performance. Evaluations are performed on [6] X. Jia, “Fabric defect detection based on open source com-
the public databases and self-collected fabric images. puter vision library OpenCV,” in Proceedings of the 2010 2nd
Comparisons with other mainstream methods indicate that International Conference on Signal Processing Systems, Dalian,
the proposed method is applicable to the automatic detec- China, July 2010.
tion task of textile defects, which can greatly improve the [7] J. Jing, H. Zhang, J. Wang, P. Li, and J. Jia, “Fabric defect
accuracy and efficiency of defect detection and enhance the detection using Gabor filters and defect classification based on
automation level of the textile industry. LBP and Tamura method,” Journal of the Textile Institute,
vol. 104, no. 1, pp. 18–27, 2013.
[8] M. Hao, J. Junfeng, and S. Zebin, “Patterned fabric defect
Data Availability detection based on LBP and HOG feature,” Journal of Elec-
tronic Measurement and Instrument, vol. 32, no. 4, pp. 95–102,
The Xuelang Tianchi AI Challenge dataset is publicly
2018.
available. [9] D. Yapi, M. S. Allili, and N. Baaziz, “Automatic fabric defect
detection using learning-based local textural distributions in
Conflicts of Interest the contourlet domain,” IEEE Transactions on Automation
Science and Engineering, vol. 15, no. 3, pp. 1014–1026, 2017.
The authors declare no conflicts of interest. [10] N. T. Deotale and T. K. Sarode, “Fabric defect detection
adopting combined GLCM, gabor wavelet features and ran-
Authors’ Contributions dom decision forest,” 3D Research, vol. 10, no. 1, p. 5, 2019.
[11] M. A. Shabir, M. U. Hassan, and X. Yu, “Tyre defect detection
All authors have read and agreed to the published version of based on GLCM and gabor filter,” in Proceedings of the 2019
the manuscript. 22nd International Multitopic Conference (INMIC), pp. 1–6,
Islamabad, Pakistan, November 2019.
Acknowledgments [12] G. Liu and X. Zheng, “Fabric defect detection based on in-
formation entropy and frequency domain saliency,” The
This research was funded by the National Natural Science Visual Computer, vol. 37, pp. 1–14, 2020.
Foundation of China under grant no. 51674265. [13] C. Chi-Ho Chan and G. K. H. Pang, “Fabric defect detection
by Fourier analysis,” IEEE Transactions on Industry Appli-
References cations, vol. 36, no. 5, pp. 1267–1276, 2000.
[14] L. Jia, C. Chen, J. Liang, and Z. Hou, “Fabric defect inspection
[1] K. Srinivasan, P. H. Dastoor, and P. Radhakrishnaiah, “FDAS: based on lattice segmentation and Gabor filtering,” Neuro-
a knowledge-based framework for analysis of defects in woven computing, vol. 238, pp. 84–102, 2017.
textile structures,” Journal of the Textile Institute Proceedings [15] X. Yang, G. Pang, and N. Yung, “Discriminative training
and Abstracts, vol. 83, no. 3, pp. 431–448, 1990. approaches to fabric defect classification based on wavelet
[2] A. Rasheed, B. Zafar, and A. Rasheed, “Fabric defect detection transform,” Pattern Recognition, vol. 37, no. 5, pp. 889–899,
using computer vision techniques: a comprehensive review,” 2004.
Mathematical Problems in Engineering 13

[16] S. Sadaghiyanfam, “Using gray-level-co-occurrence matrix [33] R. Girshick, “Fast r-cnn,” in Proceedings of the IEEE Inter-
and wavelet transform for textural fabric defect detection: a national Conference on Computer Vision, pp. 1440–1448,
comparison study,” in Proceedings of the 2018 Electric Elec- Santiago, Chile, December 2015.
tronics, Computer Science, Biomedical Engineerings’ Meeting [34] S. Ren, K. He, and R. Girshick, “Faster r-cnn: towards real-
(EBBT), pp. 1–5, Istanbul, Turkey, April 2018. time object detection with region proposal networks,” Ad-
[17] B. Yang, G. Yan, and P. Wang, “A novel graph-based tra- vances in Neural Information Processing Systems, vol. 28,
jectory predictor with pseudo-oracle,” 2021, https://arxiv.org/ pp. 91–99, 2015.
abs/2002.00391. [35] W. Liu, D. Anguelov, D. Erhan et al., “Ssd: single shot
[18] J. Wang, P. Fu, and R. X. Gao, “Machine vision intelligence for multibox detector,” in Proceedings of the European Conference
product defect inspection based on deep learning and Hough on Computer Vision, Amsterdam, The Netherlands, October
2016.
transform,” Journal of Manufacturing Systems, vol. 51,
[36] J. Redmon, S. Divvala, R. Girshick et al., “You only look once:
pp. 52–60, 2019.
unified, real-time object detection,” in Proceedings of the IEEE
[19] B. Yang, W. Zhan, and P. Wang, “Crossing or not? Context-
Conference on Computer Vision and Pattern Recognition,
based recognition of pedestrian crossing intention in the pp. 779–788, Las Vegas, NV, USA, June 2016.
urban environment,” IEEE Transactions on Intelligent [37] A. Vaswani, N. Shazeer, and N. Parmar, “Attention is all you
Transportation Systems, 2021. need,” 2017, https://arxiv.org/abs/1706.03762.
[20] J. Liu, C. Wang, and H. Su, “Multistage GAN for fabric defect [38] F. Locatello, D. Weissenborn, and T. Unterthiner, “Object-
detection,” IEEE Transactions on Image Processing, vol. 29, centric learning with slot attention,” 2020, https://arxiv.org/
pp. 3388–3400, 2019. abs/2006.15055.
[21] S. Mei, Y. Wang, and G. Wen, “Automatic fabric defect [39] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: con-
detection with a multi-scale convolutional denoising volutional block attention module,” in Proceedings of the
autoencoder network model,” Sensors, vol. 18, no. 4, p. 1064, European Conference on Computer Vision (ECCV), Munich,
2018. Germany, September 2018.
[22] T. Xian, D. Zhang, and W. Ma, “Automatic metallic surface [40] J. F. Jing, H. Ma, and H. H. Zhang, “Automatic fabric defect
defect detection and recognition with convolutional neural detection using a deep convolutional neural network,” Col-
networks,” Applied Sciences-Basel, vol. 8, no. 9, 2018. oration Technology, vol. 135, no. 3, pp. 213–223, 2019.
[23] B. Wei, K. Hao, X.-S. Tang, and L. Ren, “Fabric defect de- [41] Z. Wu, Y. Zhuo, J. Li, Y. Feng, B. Han, and S. Liao, “A Fast
tection based on faster RCNN,” in Proceedings of the Inter- monochromatic fabric defect Fast detection method based on
national Conference on Artificial Intelligence on Textile and convolutional neural network,” Journal of Computer-Aided
Apparel, Hong Kong, China, June 2018. Design & Computer Graphics, vol. 30, no. 12, p. 2262, 2018.
[24] J. Jing, D. Zhuo, and H. Zhang, “Fabric defect detection using
the improved YOLOv3 model,” Journal of Engineered Fibers
and Fabrics, vol. 15, 2020.
[25] P. M. Mahajan, S. R. Kolhe, and P. M. Patil, “A review of
automatic fabric defect detection techniques,” Advances in
Computational Research, vol. 1, no. 2, pp. 18–29, 2009.
[26] J. Cao, J. Zhang, Z. Wen, N. Wang, and X. Liu, “Fabric defect
inspection using prior knowledge guided least squares re-
gression,” Multimedia Tools and Applications, vol. 76, no. 3,
pp. 4141–4157, 2017.
[27] X. Kang and E. Zhang, “A universal and adaptive fabric defect
detection algorithm based on sparse dictionary learning,”
IEEE Access, vol. 8, pp. 221808–221830, 2020.
[28] J. Zhou, D. Semenovich, A. Sowmya, and J. Wang, “Dictionary
learning framework for fabric defect detection,” Journal of the
Textile Institute, vol. 105, no. 3, pp. 223–234, 2014.
[29] A. Kuznetsova, T. Maleva, and V. Soloviev, “Detecting apples
in orchards using YOLOv3 and YOLOv5 in general and close-
up images,” in Proceedings of the International Symposium on
Neural Networks, Cairo, Egypt, October 2020.
[30] T. Y. Lin, P. Goyal, and R. Girshick, “Focal loss for dense
object detection,” in Proceedings of the IEEE International
Conference on Computer Vision, pp. 2980–2988, Venice, Italy,
October 2017.
[31] B. Yang, W. Zhan, N. Wang, X. Liu, and J. Lv, “Counting
crowds using a scale-distribution-aware network and adaptive
human-shaped kernel,” Neurocomputing, vol. 390, pp. 207–
216, 2020.
[32] S. Ren, K. He, and R. Girshick, “Object detection networks on
convolutional feature maps,” IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. 39, no. 7, pp. 1476–
1481, 2016.