Imp Reference 3
Imp Reference 3
DOI: 10.1002/pat.5226
RESEARCH ARTICLE
1
School of Materials Science and Engineering,
Beihang University, Beijing, China Abstract
2
Key Laboratory of Advanced Ship Materials The internal void defects induced during the manufacturing process of polymer-
and Mechanics, College of Aerospace and Civil
matrix composites can significantly degrade the mechanical properties of the com-
Engineering, Harbin Engineering University,
Harbin, China posite, particularly the interlaminar shear strength (ILSS). In this study, we devel-
3
Lightweight department, Beijing New Energy oped an innovative integrated methodology based on a deep learning semantic
Automotive Technology Innovation Center
Co., Ltd., Beijing, China segmentation algorithm, named DeepLabV3+, and a theoretically driven equation
4
Technology department, Bjcomposites Co., to automatically identify voids in optical images and investigate the relationship
Ltd., Beijing, China
between the microscopic voids and macroscopic ILSS parameters of the composite
Correspondence laminates. Results suggest that for the best fine-tuned DeepLabV3+ framework,
Ling Luo, School of Materials Science and
the corresponding mean pixel accuracy and intersection over union scores on the
Engineering, Beihang University, Beijing
100191, China. testing set were 99.84% and 90.82%, respectively, thereby indicating the potential
Email: [email protected]
of the generalized trained model. In addition, detailed experiments revealed that
Funding information the proposed method can successfully obtain the ILSS values of laminates with dif-
National Key R&D Program of China, Grant/
ferent void contents. In addition, the ILSS values of the carbon/epoxy laminates
Award Number: 2017YFB0703300; National
Natural Science Foundation of China, Grant/ decreased by approximately 27% with an increase in the void content from 0.07%
Award Number: 11872086
to 3.14%.
[Correction added on 4 February 2021, after
first online publication: Article title has been KEYWORDS
corrected in this version by removing “void deep learning, Interlaminar shear strength (ILSS), optical microscopy, polymer-matrix
defects of composite laminates” from its end] composites, porosity/voids
Polym Adv Technol. 2021;32:1853–1865. wileyonlinelibrary.com/journal/pat © 2021 John Wiley & Sons Ltd 1853
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1854 LUO ET AL.
technique is expensive and can only accommodate small-sized sam- scale well-annotated datasets with representative data distribution
ples.13 In contrast, optical images have attracted significant attention characteristics are crucial for obtaining more accurate or generalizable
for obtaining accurate data on the microstructures of composite lami- models for data-driven learning.25 However, obtaining datasets that
nates at various magnifications in a cost-driven production environ- are as comprehensively annotated as ImageNet in the materials
ment. In addition, optical images can be analyzed by image domain is challenging owing to their time-consuming, labor-consum-
segmentation methods to obtain the geometric parameters of the ing, and expensive requirements. To overcome this problem, utilizing
void structures. Accurate identification of the void structures enables the transfer learning strategy17,25 and fine-tuning DeepLabV3+
the accurate calculation of macroscopic ILSS parameters of the com- models that are pre-trained from natural image datasets to optical
posite laminates using a theoretically derived equation. The commonly image tasks has emerged as an effective method.
used image segmentation methods for analyzing optical images Therefore, in this study, we developed a novel method, which is
13,14 14-16
include thresholding, machine learning, and deep learn- based on a deep learning semantic segmentation algorithm, named
ing.17-22 Conventionally, thresholding techniques, which include global DeepLabV3+, and a theoretically driven equation to automatically iden-
and local thresholding, involve the selection of threshold based on the tify voids in optical images and to obtain data on the macroscopic ILSS
image information and the division of the image based on the gray parameters from the microscopic images of the composite laminates.
level. Global thresholding is the simplest thresholding technique for The pre-trained DeepLabV3+ framework was fine-tuned to automati-
separating images: below the threshold is the target, above the cally identify and segment the voids in optical images based on the
threshold is the background. In contrast, the threshold of local training model of the void characteristics. Furthermore, the ILSS param-
thresholding is determined by splitting the images into sub-images eters of the composite laminate were accurately calculated by identify-
and calculating the threshold for each sub-image. However, this tech- ing the void structures. Compared to the conventional image
nique is more time consuming and computationally expensive com- segmentation methods, the proposed method achieved a high void
pared to the global thresholding method. In general, the thresholding identification accuracy and image processing automation. In addition,
methods are suitable for analyzing images with different greyscale the proposed deep learning concept combined with the theoretically
ranges in the target and background. However, these techniques are derived equation accurately predicted the corresponding ILSSs from the
unsuitable for analyzing images with negligible greyscale differences optical images of the composite laminates with different void contents,
or overlapped greyscale ranges, as these may result in over- which has rarely been reported in the existing literature. We believe
segmentation or under-segmentation. In addition, analyzing the opti- that the proposed method will provide useful insights for the further
cal images of composites using traditional machine learning-based development of faster and more efficient void calculation methods.
image segmentation methods, such as random forest and K-means-
based image segmentation, may result in poor image segmentation
results owing to their limited model capacity. In contrast, deep 2 | EX PE RI MENT AL
learning-based semantic segmentation can provide a per-pixel label
map for the entire image. Consequently, the pixel-level semantic seg- 2.1 | Fabrication of plates
mentation enables an extensive investigation of the void characteris-
tics of composite plates. Briefly, square plates (300 × 350 mm2) with a nominal thickness of
Tremendous efforts have been devoted to address the limitations 2 mm were prepared from CYCOM 977–2 prepreg by autoclave
of image segmentation using deep learning methods.17-22 Conse- processing following the stacking sequence of [45/0/−45/90]2s. The
quently, the application of machine learning techniques for image seg- CYCOM 977–2 was obtained from CYTEC Industries (USA), and con-
mentation has emerged as an effective method to improve the tains 35 wt% of epoxy resin. Three uncured square plates were
application of image segmentation analysis. Machine learning tech- directly cut from the prepreg, stacked, and compacted every eight
niques have been successfully applied for image segmentation in med- plies under vacuum for 15 min at room temperature to ensure the
ical applications,17 cow health detection,18,19 and analysis of porous complete removal of all the entrapped air. Subsequently, the vacuum
microstructures.20 For example, Yu et al.20 analyzed the pore charac- and release films were detached, and the final bagging was started.
teristics using a fully convolutional neural network and investigated Generally, voids are absent in composites after curing owing to the
the relationship between the microscopic pore characteristics and the precompaction step used during the plate preparation process. Hence,
macroscopic permeability parameters of sandstone. Huang et al.21 in this study, to deliberately produce different voids and different void
applied DeepLabV3+ to map retrogressive thaw slumps automatically sizes, the three plates were exposed to the humidity before
22
from Planet CubeSat images. In addition, Baheti et al. proposed a manufacturing. The three moisture pretreatments are summarized in
modified DeepLabV3+ framework using dilated Xception as the back- Table 1. Subsequently, the plates were subjected to a pressure of
bone and achieved the best segmentation accuracy in an unstructured 0.4 MPa at a constant vacuum level of 0.1 MPa. The three plates were
environment. As a state-of-the-art neural network, the DeepLabV3+ subjected to the same curing cycle, wherein the plates were cured in
23
model is a novel encoder–decoder structure that employs an autoclave for a holding time of 1 h at a pre-curing temperature of
DeepLabV324 as a powerful encoder module and a simple, yet effec- 140 C, followed by a holding time of 2.5 h at a curing temperature
tive decoder module. In addition to the advanced networks, large- of 180 C.
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LUO ET AL. 1855
TABLE 1 Plates manufacturing conditions magnification and a large area of analysis. This ensures that the
Name Moisture treatment images fully cover the entire cross-section of the metallographic
samples.
Plate 1 90%/50 C/7 days
Plate 2 90%/50 C/2 days
Plate 3 None
2.3 | Interlaminar shear strength tests
F I G U R E 1 DeepLabV3+
network structure based on
Xception65 or MobileNetV2
skeletons
deep network training, and also acts as a regularizer to avoid over- low-level features, first, the lower feature maps in the network back-
fitting and to improve the generalizability of the model. In addition, bone were selected, after which they were transformed using a 1 × 1
the ReLU activation function alleviates the vanishing gradient issues convolution operation to reduce the number of channels. Finally, the
of deep network training, and operates faster than other alternatives target segmentation was completed using a 3 × 3 convolution opera-
such as sigmoid functions. At the encoder stage, atrous convolution tion and up-sampling by four times. The training parameters for the
was employed in the DeepLabV3+ model to extract the features com- modified DeepLabV3+ model on the manually labeled optical images
puted by the deep convolutional neural networks at an arbitrary reso- were those used by Chen et al.23 The pixel-wise softmax with cross-
lution. After this, the feature map was served as an input into the entropy loss was utilized as the loss function for the image segmenta-
atrous spatial pyramid pooling (ASPP) structure. Atrous convolution is tion process. For a given sample with label d, the softmax cross-
a powerful tool that enables the explicit control of the resolution of entropy loss was computed using Equation (3).18,29
the features computed by the deep convolutional neural networks
!
and the adjustment of the filter's field-of-view to capture multi-scale expðsd Þ
Lossðs, dÞ = −log PD , df1, 2,…,Dg, ð3Þ
information, and to generalize standard convolution operations. The j = 1 exp sj
ASPP module probes an incoming convolutional feature layer with
parallel filters at multiple sampling rates and effective fields of views, where D is the total number of classes and s is the predicted output from
thus capturing objects, as well as image context at multiple scales. The the model for all classes. The DeepLabV3+ model adopted a bac-
atrous rates used in the ASPP module of the original DeepLabV3+ kpropagation algorithm with a stochastic gradient searching, known as
model are 6, 12, and 18. However, these rates are more suitable for the Adam optimizer, for the training. The Adam optimizer is an advanced
large objects. As the dataset used in this work contained small objects optimization algorithm compared to the conventional stochastic gradient
with a microvoid diameter of only a few micrometers, lower atrous descent algorithm and produces a faster convergence. During each itera-
convolution rates were employed to enable a better analysis of the tion of the training process, the loss function calculated the error between
microvoids. The ASPP with lower atrous rates, namely modified ASPP, the predicted values and the true ground values. Subsequently, the Adam
consisted of a 1 × 1 convolution and three 3 × 3 atrous convolutions optimizer was used to propagate the error back through the entire neural
with rates of 4, 8, and 16, respectively. Subsequently, the output network to ensure that the weights are adjusted accordingly. Thus, the
structure of the ASPP was converted to a channel number using a loss function was minimized through the iterative training.
1 × 1 convolution layer. The feature map was inputted into the global
pooling layer for feature extraction, and the global average pooling
was used to obtain image-level features. At the decoder stage, first, 3.2 | Transfer learning strategy
the encoder features were bilinearly up-sampled by a factor of four,
and then they were concatenated with the corresponding low-level The DeepLabV3+ model can either be learned from scratch or fine-
features from the network backbone. To obtain the corresponding tuned from pre-trained models. Learning from scratch requires
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LUO ET AL. 1857
sufficiently large numbers of labeled optical images, as the mainstream adopted as the learning model to conduct data training and image
DeepLabV3+ architecture contains tens of millions of free parameters segmentation.
to train. To overcome these limitations, we adopted the transfer learn- A schematic of the pipeline of the image segmentation process is
ing strategy.25 In the transfer learning strategy, a network that has shown in Figure 2. As a data-driven machine learning method, deep
been pre-trained characteristically on images for a different problem learning is based on knowledge learning. In addition, data collection
was employed. Subsequently, the pre-trained network was fine-tuned, and annotation are essential, as the quantity and quality of the dataset
indicating that all or some of the weights were changed during the directly affect the final performance of the method. In this study, the
additional training process on the target dataset.17 Generally, the tar- data were collected using optical microscopy. Subsequently, the void
get dataset is significantly smaller than that used for pre-training. In boundaries were manually labeled to generate pixel-level annotation
this study, we selected a DeepLabV3+ network pre-trained on a data for the deep learning training using Labelme software. The pixel-
23
COCO dataset for the transfer learning, after which the pre-trained level annotation dataset included the original optical image and the
network was fine-tuned on our dataset. The transfer learning from a corresponding manual annotation figure after partial manual annota-
natural image dataset pre-trained model is beneficial, and particularly tion. The main purpose of these optical images and their annotation
prevents the overfitting of the DeepLabV3+ model on the limited was to produce the dataset required to extract the desired void fea-
optical image dataset. tures. To improve the generalization performance and reduce the
overfitting problem of the DeepLabV3+ model, one of the most effec-
tive methods is to add additional data in the training sets, which can
3.3 | Workflow for the semantic image be effectively achieved by image augmentation. Hence, image aug-
segmentation mentation methods including the image mirror, flip, and crop methods
were adopted to increase the image data. To achieve this, 40 optical
To build the neural network structure and create its dataset, the microscopic images with a pixel size of 2048 × 1536 representing the
Pytorch deep learning network used the environment of centos7.5/ characteristics of the three composite laminates were selected, and
CUDA8/cuDNN/Pycharm2020. The DeepLabV3+ model was each of the 40 original images was cut into 12 original data images
F I G U R E 2 Pipeline of the
image segmentation process
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1858 LUO ET AL.
with a pixel size of 512 × 512 to obtain a total of 480 images. After image file paths were set, and the maximum number of epochs and model
mirroring and flipping, a total of 1920 original images were obtained. saving frequency were set as 1000 and 200, respectively. Generally, the
To ensure that all the input attributes had the same scale, the image number of iterations is the criterion for stopping the training step. In addi-
data were normalized using a standardized method. Subsequently, the tion to the training iterations, the accuracy scores, losses, and MIoUs on
image data were divided into two batches: a training set and a testing the training set of the DeepLabV3+ networks with four different back-
set. The training set (36 original optical images) was used to train the bones (original Xception65, Xception65 with modified ASPP, MobileNet
deep learning model, whereas the testing set (four original optical with ASPP, and MobileNet without ASPP) are shown in Figure 3. As
images) was used to verify the effect of the training. The paths of the shown in the image, with an increase in the number of iterations, the
training and testing images were indicated in the files as train.txt and losses approached zero, whereas the accuracy scores and MIoUs
val.txt, respectively. Before training, the DeepLabV3+ pre-trained approached one. The corresponding testing set results are shown in
models were downloaded. The pre-processed original images and Table 2. The results revealed that the three DeepLabV3+ networks with
corresponding annotated images were set as the input data and fed the original Xception65, Xception65 with modified ASPP, and MobileNet
into the training model. The training model was used to train the with ASPP backbones exhibited a good image segmentation performance,
instances, learn the void characteristics, and adjust the network as indicated by their high MIoU values (>90%), whereas the DeepLabV3+
parameters of the model to obtain the fine-tuned model. In this study, network with the MobileNet without ASPP backbone exhibited poor
four different backbones: original Xception65, Xception65 with modi- image segmentation performance, as indicated by its low MIoU value
fied ASPP, MobileNetV2 with ASPP, and MobileNetV2 without ASPP, (approximately 78.91%). The comparison of the results of the two
were adopted to conduct the data training and image segmentation. DeepLabV3+ networks with the MobileNet with and without ASPP back-
The training process is described in Section 3.1. After the best fine- bones revealed that the network with the MobileNet with ASPP back-
tuned model was obtained, it was saved and selected as the well- bone exhibited better performance than those without ASPP, as ASPP
trained deep learning model for the new image interference. could capture voids of different sizes. In addition, ASPP probes an incom-
ing convolutional feature layer with filters at multiple sampling rates and
effective field of views, thus capturing the objects, as well as the image
3.4 | Evaluation metrics context at multiple scales. The comparison of the results of the three
DeepLabV3+ networks with the original Xception65, Xception65 with
To evaluate the performance of the deep learning model, the mean pixel modified ASPP, and MobileNet with ASPP backbones revealed that the
accuracy (MPA) and mean intersection over union (MIoU) of the testing two networks with the Xception65 backbones exhibited better perfor-
set were calculated.30 The pixel accuracy is the ratio of the number of mance than the network with the MobileNetV2 backbone. This is
correctly classified pixels to the total pixel number. Herein, we assumed because the MobileNetV2-based architecture is a lightweight model and
that the total category number was two, including one void class and one is more suitable for on-device semantic segmentation supplication,
background. Yii, Yij, and Yji denote the number of true positives, false posi- whereas Xception65-based architectures have been proven to be effec-
tives, and false negatives, respectively. MPA, the average of the per-class tive in image classification.28 This indicates that the DeepLabV3+ net-
pixel accuracy, can be calculated using Equation (4): works with the Xception65 architecture backbones exhibited a more
powerful generalization ability compared to the networks with the
1 X1 Y ii MobileNetV2-based architecture backbones. As shown in the table, the
MPA = P1 : ð4Þ
2 i=0
j = 0 Y ij Xception65 with modified ASPP backbone exhibits a slightly improved
performance compared to the original Xception65. As previously dis-
The MIoU denotes the rate between the intersection and union. cussed, ASPP with lower atrous convolution rates is more suitable for
MIoU, the average of the per-class intersection over union, can be cal- microvoid observation. Therefore, for the microvoid observation, the
culated using Equation (5): DeepLabV3+ network with the Xception65 with modified ASPP back-
bone was saved and selected as the well-trained deep learning model for
1 X1 Y ii the new image interference. The well-trained deep learning model
MIoU = P1 P1 : ð5Þ
j = 0 Y ji −Y ii
2 i=0
Y +
j=0 ij exhibited high MPA and MIoU values for the automatic detection of
voids.
4 | RESULTS AND DISCUSSION 4.2 | Effect of the void contents on the ILSS
4.1 | Void identification of the optical images 4.2.1 | C-scan ultrasonic inspection results
based on the modified DeepLabV3+ model
Figure 4 shows the C-scan results of the three composite laminates
After modifying the configuration file, the image crop size and data aug- fabricated by the autoclave processes at different pressures and mois-
mentation method were defined. In addition, the training and testing ture pretreatment conditions. An ultrasonic beam traveling through a
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LUO ET AL. 1859
F I G U R E 3 Accuracy scores, losses, and MIoUs based on the iteration of the DeepLabV3+ networks with different backbones: (A) original
Xception65; (B) Xception65 with modified ASPP; (C) MobileNet with ASPP; and (D) MobileNet without ASPP
T A B L E 2 DeepLabV3+ on the testing set with four different by the ultrasonic attenuation coefficients values. In addition, a large
network backbones high attenuation region was observed in the C-scan of plate 1, indicat-
ing the presence of a large quantity of dissolved moisture in the plate
Network backbone MIoU (%) MPA (%)
owing to the moisture pretreatment. However, a uniform attenuation
Original Xception65 90.75 99.84
was observed in the C-scan of plate 2, and the mean ultrasonic atten-
Xception65 with modified ASPP 90.82 99.84
uation coefficient of plate 2 was significantly lower than that of plate
MobileNet with ASPP 90.47 99.84
1. This indicates that the void content in plate 2 was significantly
MobileNet without ASPP 78.91 99.60 lower than that of plate 1, which could be attributed to the shorter
moisture pretreatment period of plate 2 (2 d) than that of plate
1 (7 d). The C-scan of plate 3, which was not subjected to moisture
plate may be attenuated because of the irregularities found inside the pretreatment, exhibited a uniform and low attenuation than those of
laminates, that is, void size, void distribution, or bad solidification the other plates subjected to moisture pretreatments. The ultrasonic
between layers. The ultrasonic attenuation coefficients of the plates C-scans results revealed that the void defects increased with an
were calculated using Equation (1). The mean ultrasonic attenuation increase in the moisture content. These results are consistent with
coefficients of plate 1, 2, and 3 were 5.69, 3.99, and 3.77, respec- those reported by Plessix et al.,12 who reported that moisture is pri-
tively. The C-scans were presented with a range of colors indicating marily responsible for the generation of pores in resin.
different levels of attenuation (in dB/mm), which indicates the distri-
bution and location of voids. The blue area represents a low attenua-
tion region, whereas the yellow area represents a high attenuation 4.2.2 | Void microstructure analysis using the well-
region. Previous studies have reported that the void content in com- trained model
posites increases with an increase in the ultrasonic absorption coeffi-
cient (dB/mm).31 Hence, C-scan ultrasonic inspection experiments After the C-scan ultrasonic inspection, 40 metallographic samples
were conducted to qualitatively verify the accuracy of the image were prepared from three square plates, and approximately 1100
analysis-based evaluation. The C-scan results revealed that plate optical images were obtained by the optical microscopy experiments.
1 had the highest number of voids, plate 2 had the second-highest Owing to the heterogenous attenuation level of the C-scan result of
void defects, and plate 3 had the lowest number of voids, as indicated plate 1, four additional metallographic samples were prepared for
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1860 LUO ET AL.
F I G U R E 4 C-scan ultrasonic
inspection of the three different
plates: (A) plate 1; (B) plate 2; and
(C) plate 3
plate 1. The well-trained deep learning model in Section 4.1 was used in Table 3, which reveals that the standard deviations between the
to automatically segment the optical images and identify the voids of two methods are small. This confirms the void identification accuracy
the composite laminates. Based on the size of the training images, of the deep learning-based image analysis method.
first, the optical images were cut into a pixel size of 512 × 512. Subse- Several studies have investigated the geometry of voids in com-
quently, the voids were automatically separated from the optical posite laminates, and reported that the void size ranged from a few
images of the composite laminates using the well-trained deep learn- μm to several mm.32,33 The size and shape of voids have large varia-
ing model. The voids images with a pixel size of 512 × 512 were tions within the same plate, as well as among plates. Therefore, it is
assembled into a large image with a pixel size of 2048 × 1536 based important to quantitatively analyze these variations using statistical
on the original image position and size. The black areas in Figure 5 tools. Theoretically, voids can be differentiated into interlaminar and
represent the voids of the laminates, whereas the white areas repre- intralaminar flaws. Interlaminar voids are located at the ply interfaces,
sent the composite materials. Compared to the original and inference whereas intralaminar voids are predominantly the trapped air located
images, the deep learning method accurately identified the voids. In inside the laminar region and the trapped gases located inside the sin-
addition, the well-trained deep learning model automatically identi- gle yarn. In this study, voids smaller than 3 μm in diameter, or 7 μm2 in
fied, segmented, and processed the voids in the 1100 optical images. the area were considered too small to resolve.34 In addition, the pores
Subsequently, the average void contents of plates 1, 2, and 3 were inside a single yarn were too small when compared with the fiber with
calculated, and were 3.14%, 0.34%, and 0.07%, respectively. The a diameter of 7 μm, and their influence on the mechanical perfor-
trend in the void contents of the three plates is consistent with the C- mance of composite parts is negligible; thus, they were not consid-
scan results. ered. Hence, in this study, the intralaminar voids represent the
As reported by Little et al.,32 the discrepancy between the optical trapped air located inside the laminar region. To understand the void
microscopy and image analysis method may lead to a very small error size distribution, two void size categories: microvoids (0–-
in the overall porosity volume fraction. Furthermore, in addition to 6.0 × 10 μm ) and macrovoids (>6.0 × 10 μm ) were separated.
3 2 3 2
the C-scan ultrasonic inspection experiments, which were performed Based on the deep learning inference results, the statistical results of
to qualitatively verify the accuracy of the image analysis-based evalu- the void characteristics in composites were automatically obtained
ation, the image analysis methods were also compared. To verify the using a Python script, as shown in Figure 6. Microvoids smaller than
accuracy of the well-trained model, a conventional thresholding tech- 100 μm2, which refers to microvoids with a diameter smaller than
nique was employed to identify the void contents. The void content 11.29 μm, were also characterized in detail, as shown in Figure 6(A,C,
results of the three different plates using the two methods are listed E). As shown in Figure 6, the range of the void areas of the three
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1861
Standard deviation
0.02%
0.02%
0.03%
Void content: thresholding method
3.17%
0.37%
0.11%
Void content: deep learning method
Void content results of different plates
plates: (A) plate 1; (B) plate 2; and
3.14%
0.34%
0.07%
micrographs and deep learning
inference results of the three
F I G U R E 5 Typical
TABLE 3
(C) plate 3
LUO ET AL.
Plate 1
Plate 2
Plate 3
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1862 LUO ET AL.
plates ranged from 7 to 20,000 μm2. In addition, the size distribution relative humidity for 7 d. For plate 2, stacks of 8 plies were humidity-
of the micro and macrovoids of the three laminates was different. The conditioned at 50 C and relative humidity of 90% for 2 d. In contrast,
number of micro and macrovoids in plate 1 were higher than those in plate 3 was not subjected to humidity pretreatment. As shown in Fig-
plate 2, whereas plate 3 had no macrovoids. ures 5(A) and 6(B), several macrovoids were longer than 6000 μm2,
These differences could be attributed to the effect of moisture and the voids occur mostly at the ply interfaces. In addition, most of
pretreatment on the plates. As discussed in Section 2.1, for plate the voids, particularly the larger ones, did not exhibit a circular cross-
1, stacks of 8 plies were humidity-conditioned at 50 C and 90% section. Furthermore, most macrovoids exhibited an elongated
F I G U R E 6 Void area distribution: (A) plate 1 – microvoids; (B) plate 1 – macrovoids; (C) plate 2 – microvoids; (D) plate 2 – macrovoids; and
(E) plate 3 – microvoids
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LUO ET AL. 1863
cylindrical, cigar-shaped pattern running along the fiber direction. As spherical pattern, whereas the macrovoids exhibited a cylindrical
shown in Figures 5(B) and 6(D), some macrovoids were longer than shape. In addition, the size of the voids increased gradually with
6000 μm and exhibit an elliptic pattern. These observations are con-
2
increasing void content. Previous studies have demonstrated that
sistent with those reported by Mehdikhani et al.35 that voids appear the ILSS of composites is highly vulnerable to the void content and
mainly as elongated features. This pattern could be attributed to the shape. In this study, we observed that there were two possible void
void formation mechanisms of the plates.9-12,33 During the autoclave configurations in composites, namely cylindrical and spherical. Con-
process, most voids were formed by the microcavities that trapped air sequently, two theoretically derived equations for the cylindrical
at the ply interfaces during the manufacturing process or by the evap- and spherical voids were used to calculate the ILSS of the composite
oration of moisture and volatiles inside the prepregs during the stor- with voids.4,36
age and fabrication processes.11,12 Among these three possibilities,
" 12 #
this study focused on the generation of voids by the evaporation of 4V v
τcylindrical void ðV v , V f Þ = τvoid free 1 − , ð6Þ
moisture because the effect of volatiles can be ignored, and the effect π ð1 −V f Þ
of air is negligible owing to the precompaction step used during the
" 23 #
plate preparation. With an increase in the processing time, the mois- π 6V v
τspherical void ðV v , V f Þ = τvoid free 1− , ð7Þ
ture content increased until the maximum moisture content was 4 π ð1 −V f Þ
reached, thus leading to the generation of microcavities. During the
preparation process of the plates, the vacuum and applied pressure
cause air, vapor, and volatiles to more easily diffuse together, if they where τvoid free is the theoretical ILSS of the void-free composite, and
are nearby, rather than the nucleation of voids. After the initial void Vv and Vf are the void contents and fiber volume fraction, respec-
nucleation, the size and shape microcavities were transformed, owing tively. To obtain the theoretical ILSS of the void-free composite, a
to the surrounding and internal pressure and resin flow. Void expan- square plate (100 × 100 mm2) was prepared without moisture pre-
sion encounters the least resistance along the fiber direction, whereas treatment by autoclave processing under a curing pressure of
it is constrained by the stiff fibers in other directions. During the auto- 0.6 MPa, and the other conditions are shown in Section 2.1. Subse-
clave process, the largest expansion resistance occurred in the lami- quently, the ILSS of the void-free plate was measured according to
nate thickness direction because the autoclave pressure was applied ASTM D2344, as shown in Section 2.3. Optical microscopy experi-
from the top of the laminate. As shown in Figures 5(C) and 6(E), no ments were also conducted, and the results revealed that the void
macrovoids can be observed in plate 3, however, microvoids are per- content of the plate was lower than 0.05%, indicating that the com-
vasive. In addition, the voids in plate 3 exhibited irregular circular posite is a void-free composite. The experimental results revealed that
shapes and were randomly distributed. This phenomenon could be the τvoid free was 79.63 MPa.
attributed to the lack of sufficient microcavities in plate 3, which was
subjected to a precompaction step and without moisture pre-
treatment. These observations indicate that macrovoids are typically 4.3 | Calculation results and experimental
located at the intralaminar places in carbon/epoxy laminates, whereas validations
the microvoids are randomly distributed. In addition, these results
indicate that the void content increased with an increase in the mois- The experimentally measured ILSS of the composites and the respec-
ture content. This observation of the effect of moisture on the void tive predicted ILSS values of the models are listed in Table 4. As
content is consistent with those of Plessix et al.12 that an increase in shown in the table, the ILSS values of the composites predicted by
moisture content increases the void content. the cylindrical void shape model are lower compared to the values
that are predicted by the spherical void shape model. This indicates
that the cylindrical void shape model was more suitable for plate
4.2.3 | ILSS determination by void parameter 2 because it mainly contained macrovoids with elliptic patterns. In
addition, the spherical void shape model was more suitable for plate
As discussed above, the ILSS of composites is highly vulnerable to 3 as it only had circular-shaped microvoids. For plate 1, the errors of
void content and shape. As shown in Figure 5, some voids were both the cylindrical void shape model and spherical void shape model
located at the ply interface of the laminates, and some voids were were higher than 5%, which could be attributed to the fact that both
located in the plies of the laminates; the microvoids exhibited a the micro and macrovoids content was high. In addition to the void
Experimental: ILSS (MPa) Calculation: cylindrical void ILSS (MPa) Error Calculation: spherical void ILSS (MPa) Error
Plate 1 57.44 51.71 8.23% 60.34 5.05%
Plate 2 71.75 70.77 1.37% 75.25 4.88%
Plate 3 79.22 75.61 4.56% 78.10 1.41%
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1864 LUO ET AL.
shape, the void distribution in the composite laminates also varied. DATA AVAILABILITY STAT EMEN T
Voids have previously been considered as damage precursors, The data that support the findings of this study are available from the
because they serve as damage initiation sites in composites. Inter- corresponding author upon reasonable request.
laminar voids cause fracture between the layers, whereas intralaminar
voids cause through-thickness fractures among the fibers. As shown OR CID
in Table 4, the ILSS values of the carbon/epoxy unidirectional lami- Ling Luo https://orcid.org/0000-0002-5071-1260
nates decrease by approximately 27% with an increase in the void
content from 0.07% to 3.14%. This experimental observation of the RE FE RE NCE S
effect of voids on the ILSS value is consistent with those of Costa 1. Costa ML, Almeida SFM, Rezende MC. The influence of porosity on the
et al.1 that the ILSS values of carbon/epoxy fabric laminates substan- interlaminar shear strength of carbon/epoxy and carbon/bismaleimide
fabric laminates. Compos Sci Technol. 2001;61:2101-2108.
tially decrease (approximately 34%) with an increase in the void con-
2. Zhu HY, Li DH, Zhang DX, Wu BC, Chen YY. Influence of voids on
tent from 0.55% to 5.6%. interlaminar shear strength of carbon/epoxy fabric laminates. Trans
Nonferr Metal Soc. 2009;19:470-475.
3. Hernandez S, Sket F, Molina-Aldareguıa JM, Gonzalez C, LLorca J.
Effect of curing cycle on void distribution and interlaminar shear
5 | C O N CL U S I O N S
strength in polymer-matrix composites. Compos Sci Technol. 2011;71:
1331-1341.
In summary, the relationship between voids and the ILSS values of 4. Meier R, Kahraman I, Seyhan AT, Zaremba S, Drechsler K. Evaluating
polymer-matrix composites was investigated using optical microscopy vibration assisted vacuum infusion processing of hexagonal boron
nitride sheet modified carbon fabric/epoxy composites in terms of
experiments and deep learning methodologies. The microscopy
interlaminar shear strength and void content. Compos Sci Technol.
images obtained during optical microscopy were used to automatically 2016;128:94-103.
identify the voids through semantic image segmentation based on 5. Yang X, Zhan L, Jiang C, Zhao X, Guan C, Chang T. Evaluating random
artificial intelligence, and the statistical results of the void characteris- vibration assisted vacuum processing of carbon/epoxy composites in
tics in the composites were automatically identified. Subsequently, terms of interlaminar shear strength and porosity. J Compos Mater.
2019;53:1-10.
the ILSSs values of the polymer-matrix composites were calculated
6. Liebig WV, Viets C, Schulte K, Fiedler B. Influence of voids on the
according to a theoretically derived equation and were experimentally compressive failure behaviour of fibre-reinforced composites. Compos
validated. The following conclusions were made: Sci Technol. 2015;117:225-233.
7. Hapke J, Gehrig F, Huber N, Schulte K, Lilleodden ET. Compressive
failure of UD-CFRP containing void defects: in situ SEM microanaly-
1. The deep learning semantic image segmentation method was suit-
sis. Compos Sci Technol. 2011;71:1242-1249.
able for void identification in the optical images of the composites. 8. Chao X, Qi L, Tian W, Hou X, Ma W, Li H. Numerical evaluation of
The void recognition model based on a DeepLabV3+ model effec- the influence of porosity on bending properties of 2D carbon/carbon
tively and automatically identified the void characteristics in the composites. Compos Part B-Eng. 2018;136:72-80.
9. Loos AC, Springer GS. Curing of epoxy matrix composites. J Compos
optical images.
Mater. 1983;17:135-169.
2. The artificial intelligence semantic image segmentation method 10. Kardos JL, Dudukovic MP, Dave R. Void growth and resin transport
accurately identified the voids in the composites, and the ILSS during processing of thermosetting-matrix composites. Adv Polym Sci.
values of the composites were calculated using theoretically 1986;80:102-123.
11. Ledru Y, Bernhart G, Piquet R, Schmidt F, Michel L. Coupled visco-
derived equations. In addition, compared to the experimental
mechanical and diffusion void growth modelling during composite
results, the ILSS values calculated by the deep learning image seg- curing. Compos Sci Technol. 2010;70:2139-2145.
mentation method were closer to the testing results. 12. Plessix BDPD, Corre SL, Jacquemin F, Lefebure P, Sobotka V.
3. The void content increased with an increase in the moisture con- Improved simplified approach for the prediction of porosity growth
during the curing of composites parts. Compos Part A Appl Sci Manuf.
tent, thus decreasing the ILSS values of the composites.
2016;90:549-558.
4. The deep learning image segmentation method effectively
13. Tretiak I, Smith RA. A parametric study of segmentation thresholds
improved the accuracy and efficiency of void identification in opti- for X-ray CT porosity characterisation in composite materials. Compos
cal images. In addition, the deep learning method also achieved the Part A Appl Sci Manuf. 2019;123:10-24.
automatic processing of void identification in the optical images of 14. Navas VMT, Buljac A, Hild F, et al. A comparative study of image seg-
mentation methods for micromechanical simulations of ductile dam-
the composites.
age. Comput Mater Sci. 2019;159:43-65.
15. Madra A, Hajj NE, Benzeggagh M. X-ray microtomography applications
ACKNOWLEDGMENTS for quantitative and qualitative analysis of porosity in woven glass fiber
This work was supported by the National Key R&D Program of China reinforced thermoplastic. Compos Sci Technol. 2014;95:50-58.
16. Yu ZX, Wei SC, Zhang JW, et al. High-throughput, algorithmic deter-
[grant numbers 2017YFB0703300]; and the National Natural Science
mination of pore parameters from electron microscopy. Comput Mater
Foundation of China [grant numbers 11872086]. Sci. 2020;171:109216.
17. Zhu Z, Albadawy E, Saha A, Zhang J, Harowicz MR, Mazurowski MA.
CONF LICT OF IN TE RE ST Deep learning for identifying radiogenomic associations in breast can-
cer. Comput Biol Med. 2019;109:85-90.
The authors declare no potential conflict of interest.
10991581, 2021, 4, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/pat.5226 by University Of Liverpool, Wiley Online Library on [09/07/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
LUO ET AL. 1865
18. Achour B, Belkadi M, Filali I, Laghrouche M, Lahdir M. Image analysis 29. Luo L, Zhang B, Zhang G, Xu Y. Rapid prediction of cured shape types
for individual identification and feeding behaviour monitoring of dairy of composite laminates using a FEM-ANN method. Compos Struct.
cows based on convolutional neural networks (CNN). Biosyst Eng. 2020;238:111980.
2020;198:31-49. 30. Hao S, Zhou Y, Guo Y. A brief survey on semantic segmentation with
19. Wu D, Yin X, Jiang B, Jiang M, Li Z, Song H. Detection of the respira- deep learning. Neurocomputing. 2020;406:302-321.
tory rate of standing cows by combining the Deeplab V3+ semantic 31. Saenz-Castillo D, Martín MI, Calvo S, Rodriguez-Lence F, Güemes A.
segmentation model with the phase-based video magnification algo- Effect of processing parameters and void content on mechanical
rithm. Biosyst Eng. 2020;192:72-89. properties and NDI of thermoplastic composites. Compos Part A Appl
20. Yu Q, Xiong Z, Du C, et al. Identification of rock pore structures and Sci Manuf. 2019;121:308-320.
permeabilities using electron microscopy experiments and deep learn- 32. Little JE, Yuan X, Jones MI. Characterisation of voids in fibre
ing interpretations. Fuel. 2020;268:117416. reinforced composite materials. NDT&E Int. 2012;46:122-127.
21. Huang L, Luo J, Lin Z, Niu F, Liu L. Using deep learning to map retro- 33. Huang H, Talreja R. Effects of void geometry on elastic properties of
gressive thaw slumps in the Beiluhe region (Tibetan plateau) from unidirectional fiber reinforced composites. Compos Sci Technol. 2005;
CubeSat images. Remote Sens Environ. 2020;237:111534. 65:1964-1981.
22. Baheti B, Innani S, Gajre S, Talbar S. Semantic scene segmentation in 34. Fritz NK, Kopp R, Nason AK, et al. New interlaminar features and void
unstructured environment with modified DeepLabV3+. Pattern distributions in advanced aerospace-grade composites revealed via
Recogn Lett. 2020;138:223-229. automated algorithms using micro-computed tomography. Compos
23. Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder Sci Technol. 2020;193:108132.
with atrous separable convolution for semantic image segmentation. 35. Mehdikhani M, Straumit I, Gorbatikh L, Lomov SV. Detailed charac-
Paper presented at: Proceedings of ECCV Conference. 2018. terization of voids in multidirectional carbon fiber/epoxy composite
p. 833–851. laminates using X-ray micro-computed tomography. Compos Part A
24. Chen LC, Papandreou G, Schroff F, Adam H. Rethinking atrous Appl Sci Manuf. 2019;125:105532.
convolution for semantic image segmentation. CoRR. 2017. arXiv: 36. Bowles KJ, Frimpong S. Voids effects on the interlaminar shear
1706.05587. strength of unidirectional graphite-fiber-reinforced composites.
25. Shin HC, Roth HR, Gao M, et al. Deep convolutional neural networks J Compos Mater. 1992;26:1487-1509.
for computer-aided detection: CNN architectures, dataset character-
istics and transfer learning. IEEE Trans Med Imaging. 2016;35(5):1285-
1298.
26. Hsu DK, Uhl KM. A morphological study of porosity defects in How to cite this article: Luo L, Zhang B, Lei Y, et al.
graphite-epoxy composites. Review of Progress in Quantitative Nonde-
Identification of voids and interlaminar shear strengths of
structive Evaluation. Vol 6B. Boston: Springer; 1987:1175-1184.
27. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC. Mobilenetv2: polymer-matrix composites by optical microscopy experiment
inverted residuals and linear bottlenecks. Paper presented at: Pro- and deep learning methodology. Polym Adv Technol. 2021;32:
ceedings of CVRP Conference. 2018. p.4510–4520. 1853–1865. https://doi.org/10.1002/pat.5226
28. Chollet F. Xception: deep learning with depthwise separable convolu-
tions. Paper presented at: Proceedings of CVRP Conference. 2017.
p.1800–1807.