PERFORMANCE EVALUATION OF MACHINE LEARNING

ALGORITHMS FOR REGRESSION OF FINITE ELEMENT ANALYSIS

(FEA) SIMULATION DESIGN DATA

Victor Leão da Silva Diasa, Lilian Lefol Nani Guarieirob, Erick Giovani Sperandio
Nascimentob
a
, Mestrado em Modelagem Computacional e Tecnologia Industrial, Centro
Universitário SENAI CIMATEC, Brazil
b
Centro Universitário SENAI CIMATEC, Bazil

Abstract: The process of developing the design for new parts and structures through
FE simulations demands significant human and computational effort. By employing the
multi-objective optimization method using DOE and metamodels, it is possible to
achieve optimal design parameters faster and with greater precision. Thus, this study
assessed the efficiency of using Machine Learning as metamodels to represent the
behavior of FE models. Conventional methods were trained with and without data
normalization and standardization, employing cross-validation and hyperparameter
tuning. Ultimately, this analysis provides the best models for different types of design
data, making their utilization viable in certain cases.
Keywords: machine learning; multi-objective optimization; finite element analysis.

AVALIAÇÃO DO DESEMPENHO DE ALGORITMOS DE MACHINE

LEARNING PARA REGRESSÃO DOS DADOS DE DESIGN DE
SIMULAÇÃO DE ANÁLISE DE ELEMENTOS FINITOS (FEA)

Resumo: O processo de desenvolvimento do design de novas peças e/ou estruturas

através de simulações FE requer muito esforço humano e computacional. A utilização
do método de otimização multi-objetivo através do uso do DOE e de metamodelos é
possível obter resultados de parâmetros de design ótimos mais rápido e com maior
precisão. Assim, este trabalho avaliou a eficiência da utilização de Machine Learning
como metamodelos para representar o comportamento dos modelos FE. Os
treinamentos dos métodos convencionais, sem e com normalização e padronização
dos dados, foram feitos utilizando validação cruzada e o ajuste dos hyperparâmetros.
Finalmente, está análise fornece os melhores modelos para diferentes tipos de dados
de design, sendo viável a sua utilização em alguns casos.
Palavras-chave: aprendizado de máquina; otimização multi-objetivo; análise de
elementos finitos.

ISSN: 2357-7592
IX INTERNATIONAL SYMPOSIUM ON INNOVATION AND TECHNOLOGY
Engineering and the Future of the Industry - 2023
1
1. INTRODUCTION

The process of developing new parts and/or structures in engineering has

undergone significant advancements due to the increase in computational power.
However, in some cases, there are still challenges related to the high complexity of the
relationship between parameters and performance metrics. Typically, the development
process involves successive iterations of design creation, simulation, and analysis,
requiring significant efforts from the engineer and computational resources until
optimal metrics are achieved with parameters within the constraints [1]. As a result,
some methods have emerged to expedite and facilitate this process, among which
Finite Element Analysis (FEA) combined with an optimization algorithm stands out for
its higher accuracy [2,3].
However, when dealing with the design modeling of new parts using FEA, there
is a relationship between numerous parameters and objectives, making it a multi-
objective optimization problem. In such cases, the goal is to simultaneously minimize
or maximize the values of the objective equations by altering the parameters to find
the set of optimal solutions, known as the Pareto Front [4]. There are two ways to
optimize the design variables: by combining the optimization algorithm with a direct
design method or with metamodels resulting from the Design of Experiments (DOE).
Metamodels are mathematical models capable of approximating the relationships
between variables and responses. As for DOE, it is a statistical application used to
design experiments and analyze the results to identify relationships between design
variables and responses. Through multi-objective optimization by creating metamodels
from DOE, it is possible to obtain optimal results while analyzing the sensitivity between
parameters and responses of the Finite Element (FE) model [3,5].
Recently, there have been many studies on multi-objective optimization using
metamodels. Traditional statistical models such as linear regression, logistic
regression, and polynomial regression can be used as metamodels, yielding good
results for simpler systems. However, they have shown limitations in handling more
complex systems with large amounts of data, parameters, and nonlinear relationships.
In this context, Machine Learning (ML) comes into play to overcome these limitations,
as it demonstrates better performance due to its adaptability features [3,6]. Therefore,
this study aims to evaluate the performance of conventional ML techniques as
metamodels for design variables with more than one objective response.

2. METHODOLOGY

For this study, the machine learning algorithms were modeled, and their
accuracy was statistically evaluated. The methodology was divided into four main
steps: data acquisition and pre-processing, algorithm modeling and normalization,
model training and hyperparameter tuning, and statistical analysis of model
performance.
The data used in this study was extracted from the work of Shui et al. 2018,
which involves multi-objective optimization of design parameters for an electric vehicle
battery enclosure. The ANSYS software was utilized for basic design modeling and
finite element analysis to obtain the design data and responses to variations in the
ISSN: 2357-7592
IX INTERNATIONAL SYMPOSIUM ON INNOVATION AND TECHNOLOGY
Engineering and the Future of the Industry - 2023
2
dimensions of the component. Design of Experiments (DoE) was employed,
specifically the Central Composite Design (CCD) method, to define 100 different
simulation samples, varying the design parameters (battery box thickness (EW),
bottom box thickness (EB), module bottom thickness (bb), battery module long wall
thickness (bwl), battery module wide wall thickness (bww), and ambient temperature)
and determining the maximum deformation (maxdef), minimum natural frequency
(minfreq), and mass (mass) [7].
For the prediction of the aforementioned metrics, conventional regression
machine learning algorithms were employed. These included Support Vector
Regression (SVR), K-Nearest Neighbors Regressor (KNN), Random Forest Regressor
(RFR), Gradient Boosting Regressor (GBR), Histogram Gradient Boosting Regression
(HGBR), AdaBoost Regressor (ABR), Ridge Regressor (RDG), and Least Angle
Regression (LARS). In addition to using these regression algorithms, various data
normalization and standardization methods applicable to numerical data were applied.
These methods include Normalizer, MinMaxScaler, StandardScaler, and
RobustScaler, along with evaluating the algorithms without any normalization. These
techniques are used to ensure that the input data is within a specific range or
distribution, which can aid in improving the performance of the machine learning
models and their predictions.
The database was divided into 75% for training and 25% for testing. With the
training set, a 5-fold cross-validation was applied to enhance the robustness and
effectiveness of the models. Hyperparameter tuning was performed for all models to
optimize their performance. After the training and identification of the best estimators,
model evaluation was conducted using the remaining 25% of the database. The
evaluation metrics used to assess the accuracy were the Mean Absolute Error (MAE),
Root Mean Squared Error (RMSE), and the Coefficient of Determination (R2). Figure
1 illustrates the structure of the proposed method, depicting the entire workflow of data
division, model training, cross-validation, hyperparameter tuning, and final model
evaluation.

ISSN: 2357-7592
IX INTERNATIONAL SYMPOSIUM ON INNOVATION AND TECHNOLOGY
Engineering and the Future of the Industry - 2023
3
 Figure 1. Flowchart of the proposed methodology

3. RESULTS AND DISCUSSION

With the training and hyperparameter tuning completed, the R2 of the best
models obtained for each analyzed output was calculated. The results for maximum
deformation (maxdef), minimum natural frequency (minfreq), and mass (mass) are
presented in Tables 1, 2, and 3, respectively.

Table 1. Best scores “maxdef”

In maximum deformation, it is observed that none of the models performed

adequately, likely due to the models' incapacity to handle the relationship between the
ISSN: 2357-7592
IX INTERNATIONAL SYMPOSIUM ON INNOVATION AND TECHNOLOGY
Engineering and the Future of the Industry - 2023
4
design parameters of the component and deformation. The work of Li Shui
demonstrated an 85% contribution of the parameters to this objective. Notably, the
SVR model performed poorly, as seen in Figure 2 below, for the SVR model without
normalization. The low MAE and RMSE values are correlated with the extremely low
variation between the maximum and minimum values. Meanwhile, the GBR model
achieved the best results, but still with an unacceptable performance.

Figure 2. Prediction of "maxdef" with the SVR model without normalization

Table 2. Best scores “minfreq”

In the minimum natural frequency models, a significant improvement in the best

scores is evident, with the SVR, RDG, and LARS models standing out positively. These
models demonstrate the capability to generalize with small deviations from the actual
values, as shown in Figure 3.

ISSN: 2357-7592
IX INTERNATIONAL SYMPOSIUM ON INNOVATION AND TECHNOLOGY
Engineering and the Future of the Industry - 2023
5
 Figure 3. Prediction of "minfreq" with the SVR model without normalization

Table 3. Best scores “mass”

The best objective for which the models showed the best fit was the mass, as
seen in Figure 4, with R2 values above 0.71. The models' better adaptation to the mass
is related to its direct and linear relationship with the dimensions of the component.
Since mass is directly influenced by the dimensions of the design parameters, the
models were able to capture this relationship effectively, resulting in higher R2 values
and better predictive performance for the mass objective.

Figure 4. Prediction of "mass" with the SVR model without normalization

In addition to the presented results, there are two important observations to be

made. First, the Normalizer method showed poor results in all analyzed objectives and

ISSN: 2357-7592
IX INTERNATIONAL SYMPOSIUM ON INNOVATION AND TECHNOLOGY
Engineering and the Future of the Industry - 2023
6
models, as it normalizes each sample individually, which may not be suitable for the
data distribution and relationships within the dataset. Second, the K-Nearest Neighbors
(KNN) method without normalization also had poor results without any plausible
justification.

4. CONCLUSION

The study aimed to analyze conventional machine learning (ML) methods with
various data normalization and standardization techniques, in addition to
hyperparameter tuning. For this purpose, exploratory training was performed using
preprocessed data, followed by testing with previously unused data to evaluate the
effectiveness of the suggested models. The results obtained were satisfactory for the
given dataset. It is recommended to use the Support Vector Regression (SVR) model
without normalization for the objectives "minfreq" and "mass" due to its high accuracy
and fast training and prediction capabilities. Additionally, the Gradient Boosting
Regressor (GBR) model is suggested with any normalization technique, except for the
Normalizer method, for the "mass" objective.

5. REFERENCES

1
VARDHN, H., SZTIPANOVITS, J. Deep Learning based FEA Surrogate for Sub-
Sea Pressure Vessel. 6th International Conference on Computer, Software and
Modeling (ICCSM), p. 36-39, 2022.
2
SONG, T., ZHANG, Z., LIU, H. and HU, W., Multi-objective optimisation design
and performance comparison of permanent magnet synchronous motor for EVs
based on FEA. IET Electric Power Applications, 13, p. 1157-1166, 2019.
3
VON WYSOCKI, T., RIEGER, F., TSOKAKTSIDIS, D.E., GAUTERIN, F. Generating
Component Designs for an Improved NVH Performance by Using an Artificial
Neural Network as an Optimization Metamodel. Designs, 5, 36, 2021.
4
DÍAZ, N. J. G. Algoritmo de Otimização Multi-Objetivo Assistida por
Metamodelagem com Aplicações em Problemas de Aerodinâmica. Tese –
Térmica, Fluidos e Máquinas de Fluxo, Instituto de Engenharia Mecânica,
Universidade Federal de Itajubá, 2020.
5
YOU, Y.-m. Multi-Objective Optimal Design of Permanent Magnet Synchronous
Motor for Electric Vehicle Based on Deep Learning. Appl. Sci., 10, 482, 2020.
6
ULLAH, I., YAMAMOTO, T., AL MAMLOOK, R., JAMAL, A., LIU, K. A comparative
performance of machine learning algorithm to predict electric vehicles energy
consumption: A path towards sustainability. Energy & Environment, 33, p. 1583–
1612, 2021.
7
SHUI, L., CHEN, F., GARG, A. et al. Design optimization of battery pack
enclosure for electric vehicle. Struct Multidisc Optim, 58, p. 331–347, 2018.

ISSN: 2357-7592
IX INTERNATIONAL SYMPOSIUM ON INNOVATION AND TECHNOLOGY
Engineering and the Future of the Industry - 2023
7