Acta Mechanica Sinica (2021) 37(12):1718–1726

https://doi.org/10.1007/s10409-021-01143-6

INVITED REVIEW

Applying machine learning to study fluid mechanics

Steven L. Brunton1

Received: 24 August 2021 / Revised: 8 September 2021 / Accepted: 16 September 2021 / Published online: 4 January 2022
© The Author(s) 2021

Abstract
This paper provides a short overview of how to use machine learning to build data-driven models in fluid mechanics. The
process of machine learning is broken down into five stages: (1) formulating a problem to model, (2) collecting and curating
training data to inform the model, (3) choosing an architecture with which to represent the model, (4) designing a loss function
to assess the performance of the model, and (5) selecting and implementing an optimization algorithm to train the model. At
each stage, we discuss how prior physical knowledge may be embedding into the process, with specific examples from the
field of fluid mechanics.

Keywords Machine learning · Fluid mechanics · Physics-informed machine learning · Neural networks · Deep learning

1 Introduction ing model to manipulate the behavior of the fluid for some
engineering objective with active control [2–4].
The field of fluid mechanics is rich with data and rife with In either case, it is important to realize that machine learn-
problems, which is to say that it is a perfect playground for ing is not an automatic or turn-key procedure for extracting
machine learning. Machine learning is the art of building models from data. Instead, it requires expert human guidance
models from data using optimization and regression algo- at every stage of the process, from deciding on the problem,
rithms. Many of the challenges in fluid mechanics may be to collecting and curating data that might inform the model,
posed as optimization problems, such designing a wing to to selecting the machine learning architecture best capable
maximize lift while minimizing drag at cruise velocities, esti- of representing or modeling the data, to designing custom
mating a flow field from limited measurements, controlling loss functions to quantify performance and guide the opti-
turbulence for mixing enhancement in a chemical plant or mization, to implementing specific optimization algorithms
drag reduction behind a vehicle, among myriad others. These to train the machine learning model to minimize the loss
optimization tasks fit well with machine learning algorithms, function over the data. A better name for machine learning
which are designed to handle nonlinear and high-dimensional might be “expert humans teaching machines how to learn a
problems. In fact, machine learning and fluid mechanics model to fit some data,” although this is not as catchy. Par-
both tend to rely on the same assumption that there are pat- ticularly skilled (or lucky!) experts may design a learner or
terns that can be exploited, even in high-dimensional systems a learning framework that is capable of learning a variety
[1]. Often, the machine learning algorithm will model some of tasks, generalizing beyond the training data, and mim-
aspect of the fluid, such as the lift profile given a particular icking other aspects of intelligence. However, such artificial
airfoil geometry, providing a surrogate that may be optimized intelligence is rare, even more so than human intelligence.
over. Machine learning may also be used to directly solve the The majority of machine learning models are just that, mod-
fluid optimization task, such as designing a machine learn- els, which should fit directly into the decades old practice of
model-based design, optimization, and control [5].
With its unprecedented success on many challenging prob-
Executive Editor: Bernd Noack. lems in computer vision and natural language processing,
machine learning is rapidly entering the physical sciences,
B Steven L. Brunton and fluid mechanics is no exception. The simultaneous
[email protected]
promise, and over-promise, of machine learning is causing
1 Department of Mechanical Engineering, University of many researchers to have a healthy mixture of optimism and
Washington, Seattle, WA 98195, USA

123
Applying machine learning to study fluid mechanics 1719

skepticism. In both cases, there is a strong desire to under-

stand the uses and limitations of machine learning, as well as
best practices for how to incorporate it into existing research
and development workflows. It is also important to realize
that while it is now relatively simple to train a machine learn-
ing model for a well-defined task, it is still quite difficult to
create a new model that outperforms traditional numerical
algorithms and physics-based models. Incorporating par-
tially known physics into the machine learning pipeline well
tend to improve model generalization and improve inter-
pretability and explainability, which are key elements of Fig. 1 Schematic of the five stages of machine learning on an example
of reduced-order modeling. In this case, the goal is to learn a low dimen-
modern machine learning [6,7].
sional coordinate system z = f 1 (x, θ 1 ) from data in a high-dimensional
representation x, along with a dynamical system model ż = f 2 (z, θ 2 )
for how the state z evolves in time. Finally, this latent state derivative ż
2 Physics informed machine learning for must be able to approximate the high dimensional derivative ẋ through
the decoder ẋ ≈ f 3 (ż, θ 3 ). The loss function L (θ, X) defines how well
fluid mechanics the model performs, averaged over the data X. Finally, the parameters
θ = {θ 1 , θ 2 , θ 3 } are found through optimization
Applied machine learning may be separated into a few canon-
ical steps, each of which provides an opportunity to embed
prior physical knowledge: (1) choosing the problem to model tem to model, motivated by a problem that is both important
or the question to answer; (2) choosing and curating the data and tractable. Choosing a problem involves deciding on input
used to train the model; (3) deciding on a machine learning data that will be readily available in the future, and output
architecture to best represent or model this data; (4) design- data that will represent the desired output, or prediction, of
ing loss functions to quantify performance and to guide the the model. The output data should be determinable from the
learning process; and (5) implementing an optimization algo- inputs, and the functional relationship between these is pre-
rithm to train the model to minimize the loss function over cisely what the machine learning model will be trained to
the training data. See Fig. 1 for a schematic of this process capture.
on the example of reduced-order modeling. This organiza- The nature of the problem, specifically what outputs will
tion of steps is only approximate, and there are considerable be modeled given what inputs, determines the large classes of
overlaps and tight interconnections between each stage. For machine learning algorithms: supervised, unsupervised, and
example, choosing the problem to model and choosing the reinforcement learning. In supervised learning, the training
data to inform this model are two closely related decisions. data will have expert labels that should be predicted or mod-
Similarly, designing a custom loss function and implement- eled with the machine learning algorithm. These output labels
ing an optimization algorithm to minimize this loss function may be discrete, such as a categorical label of a “dog” or a
are tightly coupled. In most modern machine learning work- “cat” given an input image, in which case the task is one of
flows, it is common to iteratively revisit earlier stages based classification. If the labels are continuous, such as the aver-
on the outcome at later stages, so that the machine learning age value of lift or drag given a specified airfoil geometry,
researcher is constantly asking new questions and revising the then the task is one of regression. In unsupervised learning,
data, the architecture, the loss functions, and the optimiza- there are no expert labels, and structure must be extracted
tion algorithm to improve performance. Here, we discuss from the input data alone; thus, this is often referred to as
these canonical stages of machine learning, investigate how data mining, and constitutes a particularly challenging field
to incorporate physics, and review examples in the field of of machine learning. Again, if the structure in the data is
fluid mechanics. This discussion is largely meant to be a assumed to be discrete, then the task is clustering. After the
high-level overview, and many more details can be found in clusters are identified and characterized, these groupings may
recent reviews [5,8–10]. be used as proxy labels to then classify new data. If the struc-
ture in the data is assumed to be continuously varying, then
2.1 The problem the task is typically thought of as an embedding or dimension-
ality reduction task. Principal component analysis (PCA) or
Data science is the art of asking and answering questions proper orthogonal decomposition (POD) may be thought of
with data. The sub-field of machine learning is concerned as unsupervised learning tasks that seek a continuous embed-
with leveraging historical data to build models that may be ding of reduced dimension [11]. Reinforcement learning is
deployed to automatically answer these questions, ideally in a third, large branch of machine learning research, in which
real-time, given new data. It is critical to select the right sys- an agent learns to make control decisions to interact with an

123
1720 S. L. Brunton

environment for some high level objection [12]. Examples from the choice of a problem above. The quality and quantity
include learning how to play games [13,14], such as chess of data directly affects the resulting machine learning model.
and go. Many machine learning architectures, such as deep neural
networks, are essentially sophisticated interpolation engines,
2.1.1 Embedding physics and so having a diversity of training data is essential to these
models being useful on unseen data. For example, modern
Deciding on what phenomena to model with machine learn- deep convolutional neural networks rose to prominence with
ing is often inherently related to the underlying physics. their unprecedented classification accuracy [53] on the Ima-
Although classical machine learning has been largely applied geNet data base [54], which contains over 14 million labeled
to “static” tasks, such as image classification and the place- images with over 20,000 categories, providing a sufficiently
ment of advertisements, increasingly it is possible to apply large and rich set of examples for training. This pairing of a
these techniques to model physical systems that evolve in vast labeled data set with a novel deep learning architecture
time according to some rules or physics. For example, we is widely regarded as the beginning of the modern era of deep
may formulate a learning problem to find and represent a con- learning [55].
served quantity, such as a Hamiltonian, purely from data [15].
Alternatively, the machine learning task may be to model 2.2.1 Embedding physics
time-series data as a differential equation, with the learn-
ing algorithm representing the dynamical system [16–20]. The training data provides several opportunities to embed
Similarly, the task may involve learning a coordinate trans- prior physical knowledge. If a system is known to exhibit
formation where these dynamics become simplified in some a symmetry, such translational or rotational invariance, then
physical way; i.e., coordinate transformations to linearize or it is possible to augment and enrich the training data with
diagonalize/decouple dynamics [21–28]. shifted or rotated examples. More generally, it is often
assumed that with an abundance of training data, these physi-
2.1.2 Examples in fluid mechanics cal invariances will automatically be learned by a sufficiently
expressive architecture. However, this approach tends to
There are many physical modeling tasks in fluid mechanics require considerable resources, both to collect and curate the
that are benefiting from machine learning [5,9]. A large field data, as well as to train increasingly large models, making it
of study focuses on formulating turbulence closure model- more appropriate for industrial scale, rather than academic
ing as a machine learning problem [8,29], such as learning scale, research. In contrast, it is also possible to use physi-
models for the Reynolds stresses [30,31] or sub-gridscale cal intuition to craft new features from the training data, for
turbulence [32,33]. Designing useful input features is also example by applying a coordinate transformation that may
an important way that prior physical knowledge is incorpo- simplify the representation or training. Physical data often
rated into turbulence closure modeling [34–36]. Similarly, comes from multiple sources with different fidelity, such as
machine learning has recently been focused on the problem of from numerical simulations, laboratory experiments, and in-
improving computational fluid dynamics (CFD) solvers [37– flight tests. This is an important area of research for flight
40]. Other important problems in fluid mechanics that benefit testing and unsteady aerodynamics [52], and recently physics
from machine learning include super-resolution [41,42], informed neural networks have been used with multifidelity
robust modal decompositions [1,43,44], network and cluster data to approximate PDEs [56].
modeling [45–47], control [4,48] and reinforcement learning
[49,50], and design of experiments in cyberphysical systems 2.2.2 Examples in fluid mechanics
[51]. Aerodynamics is a large related field with significant
data-driven advances [52]. The very nature of these prob- Fluids data is notoriously vast and high-dimensional, with
lems embeds the learning process into a larger physics-based individual flow fields often requiring millions (or more!)
framework, so that the models are more physically relevant degrees of freedom to characterize. Moreover, these flow
by construction. fields typically evolve in time, resulting in a time series of
multiple snapshots. Although vast in the spatial and/or tem-
2.2 The data poral dimensions, data is often rather sparse in parameter
space, as it is expensive to numerically or experimentally
Data is the lifeblood of machine learning, and our ability to investigate multiple geometries, Reynolds numbers, etc.
build effective models relies on what data is available or may Thus there are many algorithms designed for both rich and
be collected. As discussed earlier, choosing data to inform sparse data. Other considerations involve exciting transients
a model is closely related to choosing what to model in the and observing how the system evolves when it is away from
first place, and therefore this stage cannot be strictly separated its natural state. In many other cases, fluids data might be

123
Applying machine learning to study fluid mechanics 1721

quite limited, for example given by time-series data from a tems [62]. Genetic programming has also been widely used
few pressure measurements on the surface of an airfoil, or to learn human-interpretable, yet flexible representations of
from force recordings on an experimental turbine. data for modeling [16,63–65] and control [4]. In addition,
standard linear regression and generalized linear regression
2.3 The architecture are still widely used for modeling time-series data, espe-
cially in fluids. The dynamic mode decomposition (DMD)
Once a problem has been identified, and data is collected [1,17,66] employs linear regression with a low-rank con-
and curated, it is necessary to choose an architecture with straint in the optimization to find dominant spatiotemporal
which to represent the machine learning model. Typically, coherent structures that evolve linearly in time. The sparse
a machine learning model is a function that maps inputs to identification of nonlinear dynamics (SINDy) [18] algorithm
outputs employs generalized linear regression, with either a sparsity
promoting loss function [67] or a sparse optimization algo-
y = f(x; θ ) (1) rithm [18,68], to identify a differential equation model with
as few model terms as are necessary to fit the data.
and this function is generally represented within a specified
family of functions parameterized by values in θ . For exam- 2.3.1 Embedding physics
ple, a linear regression model would model outputs as a linear
function of the inputs, with θ parameterizing this linear map, Choosing a machine learning architecture with which to
or matrix. Neural networks have emerged as a particularly model the training data is one of the most intriguing opportu-
powerful and flexible class of models to represent functional nities to embed physical knowledge into the learning process.
relationships between data, and they have been shown to be Among the simplest choices are convolutional networks for
able to approximate arbitrarily complex functions with suffi- translationally invariant systems, and recurrent networks,
cient data and depth [57,58]. There is a tremendous variety of such as long-short-time memory (LSTM) networks [20]
potential neural network architectures [11], limited only by or reservoir computing [19,69], for systems that evolve in
the imagination of the human designer. The most common time. LSTMs have recently been used to predict aeroelastic
architecture is a simple feedforward network, in which data responses across a range of Mach numbers [70]. More gener-
enters through an input layer and maps sequentially through ally, equivariant networks seek to encode various symmetries
a number of computational layers until an output layer. Each by construction, which should improve accuracy and reduce
layer consists of nodes, where data from nodes in the pre- data requirements for physical systems [71–74]. Autoen-
vious layer are combined in a weighted sum and processed coder networks enforce the physical notion that there should
through an activation function, which is typically nonlinear. be low-dimensional structure, even for high-dimensional
In this way, neural networks are fundamentally composi- data, by imposing an information bottleneck, given by a con-
tional in nature. The parameters θ determine the network striction of the number of nodes in one or more layers of the
weights for how data is passed from one layer to the next, i.e. network. Such networks uncover nonlinear manifolds where
the weighted connectivity matrices for how nodes are con- the data is compactly represented, generalizing the linear
nected in adjacent layers. The overarching network topology dimensionality reduction obtained by PCA and POD. It is
(i.e., how many layers, how large, what type of activation also possible to embed physics more directly into the archi-
functions, etc.) is specified by the architect or determined tecture, for example by incorporating Hamiltonian [75,76] or
in a meta-optimization, thus determining the family of func- Lagrangian [77,78] structure. There are numerous successful
tions that may be approximated by that class of network. examples of physics-informed neural networks (PINNs) [79–
Then, the network weights for the specific architecture are 83], which solve supervised learning problems while being
optimized over the data to minimize a given loss function; constrained to satisfy a governing physical law. Graph neural
these stages are described next. networks have also shown the ability to learn generalizable
It is important to note that not all machine learning archi- physics in a range of challenging domains [64,84,85]. Deep
tectures are neural networks, although they are one of the operator networks [86] are able to learn continuous opera-
most powerful and expressive modern architectures, powered tors, such as governing partial differential equations, from
by increasingly big data and high performance computing. relatively limited training data.
Before the success of deep convolutional networks on the
ImageNet dataset, neural networks were not even mentioned 2.3.2 Examples in fluid mechanics
in the list of top ten machine learning algorithms [59]. Ran-
dom forests [60] and support vector machines [61] are two There are numerous examples of custom neural network
other leading architectures for supervised learning. Bayesian architectures being used to enforce physical solutions for
methods are also widely used, especially for dynamical sys- applications in fluid mechanics. The work of Ling et al.

123
1722 S. L. Brunton

[30] designed a custom neural network layer that enforced generalize. The sparse identification of nonlinear dynamics
Galilean invariance in the Reynolds stress tensors that they algorithm [18] learns dynamical systems models with as few
were modeling. Related Reynolds stress models have been terms from a library of candidate terms as are needed to
developed using the SINDy sparse modeling approach [87– describe the training data. There are several formulations
89]. Hybrid models that combine linear system identification involving different loss terms and optimization algorithms
and nonlinear neural networks have been used to model com- that promote additional physical notions, such as stability
plex aeroelastic systems [90]. The hidden fluid mechanics [99] and energy conservation [100]. Stability promoting loss
(HFM) approach is a physics-informed neural network strat- functions based on notions of Lyapunov stability have also
egy that encodes the Navier–Stokes equations while being been incorporated into autoencoders, with impressive results
flexible to the boundary conditions and geometry of the on fluid systems [101].
problem, enabling impressive physically quantifiable flow
field estimations from limited data [91]. Sparse sensing 2.4.2 Examples in fluid mechanics
has also been used to recover pressure distributions around
airfoils [92]. The Fourier neural operator is a novel oper- Sparse nonlinear modeling has been used extensively in fluid
ator network that performs super-resolution upscaling and mechanics, adding sparsity-promoting loss terms to learn
simulation modeling tasks [93]. Equivariant convolutional parsimonious models that prevent overfitting and general-
networks have been designed and applied to enforce sym- ize to new scenarios. SINDy has been used to generate
metries in high-dimensional complex systems from fluid reduced-order models for how dominant coherent structures
dynamics [73]. Physical invariances have also been incor- evolve in a flow for a range of configurations [100,102–105].
porated into neural networks for subgrid-scale scalar flux These models have also been extended to develop compact
modeling [94]. Lee and Carlberg [95] recently showed how closure models [87–89]. Recently, the physical notion of
to incorporate deep convolutional autoencoder networks into boundedness of solutions, which is a fundamental concept
the broader reduced-order modeling framework [96–98], in reduced-order models of fluids [106], has been incorpo-
taking advantage of the superior dimensionality reduction rated into the SINDy modeling framework as a novel loss
capabilities of deep autoencoders. function. Other physical loss functions may be added, such
as adding the divergence of a flow field as a loss term to
2.4 The loss function promote solutions that are incompressible [107].

The loss function is how we quantify how well the model is 2.5 The optimization algorithm
performing, often on a variety of tasks. For example, the L 2
error between the model output and the true output, averaged Ultimately, machine learning models are trained using opti-
over the input data, is a common term in the loss func- mization algorithms to find the parameters θ that best fit the
tion. In addition, other terms may be added to regularize training data. Typically, these optimization problems are both
the optimization (e.g., the L 1 or L 2 norm of the parameters high-dimensional and non-convex, leading to extremely chal-
θ to promote parsimony and prevent overfitting). Thus, the lenging optimization landscapes with many local minima.
loss function typically balances multiple competing objec- While there are powerful and generic techniques for convex
tives, such as model performance and model complexity. The optimization problems [108,109], there are few generic guar-
loss function may also incorporate terms used to promote antees for convergence or global optimality in non-convex
a specific behavior across different sub-networks in a neu- optimization. Modern deep neural networks have particularly
ral network architecture. Importantly, the loss function will high-dimensional parameters θ and require large training
provide valuable information used to approximate gradients data sets, which necessitate stochastic gradient descent algo-
required to optimize the parameters. rithms. In a sense, the optimization algorithm is the engine
powering machine learning, and as such, it is often abstracted
2.4.1 Embedding physics from the decision process. However, developing advanced
optimization algorithms is the focus of intense research
Most of the physics-informed architectures described above efforts. It is also often necessary to explicitly consider the
involve custom loss functions to promote the efficient train- optimization algorithm when designing a new architecture
ing of accurate models. It is also possible to incorporate or incorporating a novel loss term.
physical priors, such as sparsity, by adding L 1 or L 0 regu-
larizing loss terms on the parameters in θ . In fact, parsimony 2.5.1 Embedding physics
has been a central theme in physical modeling for century,
where it is believed that balancing model complexity with There are several ways that the optimization algorithm may
descriptive capability is essential in developing models that be customized or modified to incorporate prior physical

123
Applying machine learning to study fluid mechanics 1723

knowledge. One approach is to explicitly add constraints The nature of this topic is mercurial, as new innovations
to the optimization, for example that certain coefficients are being introduced every day that improve our capabili-
must be non-negative, or that other coefficients must satisfy ties and challenge our previous assumptions. Much of this
a specified algebraic relationship with each other. Depend- work has deliberately oversimplified the process of machine
ing on the given machine learning architecture, it may be learning and the field of fluid mechanics. Machine learning is
possible to enforce energy conservation [100] or stabil- largely concerned with fitting functions from data, and so it
ity constraints [99] in this way. Another approach involves is important to pick the right functions to fit. The inputs to the
employing custom optimization algorithms required to min- function are the variables and parameters that we have access
imize the physically motivated loss functions above, which to or control over, and the outputs are quantities of interest
are often non-convex. In this way, the line between loss func- that we would like to accurately and efficiently approximate
tion and optimization algorithm are often blurred, as they are in the future. It is a fruitful exercise to revisit classically
typically tightly coupled. For example, promoting sparsity important problems where progress was limited by our abil-
with the L 0 norm is non-convex, and several relaxed opti- ity to represent complex functions. For example, Ling et al.
mization formulations have been developed to approximately [30] had great success revisiting the classical Reynolds stress
solve this problem. The sparse relaxed regularized regres- models of Pope [110] with powerful modern techniques.
sion (SR3) optimization framework [68] has been developed More fundamentally, machine learning is about asking and
specifically to handle challenging non-convex loss terms that answering questions with data. We can’t forget why we are
arise in physically motivated problems. asking these questions in the first place: because we are curi-
ous, and there is value in knowing the answer.
2.5.2 Examples in fluid mechanics
Acknowledgements SLB acknowledges many valuable discussions
and perspectives gained from collaborators and coauthors Petros
Loiseau [100] showed that it is possible to enforce energy Koumoutsakos, J. Nathan Kutz, Jean-Christophe Loiseau, and Bernd
conservation for incompressible fluid flows directly by Noack.
imposing skew-symmetry constraints on the quadratic terms
of a sparse generalized linear regression (i.e. SINDy) model. Disclaimer Any omission or oversight was the result of either ignorance,
forgetfulness, hastiness, or lack of imagination on my part. These notes
These constraints manifest as equality constraints on the are not meant to be exhaustive, but rather to provide a few concrete
sparse coefficients θ of the SINDy model. Because the examples from the literature to guide researchers getting started in this
standard SINDy optimization procedure is based on a sequen- field. This field is growing at an incredible rate, and these examples
tially thresholded least-squares procedure, it is possible to provide a tiny glimpse into a much larger effort. I have tried to sample
from what I consider some of the most relevant and accessible literature.
enforce these equality constraints at every stage of the regres- However, a disproportionate number of references are to work by my
sion, using the Karush–Kuhn–Tucker (KKT) conditions. The close collaborators, as this is the work I am most familiar with. If I have
SR3 optimization package [68] was developed to general- missed any important references or connections, or mis-characterized
ize and extend these constrained optimization problems to any works cited here, please let me know and I’ll try to incorporate
corrections in future versions of these notes.
more challenging constraints, and to more generic optimiza-
tion problems. This is only one of many examples of custom Open Access This article is licensed under a Creative Commons
optimization algorithms being developed to train machine Attribution 4.0 International License, which permits use, sharing, adap-
learning models with novel loss functions or architectures. tation, distribution and reproduction in any medium or format, as
long as you give appropriate credit to the original author(s) and the
source, provide a link to the Creative Commons licence, and indi-
cate if changes were made. The images or other third party material
3 Parting thoughts in this article are included in the article’s Creative Commons licence,
unless indicated otherwise in a credit line to the material. If material
is not included in the article’s Creative Commons licence and your
This brief paper has attempted to provide a high level intended use is not permitted by statutory regulation or exceeds the
overview of the various stages of machine learning, how permitted use, you will need to obtain permission directly from the copy-
physics can be incorporated at each stage, and how these tech- right holder. To view a copy of this licence, visit http://creativecomm
niques are being applied today in fluid mechanics. Machine ons.org/licenses/by/4.0/.
learning for physical systems requires careful consideration
in each of these steps, as every stage provides an opportunity
to incorporate prior knowledge about the physics. A working
definition of physics is the part of a model that generalizes, References
and this is one of the central goals of machine learning models
1. Taira, K., Brunton, S.L., Dawson, S., et al.: Modal analysis of
for physical systems. It is also important to note that machine
fluid flows: An overview. AIAA J. 55, 4013–4041 (2017)
learning is fundamentally a collaborative effort, as it is nearly 2. Rabault, J., Kuchta, M., Jensen, A., et al.: Artificial neural
impossible to master every stage of this process. networks trained through deep reinforcement learning discover

123
1724 S. L. Brunton

control strategies for active flow control. J. Fluid Mech. 865, 281– 27. Otto, S.E., Rowley, C.W.: Linearly-recurrent autoencoder net-
302 (2019) works for learning dynamics. SIAM J. Appl. Dyn. Syst. 18,
3. Ren, F., Hu, H.B., Tang, H.: Active flow control using machine 558–593 (2019)
learning: A brief review. J. Hydrodyn. 32, 247–253 (2020) 28. Champion, K., Lusch, B., Kutz, J.N., et al.: Data-driven discovery
4. Zhou, Y., Fan, D., Zhang, B., et al.: Artificial intelligence control of coordinates and governing equations. Proc. Natl. Acad. Sci.
of a turbulent jet. J. Fluid Mech. 897, A27 (2020) 116, 22445–22451 (2019)
5. Brunton, S.L., Noack, B.R., Koumoutsakos, P.: Machine learning 29. Ahmed, S.E., Pawar, S., San, O., et al.: On closures for reduced
for fluid mechanics. Annu. Rev. Fluid Mech. 52, 477–508 (2020) order models − a spectrum of first-principle to machine-learned
6. Du, M., Liu, N., Hu, X.: Techniques for interpretable machine avenues. arXiv:2106.14954 (2021)
learning. Commun. ACM 63, 68–77 (2019) 30. Ling, J., Kurzawski, A., Templeton, J.: Reynolds averaged tur-
7. Molnar, C.: Interpretable machine learning. Lulu. com (2020) bulence modelling using deep neural networks with embedded
8. Duraisamy, K., Iaccarino, G., Xiao, H.: Turbulence modeling in invariance. J. Fluid Mech. 807, 155–166 (2016)
the age of data. Annu. Rev. Fluid Mech. 51, 357–377 (2019) 31. Kutz, J.N.: Deep learning in fluid dynamics. J. Fluid Mech. 814,
9. Brenner, M., Eldredge, J., Freund, J.: Perspective on machine 1–4 (2017)
learning for advancing fluid mechanics. Phys. Rev. Fluids 4, 32. Maulik, R., San, O., Rasheed, A., et al.: Subgrid modelling for
100501 (2019) two-dimensional turbulence using neural networks. J. Fluid Mech.
10. Brenner, M.P., Koumoutsakos, P.: Machine learning and physi- 858, 122–144 (2019)
cal review fluids: An editorial perspective. Phys. Rev. Fluids 6, 33. Novati, G., de Laroussilhe, H.L., Koumoutsakos, P.: Automating
070001 (2021) turbulence modelling by multi-agent reinforcement learning. Nat.
11. Brunton, S.L., Kutz, J.N.: Data-Driven Science and Engineering: Mach. Intell. 3, 87–96 (2021)
Machine Learning, Dynamical Systems, and Control. Cambridge 34. Wang, J.X., Wu, J.L., Xiao, H.: Physics-informed machine
University Press, Cambridge (2019) learning approach for reconstructing Reynolds stress modeling
12. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduc- discrepancies based on DNS data. Phys. Rev. Fluids 2, 034603
tion, vol. 1. MIT Press, Cambridge (1998) (2017)
13. Mnih, V., Kavukcuoglu, K., Silver, D., et al.: Human-level control 35. Zhu, L., Zhang, W., Kou, J., et al.: Machine learning methods
through deep reinforcement learning. Nature 518, 529 (2015) for turbulence modeling in subsonic flows around airfoils. Phys.
14. Silver, D., Schrittwieser, J., Simonyan, K., et al.: Mastering the Fluids 31, 015105 (2019)
game of go without human knowledge. Nature 550, 354–359 36. Zhu, L., Zhang, W., Sun, X., et al.: Turbulence closure for high
(2017) Reynolds number airfoil flows by deep neural networks. Aerosp.
15. Kaiser, E., Kutz, J.N., Brunton, S.L.: Discovering conservation Sci. Technol. 110, 106452 (2021)
laws from data for control. In: 2018 IEEE Conference on Decision 37. Bar-Sinai, Y., Hoyer, S., Hickey, J., et al.: Learning data-driven
and Control (CDC), pp. 6415–6421. IEEE (2018) discretizations for partial differential equations. Proc. Natl. Acad.
16. Schmidt, M., Lipson, H.: Distilling free-form natural laws from Sci. 116, 15344–15349 (2019)
experimental data. Science 324, 81–85 (2009) 38. Thaler, S., Paehler, L., Adams, N.A.: Sparse identification of trun-
17. Schmid, P.J.: Dynamic mode decomposition of numerical and cation errors. J. Comput. Phys. 397, 108851 (2019)
experimental data. J. Fluid Mech. 656, 5–28 (2010) 39. Stevens, B., Colonius, T.: Enhancement of shock-capturing meth-
18. Brunton, S.L., Proctor, J.L., Kutz, J.N.: Discovering governing ods via machine learning. Theoret. Comput. Fluid Dyn. 34,
equations from data by sparse identification of nonlinear dynam- 483–496 (2020)
ical systems. Proc. Natl. Acad. Sci. 113, 3932–3937 (2016) 40. Kochkov, D., Smith, J.A., Alieva, A., et al.: Machine learn-
19. Pathak, J., Lu, Z., Hunt, B.R., et al.: Using machine learning to ing accelerated computational fluid dynamics. arXiv:2102.01010
replicate chaotic attractors and calculate lyapunov exponents from (2021)
data. Chaos: An Interdisciplinary. J. Nonlinear Sci. 27, 121102 41. Erichson, N.B., Mathelin, L., Yao, Z., et al.: Shallow neural net-
(2017) works for fluid flow reconstruction with limited sensors. Proc. R.
20. Vlachas, P.R., Byeon, W., Wan, Z.Y., et al.: Data-driven fore- Soc. A 476(2238), 20200097 (2020)
casting of high-dimensional chaotic systems with long short-term 42. Fukami, K., Fukagata, K., Taira, K.: Super-resolution reconstruc-
memory networks. Proc. R. Soc. A 474, 20170844 (2018) tion of turbulent flows with machine learning. J. Fluid Mech. 870,
21. Lusch, B., Kutz, J.N., Brunton, S.L.: Deep learning for universal 106–120 (2019)
linear embeddings of nonlinear dynamics. Nat. Commun. 9, 4950 43. Taira, K., Hemati, M.S., Brunton, S.L., et al.: Modal analysis of
(2018) fluid flows: Applications and outlook. AIAA J. 58(3), 998–1022
22. Wehmeyer, C., Noé, F.: Time-lagged autoencoders: Deep learning (2020)
of slow collective variables for molecular kinetics. J. Chem. Phys. 44. Scherl, I., Strom, B., Shang, J.K., et al.: Robust principal compo-
148, 1–9 (2018) nent analysis for particle image velocimetry. Phys. Rev. Fluids 5,
23. Mardt, A., Pasquali, L., Wu, H., et al.: VAMPnets: Deep learning 054401 (2020)
of molecular kinetics. Nat. Commun. 9, 5 (2018) 45. Nair, A.G., Taira, K.: Network-theoretic approach to sparsified
24. Takeishi, N., Kawahara, Y., Yairi, T.: Learning koopman invari- discrete vortex dynamics. J. Fluid Mech. 768, 549–571 (2015)
ant subspaces for dynamic mode decomposition. In: Advances in 46. Kaiser, E., Noack, B.R., Cordier, L., et al.: Cluster-based reduced-
Neural Information Processing Systems, pp. 1130–1140 (2017) order modelling of a mixing layer. J. Fluid Mech. 754, 365–414
25. Li, Q., Dietrich, F., Bollt, E.M., et al.: Extended dynamic mode (2014)
decomposition with dictionary learning: A data-driven adaptive 47. Fernex, D., Noack, B.R., Semaan, R.: Cluster-based network
spectral decomposition of the koopman operator. Chaos: An Inter- modeling–from snapshots to complex dynamical systems. Sci-
disciplinary. J. Nonlinear Sci. 27, 103111 (2017) ence. Advances 7, eabf5006 (2021)
26. Yeung, E., Kundu, S., Hodas, N.: Learning deep neural network 48. Maceda, G.Y.C., Li, Y., Lusseyran, F., et al.: Stabilization of the
representations for koopman operators of nonlinear dynamical fluidic pinball with gradient-enriched machine learning control.
systems. arXiv:1708.06850 (2017) J. Fluid Mech. 917, 45 (2021)

123
Applying machine learning to study fluid mechanics 1725

49. Fan, D., Yang, L., Wang, Z., et al.: Reinforcement learning for 74. Batzner, S., Smidt, T.E., Sun, L., et al.: Se (3)-equivariant graph
bluff body active flow control in experiments and simulations. neural networks for data-efficient and accurate interatomic poten-
Proc. Natl. Acad. Sci. 117, 26091–26098 (2020) tials. arXiv:2101.03164 (2021)
50. Verma, S., Novati, G., Koumoutsakos, P.: Efficient collective 75. Greydanus, S., Dzamba, M., Yosinski, J.: Hamiltonian neural net-
swimming by harnessing vortices through deep reinforcement works. Adv. Neural Inf. Process. Syst. 32, 15379–15389 (2019)
learning. Proc. Natl. Acad. Sci. 115, 5849–5854 (2018) 76. Finzi, M., Wang, K.A., Wilson, A.G.: Simplifying Hamiltonian
51. Fan, D., Jodin, G., Consi, T., et al.: A robotic intelligent towing and Lagrangian neural networks via explicit constraints. Adv.
tank for learning complex fluid–structure dynamics. Sci. Robot. Neural Inf. Process. Syst. 33, 1–10 (2020)
4, 36 (2019) 77. Cranmer, M., Greydanus, S., Hoyer, S., et al.: Lagrangian neural
52. Kou, J., Zhang, W.: Data-driven modeling for unsteady aerody- networks. arXiv:2003.04630 (2020)
namics and aeroelasticity. Prog. Aerosp. Sci. 125, 100725 (2021) 78. Zhong, Y.D., Leonard, N.: Unsupervised learning of Lagrangian
53. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classifica- dynamics from images for prediction and control. Adv. Neural
tion with deep convolutional neural networks. In: Advances in Inf. Process. Syst. 33, 1–12 (2020)
neural information processing systems, pp. 1097–1105 (2012) 79. Raissi, M., Perdikaris, P., Karniadakis, G.: Physics-informed neu-
54. Deng, J., Dong, W., Socher, R., et al.: Imagenet: A largescale hier- ral networks: A deep learning framework for solving forward and
archical image database. In: 2009 IEEE conference on computer inverse problems involving nonlinear partial differential equa-
vision and pattern recognition, pp. 248–255. IEEE (2009) tions. J. Comput. Phys. 378, 686–707 (2019)
55. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT 80. Pang, G., Lu, L., Karniadakis, G.E.: fpinns: Fractional physics-
Press, Cambridge (2016) informed neural networks. SIAM J. Sci. Comput. 41, A2603–
56. Meng, X., Karniadakis, G.E.: A composite neural network that A2626 (2019)
learns from multi-fidelity data: Application to function approxi- 81. Yang, L., Zhang, D., Karniadakis, G.E.: Physics-informed gen-
mation and inverse PDE problems. J. Comput. Phys. 401, 109020 erative adversarial networks for stochastic differential equations.
(2020) SIAM J. Sci. Comput. 42, A292–A317 (2020)
57. Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward 82. Mao, Z., Jagtap, A.D., Karniadakis, G.E.: Physics-informed neu-
networks are universal approximators. Neural Netw. 2, 359–366 ral networks for high-speed flows. Comput. Methods Appl. Mech.
(1989) Eng. 360, 112789 (2020)
58. Hornik, K.: Approximation capabilities of multilayer feedforward 83. Karniadakis, G.E., Kevrekidis, I.G., Lu, L., et al.: Physics-
networks. Neural Netw. 4, 251–257 (1991) informed machine learning. Nat. Rev. Phys. 3, 422–440 (2021)
59. Wu, X., Kumar, V., Quinlan, J.R., et al.: Top 10 algorithms in data 84. Battaglia, P.W., Hamrick, J.B., Bapst, V., et al.: Relational induc-
mining. Knowl. Inf. Syst. 14, 1–37 (2008) tive biases, deep learning, and graph networks. arXiv:1806.01261
60. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001) (2018)
61. Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vec- 85. Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., et al.: Learning to
tor Machines, Regularization, Optimization and Beyond. MIT simulate complex physics with graph networks. In: International
Press, Cambridge (2002) Conference on Machine Learning, pp. 8459–8468. PMLR (2020)
62. Blanchard, A., Sapsis, T.: Bayesian optimization with output- 86. Lu, L., Jin, P., Pang, G., et al.: Learning nonlinear operators via
weighted optimal sampling. J. Comput. Phys. 425, 109901 (2021) deeponet based on the universal approximation theorem of oper-
63. Bongard, J., Lipson, H.: Automated reverse engineering of non- ators. Nat. Mach. Intell. 3, 218–229 (2021)
linear dynamical systems. Proc. Natl. Acad. Sci. 104, 9943–9948 87. Beetham, S., Capecelatro, J.: Formulating turbulence closures
(2007) using sparse regression with embedded form invariance. Phys.
64. Cranmer, M.D., Xu, R., Battaglia, P., et al.: Learning symbolic Rev. Fluids 5, 084611 (2020)
physics with graph networks. arXiv:1909.05862 (2019) 88. Beetham, S., Fox, R.O., Capecelatro, J.: Sparse identification of
65. Cranmer, M., Sanchez-Gonzalez, A., Battaglia, P., et al.: Discov- multiphase turbulence closures for coupled fluid–particle flows.
ering symbolic models from deep learning with inductive biases. J. Fluid Mech. 914, A11 (2021)
arXiv:2006.11287 (2020) 89. Schmelzer, M., Dwight, R.P., Cinnella, P.: Discovery of algebraic
66. Kutz, J.N., Brunton, S.L., Brunton, B.W., et al.: Dynamic Mode Reynolds-stress models using sparse symbolic regression. Flow
Decomposition: Data-Driven Modeling of Complex Systems. Turbul. Combust. 104(2), 579–603 (2020)
SIAM, Bangkok (2016) 90. Kou, J., Zhang, W.: A hybrid reduced-order framework for com-
67. Tibshirani, R.: Regression shrinkage and selection via the lasso. plex aeroelastic simulations. Aerosp. Sci. Technol. 84, 880–894
J. R. Stat. Soc. Ser. B (Methodological) 58, 267–288 (1996) (2019)
68. Zheng, P., Askham, T., Brunton, S.L., et al.: Sparse relaxed regu- 91. Raissi, M., Yazdani, A., Karniadakis, G.E.: Hidden fluid mechan-
larized regression: SR3. IEEE Access 7, 1404–1423 (2019) ics: Learning velocity and pressure fields from flow visualizations.
69. Pathak, J., Hunt, B., Girvan, M., et al.: Model-free prediction Science 367, 1026–1030 (2020)
of large spatiotemporally chaotic systems from data: A reservoir 92. Zhao, X., Du, L., Peng, X., et al.: Research on refined recon-
computing approach. Phys. Rev. Lett. 120, 024102 (2018) struction method of airfoil pressure based on compressed sensing.
70. Li, K., Kou, J., Zhang, W.: Deep neural network for unsteady aero- Theoret. Appl. Mech. Lett. 11, 100223 (2021)
dynamic and aeroelastic modeling across multiple mach numbers. 93. Li, Z., Kovachki, N., Azizzadenesheli, K., et al.: Fourier
Nonlinear Dyn. 96, 2157–2177 (2019) neural operator for parametric partial differential equations.
71. Thomas, N., Smidt, T., Kearnes, S., et al.: Tensor field networks: arXiv:2010.08895 (2020)
Rotation-and translation-equivariant neural networks for 3d point 94. Frezat, H., Balarac, G., Le Sommer, J., et al.: Physical invariance
clouds. arXiv:1802.08219 (2018) in neural networks for subgrid-scale scalar flux modeling. Phys.
72. Miller, B.K., Geiger, M., Smidt, T.E., et al.: Relevance of rotation- Rev. Fluids 6(2), 024607 (2021)
ally equivariant convolutions for predicting molecular properties. 95. Lee, K., Carlberg, K.T.: Model reduction of dynamical systems
arXiv:2008.08461 (2020) on nonlinear manifolds using deep convolutional autoencoders. J.
73. Wang, R., Walters, R., Yu, R.: Incorporating symmetry into deep Comput. Phys. 404, 108973 (2020)
dynamics models for improved generalization. arXiv:2002.03061
(2020)

123
1726 S. L. Brunton

96. Noack, B.R., Afanasiev, K., Morzynski, M., et al.: A hierarchy 104. Deng, N., Noack, B.R., Morzyński, M., et al.: Low-order model
of low-dimensional models for the transient and post-transient for successive bifurcations of the fluidic pinball. J. Fluid Mech.
cylinder wake. J. Fluid Mech. 497, 335–363 (2003) 884, A37 (2020)
97. Benner, P., Gugercin, S., Willcox, K.: A survey of projection- 105. Deng, N., Noack, B.R., Morzyński, M., et al.: Galerkin force
based model reduction methods for parametric dynamical sys- model for transient and post-transient dynamics of the fluidic pin-
tems. SIAM Rev. 57, 483–531 (2015) ball. J. Fluid Mech. 918, A4 (2021)
98. Rowley, C.W., Dawson, S.T.: Model reduction for flow analysis 106. Schlegel, M., Noack, B.R.: On long-term boundedness of
and control. Annu. Rev. Fluid Mech. 49, 387–417 (2017) Galerkin models. J. Fluid Mech. 765, 325–352 (2015)
99. Kaptanoglu, A.A., Callaham, J.L., Hansen, C.J., et al.: Promot- 107. Wang, R., Kashinath, K., Mustafa, M., et al.: Towards physics-
ing global stability in data-driven models of quadratic nonlinear informed deep learning for turbulent flow prediction. In: Pro-
dynamics. arXiv:2105.01843 (2021) ceedings of the 26th ACM SIGKDD International Conference
100. Loiseau, J.C., Brunton, S.L.: Constrained sparse Galerkin regres- on Knowledge Discovery & Data Mining, pp. 1457–1466 (2020)
sion. J. Fluid Mech. 838, 42–67 (2018) 108. Grant, M., Boyd, S., Ye, Y.: Cvx: Matlab software for disciplined
101. Erichson, N.B., Muehlebach, M., Mahoney, M.W.: Physics- convex programming (2008)
informed autoencoders for lyapunov-stable fluid flow prediction. 109. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge
arXiv:1905.10866 (2019) University Press, Cambridge (2009)
102. Loiseau, J.C., Noack, B.R., Brunton, S.L.: Sparse reduced-order 110. Pope, S.: A more general effective-viscosity hypothesis. J. Fluid
modeling: Sensor-based dynamics to full-state estimation. J. Fluid Mech. 72, 331–340 (1975)
Mech. 844, 459–490 (2018)
103. Loiseau, J.C.: Data-driven modeling of the chaotic thermal con-
vection in an annular thermosyphon. Theoret. Comput. Fluid Dyn.
34, 339–365 (2020)

123