Elasticity Theory

A large part of geophysics concerns understanding how material deforms when it is squeezed, stretched,

or sheared. Elasticity theory is the mathematical framework which describes such deformation. By elastic,

we mean that the material rebounds to its original shape after the forces on it are removed; a rubber
eraser

is a good example of an elastic material. Play-dough, on the other hand, is a good example of a material

which is not elastic- if you squeeze it and then let go, the material stays in the shape it was when you
squeezed it. Materials of this type are described as being plastic, rather than elastic, and elasticity theory

is not suitable for understanding how these plastic materials deform. In reality, most materials are both
elastic and plastic, but on different time scales. Silly-putty is one such example- it stretches like hot

bubble gum, which is plastic behavior, but you can also roll it up and bounce it like a ball, which means

it’s also elastic. The key is the time duration of the external force- when you stretch it like gum, the force

is acting over several seconds, while when you bounce it off the floor, the impact on the floor lasts only

a fraction of a second. So silly putty is plastic for forces acting over time scales of several seconds, by

is elastic for very short –duration forces. Most of the materials that make up the earth are in fact elastic

over time scales of years, and plastic over time scales of thousands to millions of years.

Stress

Before we describe how a medium deforms under an external force, we need to be able to describe

precisely these external forces in terms of how they are oriented. Stress is defined as the force per

unit area. When a deforming force is applied to a body, the stress is the ratio of the force to the area over
which it is applied. If a force of 1 Newton(N) is applied uniformly to an area of 1 square meter, the stress

is 1 N/M2 = 1 (Kg x M/S2) / M2 = 1 Kg/(M S2) = 1 Pascal. Your intuition should tell you that it is stress,

rather than just force, which determines how a material deforms- think of an elephant inside a fancy
ballroom, in one case standing on its feet, and in another case balanced on the tip of a needle. Clearly,

the needle is going to puncture the floor much more effectively than the elephant’s feet. The reason
is because although the force is the same, ie , the mass of the elephant times gravity, the area over

which it is distributed is much smaller in the case of the needle, which, since stress is force/area, means

the stress is by far greater in the case of the needle.

There are two basic types of stress. If the force is perpendicular (normal) to the surface over which

it is acting, then the stress is termed a normal stress; if it is tangential to the surface, it is called
a shear stress. Usually, the foce is neither enitrely normal nor tangential, but is at some arbitrary
intermediate

angle, in which case it can be resolved into components which are both normal and tangential to the
surface; so the stress is composed of both normal and shearing components. The sign convention is that

tensional stresses are positive and compressional stresses negative (in geophysics).

Directionality

Stress also has directionality to it- clearly a force acting in one direction will deform an object differently
than a force which is oriented purpendicular to it. So, we need to develop how we incorporate
directionality into stress. To do this, we use a hypothetical cube of material, it can be a free standing cube,

or one imbedded deep in a larger piece of material. Consider a small parallelipiped with sides dx,dy, and

dz, as shown in Figure 1 below, and imagine that it is being stressed by

some external force. On each face,

Figure 1 The parallelipiped is in static equilbrium. Stresses acting on the shaded back face are (-sxx, -sxy, and
-sxz). Stress acting on the front face are (sxx, sxy, and sxz).

the stresses can be resolved into components in the x,y, and z directions. The stresses acting on the shaded

face are -sxx, -sxy, and -sxz. The notation is that sxy refers to the stress s acting in the y direction on the face
which is perpendicular to the x axis. The normal stress is thus -sxx and the shearing stresses are -sxy, and -sxz.

If the parallelipiped is to be in static equilibrium (not moving), then the stresses on opposite faces must
balance, and there must be no net couple which would rotate the parallelipiped. This requires that the
stresses on opposite faces be equal in magnitude and opposite in direction. The shearing stresses on
opposite faces of the parallelipiped (eg, -sxy and sxy on the back and front faces as shown in Figure 1)
provide a couple of forces, or couple, which will roate the parallelipiped. Since the parallelipiped must not
rotate, this couple must be balanced by the couple provided by the shearing stresses -syx, and syx

acting on the two side faces. This means that sxy must equal syx. . The same conditions apply to the other
shearing stresses: sxy = syx, sxz = szx, syz = szy (ie, the stress tensor must be symmetric).

Scalars versus Vectors versus Tensors

Stress and Strain are both tensor quantities, which means that they are more than just simple scalars

or vectors. Everyday scalar numbers are just that- numbers, the dollars in my wallet, the pounds that I weigh, etc. However, there are other types of quantites
that require more information than just the amount-

for example, the velocity of my Subaru headed eastbound on I-90. While I might be driving 65 mph, that

doesn’t actually describe my motion- we need the direction also, since 60mph east is very different from 60mph west if I’m analyzing rear-ending a semi going
east bound 70mph. So velocity is a vector, meaning

it has both a value and a direction associated with it. Tensors are just an extension of this, it’s a number which has any number of directions associated with it. In
the case of stress, it is a 2-direction tensor, containing the direction of the force and the direction of the perpendicular to the face on which the force is acting.
Strain is like wise a tensor with two directions, or a rank 2 tensor. A physical quantity which had

three directions associated with it would simply be a rank-3 tensor.

Numerical Mathematics: Theory, Methods and Applications. Numerical Mathematics: Theory, Methods
and Applications (NM-TMA) publishes high-quality original research papers on the construction, analysis
and application of numerical methods for solving scientific and engineering problems.

Numerical analysis
From Wikipedia, the free encyclopedia
Jump to navigationJump to search
This article includes a list of references, but its sources remain unclear because it
has insufficient inline citations. Please help to improve this article
by introducing more precise citations. (November 2013) (Learn how and when to
remove this template message)
Babylonian clay tablet YBC 7289 (c. 1800–1600 BC) with annotations. The approximation of the square root
of 2 is four sexagesimal figures, which is about six decimal figures. 1 + 24/60 + 51/602 + 10/603=
1.41421296...[1]

Numerical analysis is the study of algorithms that use numerical approximation (as opposed to
general symbolic manipulations) for the problems of mathematical analysis (as distinguished from discrete
mathematics). Numerical analysis naturally finds application in all fields of engineering and the physical
sciences, but in the 21st century also the life sciences, social sciences, medicine, business and even the arts
have adopted elements of scientific computations. As an aspect of mathematics and computer science that
generates, analyzes, and implements algorithms, the growth in power and the revolution in computing has
raised the use of realistic mathematical models in science and engineering, and complex numerical analysis
is required to provide solutions to these more involved models of the world. Ordinary differential
equations appear in celestial mechanics (planets, stars and galaxies); numerical linear algebra is important
for data analysis; stochastic differential equations and Markov chains are essential in simulating living cells
for medicine and biology.
Before the advent of modern computers, numerical methods often depended on hand interpolation in
large printed tables. Since the mid 20th century, computers calculate the required functions instead. These
same interpolation formulas nevertheless continue to be used as part of the software algorithms for
solving differential equations.
One of the earliest mathematical writings is a Babylonian tablet from the Yale Babylonian Collection (YBC
7289), which gives a sexagesimal numerical approximation of the square root of 2, the length of
the diagonal in a unit square. Being able to compute the sides of a triangle (and hence, being able to
compute square roots) is extremely important, for instance, in astronomy, carpentry and construction.[2]
Numerical analysis continues this long tradition of practical mathematical calculations. Much like
the Babylonian approximation of the square root of 2, modern numerical analysis does not seek exact
answers, because exact answers are often impossible to obtain in practice. Instead, much of numerical
analysis is concerned with obtaining approximate solutions while maintaining reasonable bounds on
errors.

General introduction[edit]
The overall goal of the field of numerical analysis is the design and analysis of techniques to give
approximate but accurate solutions to hard problems, the variety of which is suggested by the following:

 Advanced numerical methods are essential in making numerical weather prediction feasible.
 Computing the trajectory of a spacecraft requires the accurate numerical solution of a system
of ordinary differential equations.
 Car companies can improve the crash safety of their vehicles by using computer simulations of car
crashes. Such simulations essentially consist of solving partial differential equations numerically.
 Hedge funds (private investment funds) use tools from all fields of numerical analysis to attempt to
calculate the value of stocks and derivatives more precisely than other market participants.
 Airlines use sophisticated optimization algorithms to decide ticket prices, airplane and crew
assignments and fuel needs. Historically, such algorithms were developed within the overlapping field
of operations research.
 Insurance companies use numerical programs for actuarial analysis.
The rest of this section outlines several important themes of numerical analysis.
History[edit]
The field of numerical analysis predates the invention of modern computers by many centuries. Linear
interpolation was already in use more than 2000 years ago. Many great mathematicians of the past were
preoccupied by numerical analysis, as is obvious from the names of important algorithms like Newton's
method, Lagrange interpolation polynomial, Gaussian elimination, or Euler's method.
To facilitate computations by hand, large books were produced with formulas and tables of data such as
interpolation points and function coefficients. Using these tables, often calculated out to 16 decimal places
or more for some functions, one could look up values to plug into the formulas given and achieve very
good numerical estimates of some functions. The canonical work in the field is the NIST publication edited
by Abramowitz and Stegun, a 1000-plus page book of a very large number of commonly used formulas and
functions and their values at many points. The function values are no longer very useful when a computer
is available, but the large listing of formulas can still be very handy.
The mechanical calculator was also developed as a tool for hand computation. These calculators evolved
into electronic computers in the 1940s, and it was then found that these computers were also useful for
administrative purposes. But the invention of the computer also influenced the field of numerical analysis,
since now longer and more complicated calculations could be done.
Direct and iterative methods[edit]
Direct vs iterative methods
Consider the problem of solving
3x3 + 4 = 28
for the unknown quantity x.
Direct method
3x3 + 4 = 28.
Subtract 4 3x3 = 24.
Divide by 3 x3 = 8.
Take cube roots x = 2.
For the iterative method, apply
3
the bisection method to f(x) = 3x − 24. The
initial values are a = 0, b = 3, f(a) = −24, f(b)
= 57.
Iterative method
a b mid f(mid)
0 3 1.5 −13.875
1.5 3 2.25 10.17...
 1.5 2.25 1.875 −4.22...
1.875 2.25 2.0625 2.32...
We conclude from this table that the
solution is between 1.875 and 2.0625. The
algorithm might return any number in that
range with an error less than 0.2.
Discretization and numerical
integration[edit]

In a two-hour race, we have measured the

speed of the car at three instants and
recorded them in the following table.
Time 0:20 1:00 1:40
km/h 140 150 180
A discretization would be to say that the
speed of the car was constant from 0:00 to
0:40, then from 0:40 to 1:20 and finally from
1:20 to 2:00. For instance, the total distance
traveled in the first 40 minutes is
approximately (2/3 h × 140 km/h) = 93.3 km.
This would allow us to estimate the total
distance traveled
as 93.3 km + 100 km + 120 km = 313.3 km,
which is an example of numerical
integration (see below) using a Riemann
sum, because displacement is the integral of
velocity.
Ill-conditioned problem: Take the
function f(x) = 1/(x − 1). Note that f(1.1) = 10
and f(1.001) = 1000: a change in x of less
than 0.1 turns into a change in f(x) of nearly
1000. Evaluating f(x) near x = 1 is an ill-
conditioned problem.
Well-conditioned problem: By contrast,
evaluating the same function f(x) =
1/(x − 1) near x = 10 is a well-conditioned
problem. For instance, f(10) = 1/9 ≈ 0.111
and f(11) = 0.1: a modest change in x leads
to a modest change in f(x).

Direct methods compute the solution to a problem in a finite number of steps. These methods would give
the precise answer if they were performed in infinite precision arithmetic. Examples include Gaussian
elimination, the QR factorization method for solving systems of linear equations, and the simplex
method of linear programming. In practice, finite precision is used and the result is an approximation of
the true solution (assuming stability).
In contrast to direct methods, iterative methods are not expected to terminate in a finite number of steps.
Starting from an initial guess, iterative methods form successive approximations that converge to the exact
solution only in the limit. A convergence test, often involving the residual, is specified in order to decide
when a sufficiently accurate solution has (hopefully) been found. Even using infinite precision arithmetic
these methods would not reach the solution within a finite number of steps (in general). Examples
include Newton's method, the bisection method, and Jacobi iteration. In computational matrix algebra,
iterative methods are generally needed for large problems.
Iterative methods are more common than direct methods in numerical analysis. Some methods are direct
in principle but are usually used as though they were not, e.g. GMRES and the conjugate gradient method.
For these methods the number of steps needed to obtain the exact solution is so large that an
approximation is accepted in the same manner as for an iterative method.
Discretization[edit]
Furthermore, continuous problems must sometimes be replaced by a discrete problem whose solution is
known to approximate that of the continuous problem; this process is called discretization. For example,
the solution of a differential equation is a function. This function must be represented by a finite amount
of data, for instance by its value at a finite number of points at its domain, even though this domain is
a continuum.

Generation and propagation of errors[edit]

The study of errors forms an important part of numerical analysis. There are several ways in which error
can be introduced in the solution of the problem.
Round-off[edit]
Round-off errors arise because it is impossible to represent all real numbers exactly on a machine with
finite memory (which is what all practical digital computers are).
Truncation and discretization error[edit]
Truncation errors are committed when an iterative method is terminated or a mathematical procedure is
approximated, and the approximate solution differs from the exact solution. Similarly, discretization
induces a discretization error because the solution of the discrete problem does not coincide with the
solution of the continuous problem. For instance, in the iteration in the sidebar to compute the solution

of , after 10 or so iterations, we conclude that the root is roughly 1.99 (for example). We therefore
have a truncation error of 0.01.
Once an error is generated, it will generally propagate through the calculation. For instance, we have
already noted that the operation + on a calculator (or a computer) is inexact. It follows that a calculation of

the type is even more inexact.

What does it mean when we say that the truncation error is created when we approximate a mathematical
procedure? We know that to integrate a function exactly requires one to find the sum of infinite
trapezoids. But numerically one can find the sum of only finite trapezoids, and hence the approximation of
the mathematical procedure. Similarly, to differentiate a function, the differential element approaches
zero but numerically we can only choose a finite value of the differential element.
Numerical stability and well-posed problems[edit]
Numerical stability is an important notion in numerical analysis. An algorithm is called numerically stable if
an error, whatever its cause, does not grow to be much larger during the calculation. This happens if the
problem is well-conditioned, meaning that the solution changes by only a small amount if the problem data
are changed by a small amount. To the contrary, if a problem is ill-conditioned, then any small error in the
data will grow to be a large error.
Both the original problem and the algorithm used to solve that problem can be well-conditioned and/or ill-
conditioned, and any combination is possible.
So an algorithm that solves a well-conditioned problem may be either numerically stable or numerically
unstable. An art of numerical analysis is to find a stable algorithm for solving a well-posed mathematical
problem. For instance, computing the square root of 2 (which is roughly 1.41421) is a well-posed problem.

Many algorithms solve this problem by starting with an initial approximation x0 to , for instance x0 =
1.4, and then computing improved guesses x1, x2, etc. One such method is the famous Babylonian method,
which is given by xk+1 = xk/2 + 1/xk. Another method, which we will call Method X, is given by xk+1 = (xk2 −
2)2 + xk.[3] We have calculated a few iterations of each scheme in table form below, with initial guesses x0 =
1.4 and x0 = 1.42.

Babylonian Babylonian Method X Method X

x0 = 1.4 x0 = 1.42 x0 = 1.4 x0 = 1.42

x1 = 1.4142857... x1 = 1.41422535... x1 = 1.4016 x1 = 1.42026896

x2 = 1.414213564... x2 = 1.41421356242... x2 = 1.4028614... x2 = 1.42056...

... ...

x1000000 = 1.41421... x27 = 7280.2284...

Observe that the Babylonian method converges quickly regardless of the initial guess, whereas Method X
converges extremely slowly with initial guess x0 = 1.4 and diverges for initial guess x0 = 1.42. Hence, the
Babylonian method is numerically stable, while Method X is numerically unstable.
Numerical stability is affected by the number of the significant digits the machine keeps on, if we
use a machine that keeps only the four most significant decimal digits, a good example on loss of
significance is given by these two equivalent functions

If we compare the results of

and

by looking to the two results above, we realize that loss of significance (caused here
by catastrophic cancelation) has a huge effect on the results, even though both functions are
equivalent, as shown below

The desired value, computed using infinite precision, is 11.174755...

 The example is a modification of one taken from Mathew; Numerical
methods using Matlab, 3rd ed.

Areas of study[edit]
The field of numerical analysis includes many sub-disciplines. Some of the
major ones are:
Computing values of functions[edit]
Interpolation: We have observed
the temperature to vary from 20
degrees Celsius at 1:00 to 14
degrees at 3:00. A linear
interpolation of this data would
conclude that it was 17 degrees at
2:00 and 18.5 degrees at 1:30pm.
Extrapolation: If the gross domestic
product of a country has been
growing an average of 5% per year
and was 100 billion dollars last year,
we might extrapolate that it will be
105 billion dollars this year.

Regression: In linear regression,

given n points, we compute a line
that passes as close as possible to
those n points.

Optimization: Say you sell

lemonade at a lemonade stand, and
notice that at $1, you can sell 197
glasses of lemonade per day, and
that for each increase of $0.01, you
will sell one glass of lemonade less
per day. If you could charge $1.485,
you would maximize your profit, but
due to the constraint of having to
charge a whole cent amount,
charging $1.48 or $1.49 per glass
will both yield the maximum income
of $220.52 per day.
 Differential equation: If you set up
100 fans to blow air from one end
of the room to the other and then
you drop a feather into the wind,
what happens? The feather will
follow the air currents, which may
be very complex. One
approximation is to measure the
speed at which the air is blowing
near the feather every second, and
advance the simulated feather as if
it were moving in a straight line at
that same speed for one second,
before measuring the wind speed
again. This is called the Euler
method for solving an ordinary
differential equation.

One of the simplest problems is the evaluation of a function at a given point.

The most straightforward approach, of just plugging in the number in the
formula is sometimes not very efficient. For polynomials, a better approach is
using the Horner scheme, since it reduces the necessary number of
multiplications and additions. Generally, it is important to estimate and
control round-off errors arising from the use of floating point arithmetic.
Interpolation, extrapolation, and regression[edit]
Interpolation solves the following problem: given the value of some unknown
function at a number of points, what value does that function have at some
other point between the given points?
Extrapolation is very similar to interpolation, except that now we want to find
the value of the unknown function at a point which is outside the given points.
Regression is also similar, but it takes into account that the data is imprecise.
Given some points, and a measurement of the value of some function at these
points (with an error), we want to determine the unknown function. The least
squares-method is one popular way to achieve this.
Solving equations and systems of equations[edit]
Another fundamental problem is computing the solution of some given
equation. Two cases are commonly distinguished, depending on whether the

equation is linear or not. For instance, the equation is linear while is

not.
Much effort has been put in the development of methods for solving systems
of linear equations. Standard direct methods, i.e., methods that use
some matrix decomposition are Gaussian elimination, LU
decomposition, Cholesky decomposition for symmetric (or hermitian)
and positive-definite matrix, and QR decomposition for non-square
matrices. Iterative methods such as the Jacobi method, Gauss–Seidel
method, successive over-relaxation and conjugate gradient method are usually
preferred for large systems. General iterative methods can be developed using
a matrix splitting.
Root-finding algorithms are used to solve nonlinear equations (they are so
named since a root of a function is an argument for which the function yields
zero). If the function is differentiable and the derivative is known,
then Newton's method is a popular choice. Linearization is another technique
for solving nonlinear equations.
Solving eigenvalue or singular value problems[edit]
Several important problems can be phrased in terms of eigenvalue
decompositions or singular value decompositions. For instance, the spectral
image compression algorithm[4] is based on the singular value decomposition.
The corresponding tool in statistics is called principal component analysis.
Optimization[edit]
Main article: Mathematical optimization
Optimization problems ask for the point at which a given function is maximized
(or minimized). Often, the point also has to satisfy some constraints.
The field of optimization is further split in several subfields, depending on the
form of the objective function and the constraint. For instance, linear
programming deals with the case that both the objective function and the
constraints are linear. A famous method in linear programming is the simplex
method.
The method of Lagrange multipliers can be used to reduce optimization
problems with constraints to unconstrained optimization problems.
Evaluating integrals[edit]
Main article: Numerical integration
Numerical integration, in some instances also known as numerical quadrature,
asks for the value of a definite integral. Popular methods use one of
the Newton–Cotes formulas (like the midpoint rule or Simpson's rule)
or Gaussian quadrature. These methods rely on a "divide and conquer"
strategy, whereby an integral on a relatively large set is broken down into
integrals on smaller sets. In higher dimensions, where these methods become
prohibitively expensive in terms of computational effort, one may use Monte
Carlo or quasi-Monte Carlo methods (see Monte Carlo integration), or, in
modestly large dimensions, the method of sparse grids.
Differential equations[edit]
Main articles: Numerical ordinary differential equations and Numerical partial
differential equations
Numerical analysis is also concerned with computing (in an approximate way)
the solution of differential equations, both ordinary differential equations
and partial differential equations.
Partial differential equations are solved by first discretizing the equation,
bringing it into a finite-dimensional subspace. This can be done by a finite
element method, a finite differencemethod, or (particularly in engineering)
 a finite volume method. The theoretical justification of these methods often
involves theorems from functional analysis. This reduces the problem to the
solution of an algebraic equation.

Software[edit]
Main articles: List of numerical analysis software and Comparison of numerical
analysis software
Since the late twentieth century, most algorithms are implemented in a variety
of programming languages. The Netlib repository contains various collections
of software routines for numerical problems, mostly in Fortran and C.
Commercial products implementing many different numerical algorithms
include the IMSL and NAG libraries; a free-software alternative is the GNU
Scientific Library.
There are several popular numerical computing applications such
as MATLAB, TK Solver, S-PLUS, and IDL as well as free and open source
alternatives such as FreeMat, Scilab, GNU Octave (similar to Matlab),
and IT++ (a C++ library). There are also programming languages such
as R (similar to S-PLUS) and Python with libraries such
as NumPy, SciPy and SymPy. Performance varies widely: while vector and
matrix operations are usually fast, scalar loops may vary in speed by more than
an order of magnitude.[5][6]
Many computer algebra systems such as Mathematica also benefit from the
availability of arbitrary precision arithmetic which can provide more accurate
results.
Also, any spreadsheet software can be used to solve simple problems relating
to numerical analysis.

Analytical vs Numerical Solutions

In mathematics, some problems can be solved analytically and numerically.

 An analytical solution involves framing the problem in a well-understood form and calculating the exact
solution.
 A numerical solution means making guesses at the solution and testing whether the problem is solved well
enough to stop.
An example is the square root that can be solved both ways.

We prefer the analytical method in general because it is faster and because the solution is exact.
Nevertheless, sometimes we must resort to a numerical method due to limitations of time or hardware
capacity.

A good example is in finding the coefficients in a linear regression equation that can be calculated
analytically (e.g. using linear algebra), but can be solved numerically when we cannot fit all the data into
the memory of a single computer in order to perform the analytical calculation (e.g. via gradient descent).
 Sometimes, the analytical solution is unknown and all we have to work with is the numerical approach.

Analytical Solutions
Many problems have well-defined solutions that are obvious once the problem has been defined.

A set of logical steps that we can follow to calculate an exact outcome.

For example, you know what operation to use given a specific arithmetic task such as addition or
subtraction.

In linear algebra, there are a suite of methods that you can use to factorize a matrix, depending on if the
properties of your matrix are square, rectangular, contain real or imaginary values, and so on.
We can stretch this more broadly to software engineering, where there are problems that turn up again
and again that can be solved with a pattern of design that is known to work well, regardless of the specifics
of your application. Such as the visitor pattern for performing an operation on each item in a list.
Some problems in applied machine learning are well defined and have an analytical solution.

For example, the method for transforming a categorical variable into a one hot encoding is simple,
repeatable and (practically) always the same methodology regardless of the number of integer values in
the set.
Unfortunately, most of the problems that we care about solving in machine learning do not have analytical
solutions.

Numerical Solutions
There are many problems that we are interested in that do not have exact solutions.

Or at least, analytical solutions that we have figured out yet.

We have to make guesses at solutions and test them to see how good the solution is. This involves framing
the problem and using trial and error across a set of candidate solutions.

In essence, the process of finding a numerical solution can be described as a search.

These types of solutions have some interesting properties:

 We often easily can tell a good solution from a bad solution.

 We often don’t objectively know what a “good” solution looks like; we can only compare the goodness
between candidate solutions that we have tested.
 We are often satisfied with an approximate or “good enough” solution rather than the single best solution.
This last point is key, because often the problems that we are trying to solve with numerical solutions are
challenging (as we have no easy way to solve them), where any “good enough” solution would be useful. It
also highlights that there are many solutions to a given problem and even that many of them may be good
enough to be usable.
Most of the problems that we are interested in solving in applied machine learning require a numerical
solution.
It’s worse than this.

The numerical solutions to each sub-problem along the way influences the space of possible solutions for
subsequent sub-problems.

Numerical Solutions in Machine Learning

Applied machine learning is a numerical discipline.

The core of a given machine learning model is an optimization problem, which is really a search for a set of
terms with unknown values needed to fill an equation. Each algorithm has a different “equation” and
“terms“, using this terminology loosely.
The equation is easy to calculate in order to make a prediction for a given set of terms, but we don’t know
the terms to use in order to get a “good” or even “best” set of predictions on a given set of data.
This is the numerical optimization problem that we always seek to solve.

It’s numerical, because we are trying to solve the optimization problem with noisy, incomplete, and error-
prone limited samples of observations from our domain. The model is trying hard to interpret the data and
create a map between the inputs and the outputs of these observations

Analytical is exact, numerical is approximate. For example, some differential equations cannot be solved
exactly (analytic or closed form solution) and we must rely on numerical techniques to solve them.Dec 16,
2015