TU M unchen

1. Foundations of Numerics from Advanced Mathematics

Linear Algebra
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 1
TU M unchen
1.1. Linear Algebra
Mathematical Structures
a mathematical structure consists of one or several sets and one or several
operations dened on the set(s)
special elements:
neutral element (of an operation)
inverse element (of some element x)
a group: a structure to add and subtract
a eld: a structure to add, subtract, multiply, and divide
a vector space: a set of vectors over a eld with two operations: scalar
multiplication, addition of vectors, obeying certain axioms (which?)
note: sometimes, the association with classical (geometric) vectors is helpful,
sometimes it is more harmful
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 2
TU M unchen
Exercise Mathematical Structures
Show that the possible manipulations of
the Rubiks Cube with the operation ex-
ecute after are a group.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 3
TU M unchen
Exercise Mathematical Structures Solution
Show that the possible manipulations of
the Rubiks Cube with the operation ex-
ecute after are a group.
Closure: executing any two manipulations after one
another is a Cube minipulation, again.
Associativity: the result of a sequence of three
manipulations is obviously always the same no matter how
you group them (the rst two or the last two together).
Identity: obviously included (just do nothing).
Invertibility: execute a manipulation in backward direction.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 4
TU M unchen
Exercise Mathematical Structures
Show that the rational numbers with the operations + (add) and
(multiply) are a eld.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 5
TU M unchen
Exercise Mathematical Structures Solution
Show that the rational numbers with the operations + (add) and
(multiply) are a eld.
Closure: obviously closed under + and .
Identity: 0 for +, 1 for .
Invertibility: each element q has an inverse q under +
and
1
q
under . The latter holds for all elements except from
the neutral element of +, i.e., 0.
Associativity: well-known for both + and .
Commutativity: also known from school (a + b = b + a,
a b = b a).
Distributivity: dito (a (b + c) = a b + a c).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 6
TU M unchen
Exercise Mathematical Structures
Is the set of N N matrices (N N) matrices with real numbers
as entries over the eld of real numbers a vector space?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 7
TU M unchen
Exercise Mathematical Structures Solution
Is the set of N N matrices (N N) matrices with real numbers
as entries over the eld of real numbers a vector space?
The answer is yes. Look up the axioms and show that they hold
for the xample on your own.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 8
TU M unchen
Vector Spaces
a linear combination of vectors
linear (in)dependence of a set of vectors
the span of a set of vectors
a basis of a vector space
denition?
why do we need a basis?
is a vectors basis representation unique?
is there only one basis for a vector space?
the dimension of a vector space
does innite dimensionality exist?
important applications:
(analytic) geometry
numerical and functional analysis: function spaces are vector spaces
(frequently named after mathematicians: Banach spaces, Hilbert spaces,
Sobolev spaces, ...)
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 9
TU M unchen
Exercise Vector Spaces
Is the set of vectors
__
1
0
_
,
_
0
1
_
,
_
1
3
__
linearly
independent?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 10
TU M unchen
Exercise Vector Spaces Solution
Is the set of vectors
__
1
0
_
,
_
0
1
_
,
_
1
3
__
linearly
independent?
The set of vectors is not linearly independent, since the third
element can easily be written as a linear combination of the rst
two:
_
1
3
_
= 1
_
1
0
_
+ 3
_
0
1
_
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 11
TU M unchen
Exercise Vector Spaces
span
__
1
0
0
_
,
_
0
0
1
__
= ?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 12
TU M unchen
Exercise Vector Spaces Solution
span
__
1
0
0
_
,
_
0
0
1
__
=
__
a
0
b
_
; a, b R
_
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 13
TU M unchen
Exercise Vector Spaces
Consider the set of all possible polynomials with real
coefcients as a vector space over the eld of real numbers.
Whats the dimension of this space? Give a basis.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 14
TU M unchen
Exercise Vector Spaces Solution
Consider the set of all possible polynomials with real
coefcients as a vector space over the eld of real numbers.
Whats the dimension of this space? Give a basis.
The space is innite dimensional, a basis is for example
_
1, x, x
2
, x
3
, . . .
_
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 15
TU M unchen
Linear Mappings
denition in the vector space context; notion of a homomorphism
image and kernel of a homomorphism
matrices, transposed and Hermitian of a matrix
relations of matrices and homomorphisms
meaning of injective, surjective, and bijective for a matrix; rank of a matrix
meaning of the matrix columns for the underlying mapping
matrices and systems of linear equations
basis transformation and coordinate transformation
mono-, epi-, iso-, endo-, and automorphisms
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 16
TU M unchen
Exercise Linear Mappings
Is the mapping f : R
3
R
3
,

x 5

x +
_
1
2
3
_
linear?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 17
TU M unchen
Exercise Linear Mappings Solution
Is the mapping f : R
3
R
3
,

x 5

x +
_
1
2
3
_
linear?
f is not linear, since f (

x) = f (

x).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 18
TU M unchen
Exercise Linear Mappings
Whats the linear mapping f : R
2
R
2
corresponding to the
matrix
_
4 0
3 2
_
?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 19
TU M unchen
Exercise Linear Mappings Solution
Whats the linear mapping f : R
2
R
2
corresponding to the
matrix
_
4 0
3 2
_
?
f
__
x
y
__
=
_
4x
3x + 2y
_
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 20
TU M unchen
Exercise Linear Mappings
Give the rank of the matrix
_
_
_
1 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
_
_
_
.
Is the corresponding linear mapping injective, surjective,
bijective?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 21
TU M unchen
Exercise Linear Mappings Solution
Give the rank of the matrix
_
_
_
1 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
_
_
_
.
Is the corresponding linear mapping injective, surjective,
bijective?
The rank is three. Thus, the corresponding linear mapping is
neither injective, nor surjective or bijective.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 22
TU M unchen
Examples Linear Mappings
Monomorphism:
_
0 1
0 0
1 0
_
Epimorphism:
_
1 0 0
0 1 0
0 0 0
_
Iso-/Automorphism:
_
0 1
1 0
_
Endomorphism:
_
2 1 0
0 1 2
1 0 1
_
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 23
TU M unchen
Determinants
denition
properties
meaning
occurrences
Cramers rule
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 24
TU M unchen
Determinants Denition
det(A) =

a
1,1
a
1,2
a
1,N
a
2,1
a
2,2
a
2,N
.
.
.
.
.
.
.
.
.
a
N,1
a
N,N

=
a
1,1

a
2,2
a
2,N
.
.
.
.
.
.
.
.
.
.
.
.
a
N,2
a
N,N

a
1,2

a
2,1
a
2,3
a
2,N
a
3,1
a
3,N
.
.
.
.
.
.
a
N,1
a
N,3
a
N,N

+ . . .
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 25
TU M unchen
Exercise Determinants
det(A) = 0 A denes a . . .morphism.
det(A) = 0 A denes a . . .morphism.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 26
TU M unchen
Exercise Determinants Solution
det(A) = 0 A denes an Endomorphism.
det(A) = 0 A denes an Automorphism.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 27
TU M unchen
Exercise Determinants
det(A B) =?
det
_
A
1
_
=?
det
_
A
T
_
=?
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 28
TU M unchen
Exercise Determinants Solution
det(A B) = det(A) det(B).
det
_
A
1
_
= det(A)
1
.
det
_
A
T
_
= det(A).
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 29
TU M unchen
Exercise Determinants
Determine the solution of the linear system
2x
1
+ x
2
= 4
2x
2
+ x
3
= 0
x
1
+ x
2
+ x
3
= 3
with the help of determinants.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 30
TU M unchen
Exercise Determinants Solution
Determine the solution of the linear system
2x
1
+ x
2
= 4
2x
2
+ x
3
= 0
x
1
+ x
2
+ x
3
= 3
with the help of determinants.
x
1
=

4 1 0
0 2 1
3 1 1

2 1 0
0 2 1
1 1 1

=
7
3
; x
2
=

2 4 0
0 0 1
1 3 1

2 1 0
0 2 1
1 1 1

=
7
3
; x
3
=

2 1 4
0 2 0
1 1 3

2 1 0
0 2 1
1 1 1

=
4
3
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 31
TU M unchen
Eigenvalues
notions of eigenvalue, eigenvector, and spectrum
similar matrices A, B:
S : B = SAS
1
(i.e.: A and B as two basis representations of the same endomorphism)
resulting objective: look for the best / cheapest representation (diagonal form)
important: matrix A is diagonalizable iff there is a basis consisting of
eigenvectors only
characteristic polynomial, its roots are the eigenvalues
Jordan normal form
important:
spectrum characterizes a matrix
many situations / applications where eigenvalues are crucial
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 32
TU M unchen
Exercise Eigenvalues
Diagonalize the matrix
_
3 2
2 3
_
. Give both eigenvalues and
eigenvectors and the basis transformation matrix transforming
the given matrix in diagonal form.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 33
TU M unchen
Exercise Eigenvalues Solution
Diagonalize the matrix
_
3 2
2 3
_
. Give both eigenvalues and
eigenvectors and the basis transformation matrix transforming
the given matrix in diagonal form.
Eigenvalues:

3 2
2 3

= 9 6 +
2
4 = 5 6 +
2

1,2
=
6

3620
2
= 3 2
1
= 5,
2
= 1.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 34
TU M unchen
Exercise Eigenvalues Solution
Diagonalize the matrix
_
3 2
2 3
_
. Give both eigenvalues and
eigenvectors and the basis transformation matrix transforming
the given matrix in diagonal form.
Eigenvector for
1
= 5:

2 2
2 2

x
y

0
0

x = y

x
1
=

1
1

Eigenvector for
2
= 1:

2 2
2 2

x
y

0
0

x = y

x
2
=

1
1

Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics

Linear Algebra, October 23, 2012 35
TU M unchen
Exercise Eigenvalues Solution
Diagonalize the matrix
_
3 2
2 3
_
. Give both eigenvalues and
eigenvectors and the basis transformation matrix transforming
the given matrix in diagonal form.
The basis transformation matrix thus is

1 1
1 1

and results in the diagonal matrix

5 0
0 1

.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 36
TU M unchen
Scalar Products and Vector Norms
notions of a linear form and a bilinear form
scalar product: a positive-denite symmetric bilinear form
examples of vector spaces and scalar products
vector norms:
denition: positivity, homogeneity, triangle inequality
meaning of triangle inequality
examples: Euclidean, maximum, and sum norm
normed vector spaces
Cauchy-Schwarz inequality
notions of orthogonality and orthonormality
turning a basis into an orthonormal one: Gram-Schmidt orthogonalization
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 37
TU M unchen
Exercise Scalar Products and Vector Norms
Are the following operators scalar products in the vector space
of continuous functions on the interval [a; b]?
f , g
1
:=
_
b
a
f (x) g(x)dx
f , g
2
:=
_
b
a
f (x) g(x)
2
dx
f , g
3
:=
_
b
a
f
+
(x) g(x)dx
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 38
TU M unchen
Exercise Scalar Products and Vector Norms Solution
Are the following operators scalar products in the vector space
of continuous functions on the interval [a; b]?
f , g
1
:=
_
b
a
f (x) g(x)dx Yes!
f , g
2
:=
_
b
a
f (x) g(x)
2
dx No! (not linear in g)
f , g
3
:=
_
b
a
f
+
(x) g(x)dx No! (not positive denite)
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 39
TU M unchen
Exercise Scalar Products and Vector Norms
Proof that a set {

x
1
,

x
2
, . . . ,

x
N
} of non-zero orthogonal vectors
in a vector space with scalar product (, ) always is a basis of
its span.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 40
TU M unchen
Exercise Scalar Products and Vector Norms Solution
Proof that a set {

x
1
,

x
2
, . . . ,

x
N
} of non-zero orthogonal vectors
in a vector space with scalar product (, ) always is a basis of
its span.
Proof by contradiction:
Assume that the set is not linearly independent. Then, there is a element

x
i
taht can be
written as a linear combination

x
i
=

kI

k

x
k
of other elements, where the index set
I {1, 2, . . . , N} does not contain i . With this, we get
0 = (

x
i
,

x
i
) =

x
i
,

kI

k

x
k

kI

k
(

x
i
,

x
k
) = 0.
Contradiction. Thus, the vector set is linearly independent and, thus, is a basis of its
span.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 41
TU M unchen
Exercise Scalar Products and Vector Norms
Transform
__
1
1
1
_
,
_
1
1
0
_
,
_
1
0
0
__
into an orthogonal basis
of R
3
.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 42
TU M unchen
Exercise Scalar Products and Vector Norms Solution
Transform
__
1
1
1
_
,
_
1
1
0
_
,
_
1
0
0
__
into an orthogonal basis
of R
3
.
Gram-Schmidt orthogonalization:

x
1
=

1
1
1

,

x
2
=

1
1
0

x
1
,

1
1
0

(
x
1
,x
1
)

x
1
=

1
1
0

2
3

x
1
=

1
3
1
3

2
3

x
3
=

1
0
0

x
1
,

1
0
0

(
x
1
,x
1
)

x
1

x
2
,

1
0
0

(
x
2
,x
2
)

x
2
=

7
15

8
15
1
15

.
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 43
TU M unchen
Matrix Norms
denition:
properties corresponding to those of vector norms
plus sub-multiplicativity:
AB A B
plus consistency
Ax A x
matrix norms can be induced from corresponding vector norms: Euclidean,
maximum, sum
A := max
x=1
Ax
alternative: completely new denition, for example Frobenius norm (consider
matrix as a vector, then take Euclidean norm)
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 44
TU M unchen
Classes of Matrices
symmetric: A = A
T
skew-symmetric: A = A
T
Hermitian: A = A
H
=

A
T
s.p.d. (symmetric positive denite): x
T
Ax > 0 x = 0
orthogonal: A
1
= A
T
(the whole spectrum has modulus 1)
unitary: A
1
= A
H
(the whole spectrum has modulus 1)
normal: AA
T
= A
T
A or AA
H
= A
H
A, resp. (for those and only those matrices
there exists an orthonormal basis of eigenvectors)
Miriam Mehl: 1. Foundations of Numerics from Advanced Mathematics
Linear Algebra, October 23, 2012 45