Type equation here .

TABLE CONTENT
Acknowledgement………………………………………………………………………….i
Abstract………………………………………………………………………………..…...ii
CHAPTER ONE………………………………………………………………………...….1
INTRODUCTION……………………………………………………………….….……..1
1.1 Overview of Eigenvalues…………………………………………………….………...1
1.2 Importance of Eigenvalues………………………………………………….………….1
CHAPTER TWO…………………………………………………………………………..2
Mathematical Foundations…………………………………………………………………2
2.1 The Characteristic Equation…………………………………………………………....2
2.2 Numerical Eigenvalue Decomposition ……………………………………………...2
2.3 Eigenvalues and Matrix Properties…………………………………………………….3
CHAPTER THREE ……………………………………………………………………….4
The Power Method………………………………………………………………………...4
3.1 Introduction to the Power Method………………………………………………….….4
3.2 Steps of the Power Method………………………………………………………….....4
3.3 Convergence Analysis………………………………………………………………....6
3.4 Limitations of the Power Method……………………………………………………..7
CHAPTER FOUR………………………………………………………………………...9
Solving Numerical Problems Using the Power Method…………………………………..9
4.1 Computing the Dominant Eigenvalue…………………………………………………9
4.2 Slow Convergence with Close Eigenvalues…………………………………………..11
4.3 Real-World Application……………………………………………………..………..13
CHAPTER FIVE …………………………………………………………………………15
Applications in Numerical Analysis……………………………………….……………..15
5.1 Iterative Solvers for Linear Systems………………………………………………….15
5.2 Stability Analysis of Numerical Schemes………………………………….…………16
5.3 Dimensional Reduction in Data Science……………………………..……………….17
5.4 Computational Modeling…………………………………………………….……….18
CHAPTER SIX………………………………………………………………...………...19
Advanced Numerical Techniques and Extensions……………………………………….19
6.1 Accelerating the Power Method……………………………………………………...19
6.2 Finding Multiple Eigenvalues………………………………………………………..20
6.3 Generalizations of the Power Method…………………………………………..……21
CHAPTER SEVEN ……………………………………………………………………...22
Conclusion………………………………………………………………………………..22
References………………………………………………………………………………..23

Acknowledgment

First of all I want to thanks GOD for his protection and guidance to prepare this
seminar successfully. Next to this I would like to express us thanks to the Department of
Applied Mathematics, Adama Science and Technology University, for providing me with the
necessary knowledge, assistance and facilities to conduct my seminar work. Next, I would
like to express-my deep appreciation to my advisor, Mesfin Zewude (Phd) for his enthusiasm,
guidance and constant encouragement throughout the seminar period. His regular advice and
suggestion made me work easier and proficient. I really appreciate the time he has taken to
supervise me on skill and knowledge, thank you once again. Last but not least, a special word
of thanks also goes tomy family and friends for their continuous and unconditional support,
love and encouragement throughout the progress of this seminar
 Abstract

Mathematics plays an important role in our everyday life. Fixed point iteration theory is a
fascinating subject, with an enormous number of applications in various fields of
mathematics.Maybe due to this transversal character, I have always experienced some
difficulties to find a book (unless expressly devoted to fixed points) treating the argument in a
unitary fashion. Inmost cases, It is noticed that fixed points pop up when they are needed. On
the contrary, It is believed that they should deserve a relevant place in any general textbook,
and particularly, in a functional analysis textbook. This is mainly the reason that made me
decide to write down these notes. Ii is tried to collect most of the significant results of the
field, and then to present various related applications
 CHAPTER ONE

INTRODUCTION
Eigenvalue is a scalar value associated with a square matrix (or a linear transformation) that
indicates how much a corresponding eigenvector is stretched or compressed when the matrix
is applied to it. In simple terms, an eigenvalue tells you how much the matrix "scales" its
eigenvector, without changing its direction.
Eigenvalues refer to a set of scalar values associated with a square matrix or a linear
transformation. Eigenvalues play a crucial role in understanding the behavior of a matrix and
its transformations. They are primarily used in areas like solving systems of linear equations,
matrix diagonalization, stability analysis, and optimization, among others.

1.1 Overview of Eigenvalues

Eigenvalues are the special set of scalars associated with the system of linear equations. It is
mostly used in matrix equations. ‘Eigen’ is a German word that means ‘proper’ or
‘characteristic’. Therefore, the term eigenvalue can be termed as characteristic
value, characteristic root, proper values or latent roots as well. In simple words, the
eigenvalue is a scalar that is used to transform the eigenvector. The basic equation is
Ax = λx
A: This is a square matrix (of size n×n ) that represents a linear transformation or linear
operator.
x: is a vector
λ: This is a scalar (a real or complex number) called the eigenvalue.
The number or scalar value “λ” is an eigenvalue of A

1.2 Importance of Eigenvalues

Eigenvalues are fundamental in many areas of mathematics, physics, and engineering

because they provide essential insights into the properties and behaviors of linear
transformations, matrices, and systems. Their importance extends across diverse fields, from
solving systems of differential equations to machine learning and quantum mechanics. Here's
an overview of why eigenvalues are so crucial.

CHAPTER TWO
 Mathematical Foundations

Eigenvalues arise from the study of linear transformations and matrices, and their calculation
involves solving the characteristic equation det(A−λI)=0
The roots of this equation are the eigenvalues, and each eigenvalue corresponds to a non-zero
eigenvector that is scaled by the matrix. The properties and applications of eigenvalues are
vast, influencing areas from stability analysis to machine learning and quantum mechanics.

2.1 The Characteristic Equation

The characteristic equation forms the mathematical basis for finding eigenvalues.

 Derivation: For a square matrix AA, the eigenvalues are the roots of det(A−λI)=0 where
λ\ lambda is a scalar, and I is the identity matrix.
 Numerical Significance: Solving the characteristic polynomial can be computationally
expensive for large matrices. Direct computation is avoided for matrices of high order
due to potential numerical instability and inefficiency.
 Practical Approach: Iterative methods, such as the Power Method, provide approximate
solutions to dominant eigenvalues without explicitly solving this polynomial.

2.2 Numerical Eigenvalue Decomposition

Definition: The equation A=V ∆ V − 1 represents the Eigenvalue Decomposition (or

Spectral Decomposition) of a matrix A. This is a way of expressing the matrix A in terms
of its eigenvectors and eigenvalues.

For a square matrix A (usually n×n) that is diagonalizable, the eigenvalue decomposition
expresses A as a product of three matrices:
A=V ∆ V − 11

where:

 V is an invertible matrix whose columns are the eigenvectors of A.

 ∆ is a diagonal matrix containing the eigenvalues of A along its diagonal.
V is the inverse of the matrix V.
−1


Numerical Challenges:
Computing eigenvalue decomposition directly for large matrices is often computationally
intensive.

Iterative methods are preferred for approximate solutions, especially when matrices are
sparse or structured.

Applications: Eigenvalue decomposition aids in solving systems of equations, matrix

inversion, and stability analysis.

2.3 Eigenvalues and Matrix Properties

 Trace and Determinant:

o The trace of A equals the sum of its eigenvalues.

o The determinant of A equals the product of its eigenvalues.

 Matrix Behavior :

o Invertibility: A matrix is invertible if none of its eigenvalues are zero.

o Stability: In iterative processes, the spectral radius (largest absolute
eigenvalue) governs the convergence.
 Chapter 3:

The Power Method

3.1 Introduction to the Power Method

The power method is a fundamental technique in numerical analysis for computing the
dominant eigenvalue and eigenvector of a matrix. It's an iterative approach that repeatedly
multiplies a matrix by a vector, converging to the largest eigenvalue in magnitude.

This method forms the basis for more advanced eigenvalue algorithms and has wide-ranging
applications. From structural engineering to quantum mechanics, the power method helps
solve complex problems by approximating a matrix's most influential characteristics
efficiently.

3.2 Steps of the Power Method

Basic power iteration

 Starts with an initial guess vector x0x0 and iteratively applies the matrix AA to it
 Normalizes the resulting vector after each iteration to prevent overflow or underflow
 Computes the Rayleigh quotient Xk T AXk to estimate the eigenvalue at each step
T
Xk Xk
 Continues until the change in the eigenvalue estimate falls below a specified tolerance

3.3 Convergence Analysis

 Critical aspect of numerical analysis, focusing on the efficiency and accuracy of

iterative methods
 Provides insights into the algorithm's performance and helps in selecting appropriate
stopping criteria
 Guides the development of more advanced eigenvalue computation techniques

Rate of convergence

 Determined by the ratio of the magnitudes of the two largest eigenvalues ∣λ2/λ1∣
 Linear convergence achieved when this ratio is less than 1
 Slower convergence occurs when the ratio is close to 1, indicating closely spaced
eigenvalues
 Affects the number of iterations required to reach a desired level of accuracy

3.4 Limitations of the Power Method

 Convergence is Slow for Close Eigenvalues:
Requires an Appropriate Initial Vector
Provides a balanced view of the power method's strengths and weaknesses in numerical
analysis
Guides the selection of appropriate eigenvalue computation techniques for specific
problem types
Highlights the trade-offs between simplicity, efficiency, and robustness in numerical
algorithms

CHAPTER FOUR

Solving Numerical Problems Using the Power Method

4.1 Computing the Dominant Eigenvalue

Determine the dominant eigenvalue of A = [ 13 24] using power method
Solution Let the initial approximation to the eigenvector be v0. Then, the power method is
given by
yk+1 = Avk,
vk+1 = yk+1/mk+1

where mk+1 is the largest element in magnitude of yk+1. The dominant eigenvalue in
magnitude is given by

( Yk +1 ) r ,r = 1,2,….,n
λ 1= lim
k →∞ ( Vk ) r

and vk+1 is the required eigen vector.

Let v₀ = [1 1]ᵀ.
We have the ff Type equation here . results
y1 = Av0 = [ ][ ] [ ] [
1 2 1
3 4 1
=
3
7
=7
0.42857
1 ]
So v1 =[ 0.42857
1 ]

y2 = Av1 =[
3 4 ] [ 1 ] [ 5.28571 ]
= 5.28571[
1 ]
1 2 0.42857 2.28571 0.45946
=

So v2 =[ 0.4594
1
6
]
y3 = Av2 =[ ] [ ] =[ ] =5.37838 [
1 ]
1 2 0.45946 2.45946 0.45729
3 4 1 5.37187

So v3 = [
1 ]
0.45729

y4 = Av3 = [ 13 24] [ 0.45729

1 ] =[
2.45729
5.37187 ] =¿5.37187
[ 1 ]
0.45744

So v4 = [
1 ]
0.45744

Y5 = Av4 = [ ] [ ] =[ ] = 5.37232[
1 ]
1 2 0.45744 2.45744 0.45743
3 4 1 5.37232

So v5 = [
1 ]
0.45743
Y6 = Av5 = [ 13 24] [ 0.45743
1 ] =[
2.45743
5.37229 ] = 5.37229[
1 ]
0.45743

So v6 = [
1 ]
0.45743

Finally, we find the dominant eigenvalue: λ₁ = lim (k→∞) [(yₖ+1)ᵣ /(vₖ)ᵣ], r = 1, 2.

We obtain the ratios as

2 . 45743
. = 5.37225 , 5.37229.
0 . 45743

The magnitude of the error between the ratios is | 5.37225 – 5.37229 | = 0.00004 < 0.00005.

Hence, the dominant eigenvalue, correct to four decimal places is 5.3722.

4.2 Slow Convergence with Close Eigenvalues

When eigenvalues of a matrix are close in magnitude, the Power Method, a commonly used
iterative algorithm to compute the dominant eigenvalue and eigenvector, can converge very
slowly. This happens because the ratio between the dominant eigenvalue and the next largest
eigenvalue (the spectral gap) significantly influences the rate of convergence.

Why Slow Convergence Happens

Spectral Gap:

In the Power Method, the convergence rate depends on the ratio:

Rate of Convergence ∝∣λ2/λ1∣

.where λ1 is the dominant eigenvalue andλ2 is the second-largest eigenvalue (in absolute
magnitude). If ∣λ2∣is close to ∣λ1∣, the convergence is slow because the difference between
successive approximations of the eigenvector becomes small

Components in Initial Guess:

If the initial guess for the eigenvector has significant components aligned with both λ1 and λ2
, the method struggles to suppress the influence of λ2 over iterations.
.
4.3 Real-World Application
  Demonstrates the practical relevance of eigenvalue problems in various scientific and
engineering fields
 Illustrates how fundamental numerical techniques can be applied to solve complex
real-world problems
 Provides a bridge between theoretical concepts and their implementation in
computational algorithms and it’s used in many application

example

1 Google PageRank Algorithm

 Problem: Ranking web pages by importance.

 Application: The PageRank algorithm models the web as a graph where each page is
a node, and hyperlinks are edges. It uses a stochastic matrix (transition matrix) to
represent link probabilities.
 Role of Power Method: The dominant eigenvector of this matrix provides the
PageRank scores for all pages. The Power Method is employed due to its efficiency in
handling large sparse matrices.
 Impact: This algorithm revolutionized search engines, enabling Google to rank web
pages effectively.

2. Principal Component Analysis (PCA)

 Problem: Dimensionality reduction and feature extraction in datasets.

 Application: PCA identifies the directions (principal components) of maximum
variance in data. The first principal component corresponds to the dominant
eigenvector of the covariance matrix.
 Role of Power Method: Efficient computation of the largest eigenvector for high-
dimensional data.
 Impact: Widely used in machine learning, image processing, and data analytics for
reducing complexity and noise.

3. Structural Engineering and Vibrations

 Problem: Understanding the natural vibration modes of buildings, bridges, and

machines.
 Application: The eigenvalues and eigenvectors of stiffness or mass matrices provide
information about natural frequencies and mode shapes of structures.
  Role of Power Method: Determines the fundamental mode (largest eigenvalue) of
vibration, which is critical for stability analysis.
 Impact: Helps in designing safe structures that can withstand vibrations and external
forces.

4. Network Analysis and Social Networks

 Problem: Identifying influential nodes or users in a network.

 Application: Eigenvector centrality measures the influence of a node in a network by
considering both direct and indirect connections.
 Role of Power Method: Computes the dominant eigenvector of the adjacency matrix
to assign centrality scores.
 Impact: Used in analyzing social networks (e.g., Facebook, LinkedIn), biological
networks, and transportation systems.

5. Markov Chains and Stochastic Processes

 Problem: Finding steady-state probabilities in stochastic systems.

 Application: Markov chains describe systems where the future state depends only on
the current state. The steady-state distribution corresponds to the dominant
eigenvector of the transition matrix.
 Role of Power Method: Computes the steady-state probabilities efficiently,
especially for large systems.
 Impact: Applications in inventory management, weather prediction, and economics.

6. Quantum Mechanics and Physics

 Problem: Determining the ground state of quantum systems.

 Application: The Hamiltonian matrix represents the energy states of a quantum
system. The ground state corresponds to the dominant eigenvector.
 Role of Power Method: Approximation of the ground state and energy eigenvalues
for large systems.
 Impact: Used in quantum computing, material science, and molecular simulations.

7. Image Compression

 Problem: Reducing the storage requirements for images while retaining quality.
  Application: Image compression techniques like Singular Value Decomposition
(SVD) involve finding eigenvectors of image matrices.
 Role of Power Method: Efficient computation of dominant singular values and
corresponding vectors.
 Impact: Enables efficient storage and transmission of images (e.g., JPEG
compression).

8. Recommendation Systems

 Problem: Predicting user preferences in online platforms (e.g., Netflix, Amazon).

 Application: Matrix factorization techniques (e.g., collaborative filtering) involve
computing eigenvectors to identify latent features of users and items.
 Role of Power Method: Approximation of dominant eigenvectors in user-item
interaction matrices.
 Impact: Improves the accuracy and efficiency of recommendation systems.

9. Financial Modeling

 Problem: Risk analysis and portfolio optimization.

 Application: Covariance matrices in financial data analysis are used to study
correlations between asset returns. The dominant eigenvalue captures the most
significant variance.
 Role of Power Method: Helps in identifying dominant market factors and optimizing
investment portfolios.
 Impact: Assists in managing risk and maximizing returns.

10. Power Systems and Electrical Engineering

 Problem: Stability analysis of power grids.

 Application: Eigenvalue analysis of system matrices determines the stability and
oscillation modes of power systems.
 Role of Power Method: Identifies the dominant eigenvalue and eigenvector related
to critical system behaviors.
 Impact: Ensures reliability and efficiency in power generation and distribution.
 CHAPTER FIVE

Applications in Numerical Analysis

5.1 Iterative Solvers for Linear Systems

Iterative solvers for linear systems are numerical methods used to approximate the solution
of large systems of linear equations, especially when direct methods (e.g., Gaussian
elimination, LU ,Gauss Jacobi and Gauss seidel decomposition and etc ) become
computationally expensive or impractical. These methods are particularly effective for sparse
or structured matrices, such as those arising in engineering, physics, and computational
science

E. G
Jacobi iterative method
.In the Jacobi method, an initial (first) value is assumed for each of the
unknowns, X 1(1) , x 2(2),…, xn(n)
If no information is available regarding the approximate values of the unknown, the initial
value of all the unknowns can be assumed to be zero.

Gauss- Seidel iterative method

In the Gauss-Seidel method, initial (first) values are assumed for the
unknowns x2, x3, .. ., xn (all of the unknowns except x1).
If no information is available regarding the approximate value of the unknowns, the
initial value of all the unknowns can be assumed to be zero.

5.2 Stability Analysis of Numerical Schemes

In numerical analysis, stability of a numerical scheme refers to the ability of the scheme to
produce bounded solutions over time (for time-stepping methods) or during iterations (for
iterative methods). When solving differential equations or other mathematical problems
numerically, stability is crucial to ensuring that small errors introduced during computation
do not grow uncontrollably, leading to incorrect results. The concept of stability applies to a
wide range of numerical methods, from finite difference methods for solving partial
differential equations (PDEs) to iterative solvers for linear systems.
There are different ways to formalize the concept of stability. The following definitions of
forward, backward, and mixed stability are often used in numerical linear algebra

5.3 Dimensional Reduction in Data Science

Dimensionality reduction is a crucial concept in data science and numerical analysis,

especially when dealing with high-dimensional datasets. The primary goal is to reduce the
number of variables or features in the data while retaining as much relevant information as
possible. This can help improve model performance, make data visualization easier, and
reduce computational costs.

In the context of numerical analysis, dimensionality reduction techniques often rely on

mathematical concepts such as linear algebra, matrix decomposition, and optimization. Here's
an overview of dimensionality reduction, focusing on its relationship with numerical analysis:

1. Understanding Dimensionality Reduction

Dimensionality reduction techniques aim to transform data from a high-dimensional space

(many features or variables) into a lower-dimensional space while preserving the essential
patterns or structures. This is often necessary because high-dimensional data can be
computationally expensive, hard to visualize, and prone to overfitting.

2. Numerical Analysis Concepts Used in Dimensionality Reduction

Numerical analysis plays a vital role in the development and implementation of
dimensionality reduction methods. Some of the core concepts from numerical analysis that
are used in dimensionality reduction include:

a. Linear Algebra

 Eigenvectors and Eigenvalues: Many dimensionality reduction techniques, such as

Principal Component Analysis (PCA), rely on eigenvectors and eigenvalues of
matrices (covariance or correlation matrices). These are foundational concepts in
linear algebra.
o In PCA, the eigenvectors correspond to the principal components (directions
in which the data varies the most), and the eigenvalues represent the amount of
variance captured by each principal component.
 Matrix Decomposition: Techniques like Singular Value Decomposition (SVD)
decompose matrices into simpler components, which is widely used in dimensionality
reduction methods like PCA and Latent Semantic Analysis (LSA).
 QR Decomposition: Some dimensionality reduction methods (such as some
variations of PCA) rely on QR decomposition, a technique used to factorize a matrix
into an orthogonal matrix and an upper triangular matrix.

b. Matrix Factorization

 Techniques like Non-negative Matrix Factorization (NMF) and Independent

Component Analysis (ICA) decompose data matrices into products of smaller
matrices with specific constraints (e.g., non-negativity or statistical independence).
 These matrix factorization techniques often involve solving optimization problems to
minimize reconstruction error or other loss functions.

3. Popular Dimensionality Reduction Techniques in Data Science

Several dimensionality reduction methods in data science are grounded in numerical analysis:

a. Principal Component Analysis (PCA)

Mathematical Foundation: PCA is based on eigenvectors and eigenvalues of the

covariance matrix of the data.

o The first principal component corresponds to the eigenvector with the largest
eigenvalue.
o The second principal component is the eigenvector corresponding to the
second-largest eigenvalue, and so on.
  Numerical Techniques Used: Eigenvalue decomposition of the covariance matrix,
Singular Value Decomposition (SVD).

b. Linear Discriminant Analysis (LDA)

 Objective: LDA is a supervised dimensionality reduction technique used to find a

lower-dimensional representation of the data that maximizes class separability.
 Mathematical Foundation: LDA involves finding the projection that maximizes the
ratio of between-class variance to within-class variance.
 Numerical Techniques Used: Eigenvalue decomposition of the scatter matrices,
solving optimization problems.

c. Autoencoders (Deep Learning)

 Mathematical Foundation: Autoencoders minimize the reconstruction error, which

is the difference between the original and the reconstructed data.
 Numerical Techniques Used: Backpropagation and gradient descent to optimize the
parameters of the neural network.

 5.4 Computational Modeling

Computational modeling is a key concept in numerical analysis, particularly in the context of

solving mathematical problems that are difficult or impossible to solve analytically. It
involves creating mathematical models of real-world systems or phenomena and using
numerical methods to solve these models. These models and solutions are then used to
simulate, analyze, and make predictions about complex systems.

Here are some key aspects of computational modeling related to numerical concept
1. Mathematical Models • Definition: A mathematical model is an abstract representation of
a system using mathematical language.
• Types: •
Deterministic Models
Stochastic Models
2. Numerical Methods • Numerical methods are techniques used to obtain approximate
solutions to mathematical problems that cannot be solved analytically.
Types of numerical method

Finite Difference Method (FDM):

Finite Element Method (FEM) and so on

3. Simulation • Definition: Simulation involves creating a computational model that mimics

the behavior of a real-world process or system over time.
4. Validation and Verification • Validation: Ensures that the model accurately represents
the real-world system it is intended to simulate. •

CHAPTER SIX

Advanced Numerical Techniques and Extensions

6.1 Accelerating the Power Method

Accelerating the Power Method in numerical analysis refers to techniques that enhance the
convergence speed of the Power Method when calculating the dominant eigenvalue and
eigenvector of a matrix. The standard Power Method can be slow, particularly if the dominant
eigenvalue is close to the next largest eigenvalue.
Here are some key concepts and methods used to accelerate the Power Method:

. Eigenvalues and Eigenvectors:

Convergence Rate:
3. Normalization
 Acceleration Techniques

1 Shifted Power Method: Introduces a shift σ to the matrix, effectively transforming the
problem to find the dominant eigenvalue of A - σ I . • This can help bring λ₁ closer to zero,
improving convergence. Formula:


bₖ₊₁ = (A - σ I)⁻¹

2. Deflation: A' = A - λ₁
3. Rayleigh Quotient Iteration

(A - λₖ I)
4. Inverse Power Method

bₖ₊₁ = A⁻¹
5 Chebyshev Acceleration

6.2 Finding Multiple Eigenvalues

Finding multiple eigenvalues (also known as computing the eigenvalues of a matrix) involves
techniques that can efficiently identify not just the largest eigenvalue, but also additional
eigenvalues and their corresponding eigenvectors. Here's a detailed overview of methods for
finding multiple eigenvalues

Finding multiple eigenvalues involves methods like solving the characteristic polynomial,
using iterative algorithms such as QR, Power Method with deflation, subspace iteration,
Lanczos for sparse matrices, and Jacobi for symmetric matrices.

There are finding some way

A, When the linear independent solution

Formula : y = c1e +c2e λ1 t λ2 t

+, ... ,+cn e
λ nt

B If It may happen that a matrix has some “repeated” eigenvalues. That is, the characteristic

equation det( A − λΙ ) =0may have repeated roots

Suppose the nxn matrix P has n real eigenvalues (not necessarily distinct)
λ 1 , λ 2 ,... , λ n∧there are linearly independent corresponding eigenvectors n
Then the general solution¿ can written as