Solving System of Linear Equations

A system of linear equations consists of two or more linear equations made up of two or more
variables, such that all equations in the system are considered simultaneously. To find the unique
solution to a system of linear equations, we must find a numerical value for each variable in the
system that will satisfy all of the system’s equations at the same time. Some linear systems may
not have a solution, while others may have an infinite number of solutions. In order for a linear
system to have a unique solution, there must be at least as many equations as there are variables.
Even so, this does not guarantee a unique solution.
Example: Here are two linear equations:
2x + y = 5
−x + y = 2
Together they are a system of linear equations.

Can you discover the values of x and y yourself? (Just have a go, play with them a bit.)

2x + y =5 to y=5-2x then substitute to 2nd equation

-x + y = 2 to -x+(5-2x) = 2 -x + 5 – 2x = 2 3x = 3 x=1
Substitute the value of x to any of the two equation 2(1) + y = 5 y=3

Example: You versus Horse

You can run 0.2 km every minute.

The Horse can run 0.5 km every minute. But it takes 6 minutes to saddle the horse.
How far can you get before the horse catches you?
We can make two equations (d=distance in km, t=time in minutes)
You run at 0.2km every minute, so d = 0.2t
The horse runs at 0.5 km per minute, but we take 6 off its time: d = 0.5(t−6)

So we have a system of equations (that are linear):

d = 0.2t
d = 0.5(t−6)
we are looking for how far can you get before the horse catches you?
Equating both your distances we’ll have 0.2t=0.5(t-6) 0.2t=0.5t-3 0.3t=3 t=10
Now we know at time t=10secs, the horse will catch you and by substituting that time to any of
the two equations, we will know the distanced = 0.2t d=0.2(10) d=2km.

system of linear equations: A set of two or more equations made up of two or more variables
that are considered simultaneously.
dependent system: A system of linear equations in which the two equations represent the same
line; there are an infinite number of solutions to a dependent system.
inconsistent system: A system of linear equations with no common solution because they
represent parallel lines, which have no point or line in common.
independent system: A system of linear equations with exactly one solution pair (x,y)

Types of Linear Systems and Their Solutions

In general, a linear system may behave in any one of three possible ways:
1. The system has a single unique solution.
2. The system has no solution.
3. The system has infinitely many solutions.
Each of these possibilities represents a certain type of system of linear equations in two
variables. Each of these can be displayed graphically, as below. Note that a solution to a system
of linear equations is any point at which the lines intersect.
  An independent system has exactly one solution pair (x,y). The point where the two lines
intersect is the only solution.
 An inconsistent system has no solution. Notice that the two lines are parallel and will
never intersect.
 A dependent system has infinitely many solutions. The lines are exactly the same, so
every coordinate pair on the line is a solution to both equations.

Direct Methods of Solving System of Linear Equations

Gausss Elimination method

Solving three-variable, three-equation linear systems is more difficult, at least initially, than
solving the two-variable systems, because the computations involved are messier. You will need
to be very neat in your working, and you should plan to use lots of scratch paper. The method for
solving these systems is an extension of the two-variable solving-by-addition method, so make
sure you know this method well and can use it consistently correctly.
Though the method of solution is based on addition/elimination, trying to do actual addition
tends to get very messy, so there is a systematized method for solving the three-or-more-
variables systems. This method is called "Gaussian elimination" (with the equations ending up in
what is called "row-echelon form").
The goal when solving a system of equations is to place the augmented matrix into reduced row-
echelon form, if possible.
There are three elementary row operations that you may use to accomplish placing a matrix into
reduced row-echelon form.

Each of the requirements of a reduced row-echelon matrix can satisfied using the elementary row
operations.

 If there is a row of all zeros, then it is at the bottom of the matrix.

Interchange two rows of a matrix to move the row of all zeros to the bottom.
 The first non-zero element of any row is a one. That element is called the leading one.
Multiply (divide) the row by a non-zero constant to make the first non-zero element into a one.
 The leading one of any row is to the right of the leading one of the previous row.
Multiply a row by a non-zero constant and add it to another row, replacing that row. The point of
this elementary row operation is to make numbers into zeros. By making the numbers under the
leading ones into zero, it forces the first non-zero element of any row to be to the right of the
leading one of the previous row.
 All elements above and below a leading one are zero.
Multiply a row by a non-zero constant and add it to another row, replacing that row. The point of
this elementary row operation is to make numbers into zero. The difference here is that you're
clearing (making zero) the elements above the leading one instead of just below the leading one.

Let's start simple, and work our way up to messier examples.

Solve the following system of equations.

5x + 4y – z=0
10y – 3z = 11
z=3

5+ 4−1=0
[ 0 10−3=11
0 0 1=3 ]
It's fairly easy to see how to proceed in this case. I'll just back-substitute the z-value from the
third equation into the second equation, solve the result for y, and then plug z and y into the first
equation and solve the result for x.
10y – 3(3) = 11
10y – 9 = 11
10y = 20
y=2

5x + 4(2) – (3) = 0
5x + 8 – 3 = 0
5x + 5 = 0
5x = –5
x = –1

Then the solution is (x, y, z) = (–1, 2, 3).

The reason this system was easy to solve is that the system was "triangular"; this refers to the
equations having the form of a triangle, because of the lower equations containing only the later
variables. system of equations with "triangle" of terms highlighted

Solve the following system using Gaussian elimination:

  

For this system, the augmented matrix (vertical line omitted) is

  

First, multiply row 1 by 1/2:

  

Now, adding −1 times the first row to the second row yields zeros below the first entry in the
first column:

  

Interchanging the second and third rows then gives the desired upper‐triangular coefficient
matrix: 

The third row now says z = 4. Back‐substituting this value into the second row gives y = 1, and
back‐substitution of both these values into the first row yields x = −2. The solution of this system
is therefore (x, y, z) = (−2, 1, 4).

Gauss Elimination with Row Pivoting

The objective of pivoting is to make an element above or below a leading one into a zero.

The "pivot" or "pivot element" is an element on the left hand side of a matrix that you want the
elements above and below to be zero.

Normally, this element is a one. If you can find a book that mentions pivoting, they will usually
tell you that you must pivot on a one. If you restrict yourself to the three elementary row
operations, then this is a true statement.

However, if you are willing to combine the second and third elementary row operations, you
come up with another row operation (not elementary, but still valid).

 You can multiply a row by a non-zero constant and add it to a non-zero multiple of
another row, replacing that row.
So what? If you are required to pivot on a one, then you must sometimes use the second
elementary row operation and divide a row through by the leading element to make it into a one.
Division leads to fractions. While fractions are your friends, you're less likely to make a mistake
if you don't use them.

What's the catch? If you don't pivot on a one, you are likely to encounter larger numbers. Most
people are willing to work with the larger numbers to avoid the fractions.

The Pivot Process

Pivoting works because a common multiple (not necessarily the least common multiple) of two
numbers can always be found by multiplying the two numbers together. Let's take the example
we had before, and clear the first column.

  x y z rh  
s
  3 2 - 3  
4
  2 3 3 15  
  5 - 1 14  
3

Helpful Hints

 Although you do not have to pivot on a one, it is highly desirable. Pivoting on a one
means that you're multiplying by 1 (which is easy to do).
 It is nice to pivot on the main diagonal, but not absolutely necessary. Some people like to
start in the upper left and work their way down to the lower right.
 As long as you pivot only once per row and column, the columns which have been
cleared will remain cleared.
 Since the point of pivoting is to clear the pivot column, picking a column which already
has zeros in it saves time because you don't have to change the row that contains the zero.

Selecting a Pivot

 Pick the column with the most zeros in it.

 Use a row or column only once
 Pivot on a one if possible
 Pivot on the main diagonal
 Never pivot on a zero
 Never pivot on the right hand side

Since there is no one in the first row, we have two options: Either we divide the first row by
three and work with fractions, or we pivot on the three and get large numbers. That is the option
I'm going to use. I'll pivot on the three in R1C1. Go ahead and circle that as the pivot element.
Depending on your browser, you may see the pivot elements circled in red or just with a * in
front of it.

  x y z rh  
s
  *3 2 - 3  
4
  2 3 3 15  
  5 - 1 14  
3

The idea is to make the boxed (yellow) numbers into zero. Using the combined row operation
(this is not an elementary operation), that could be done by 3R2 - 2R1 → R2 and 3R3 - 5R1 → R3.

The only row not being changed is the row containing the pivot element (the 3). The whole point
of the pivot process is to make the boxed values into zero. Go ahead and rewrite the pivot row
and clear (make zero) the pivot column.

  x y z rh  
s
  *3 2 - 3  
4
  0        
  0        

To replace the values in row 2, each new element is obtained by multiplying the element being
replaced in the second row by 3 and subtracting 2 times the element in the first row from the
same column as the element being replaced.

To perform the pivot, place one finger on the pivot (circled number), and one finger on the
element being replaced. Multiply these two numbers together. Now, place one finger on the
boxed number in the same row as the element you're replacing and the other finger in the pivot
row and the same column as the number your replacing. Multiply these two numbers together.
Take the product with the pivot and subtract the product without the pivot.

  x y z rh  
s
  *3 2 - 3  
4
  2 3 3 15  
  5 - 1 14  
3
To replace the 3 in R2C2, you would take 3(3) - 2(2) = 9 - 4 = 5.
To replace the 3 in R2C3, you would take 3(3) - 2(-4) = 9 +8 = 17.
To replace the 15 in R2C4, you would take 3(15) - 2(3) = 45 - 6 = 39.
To replace the -3 in R3C2, you would take 3(-3) - 5(2) = -9 - 10 = -19.
To replace the 1 in R3C3, you would take 3(1) - 5(-4) = 3 + 20 = 23
To replace the 14 in R3C4, you would take 3(14) - 5(3) = 42 - 15 = 27.

Here's how the process looks.

  x y z rhs  
  pivot row, copy pivot row, pivot row, pivot row,  
3 copy copy copy
2 -4 3
  pivot column, 3(3) - 2(2) 3(3) - 2(-4) 3(15) - 2(3)  
clear 5 17 39
0
  pivot column, 3(-3) - 5(2) 3(1) - 5(-4) 3(14) - 5(3)  
clear -19 23 27
0

Or, if you remove the comments, the matrix after the first pivot looks like this.

  x y z rh  
s
  3 2 -4 3  
  0 5 17 39  
  0 - 23 27  
19

It is now time to repeat the entire process. We go through and pick another place to pivot. We
would like it to be on the main diagonal, a one, or have zeros in the column. Unfortunately, we
can't have any of those. But since we have to multiply all the other numbers by the pivot, we
want it to be small, so we'll pivot on the 5 in R2C2 and clear out the 2 and -19.

  x y z rh  
s
  3 2 -4 3  
  0 *5 17 39  
  0 - 23 27  
19

Begin by copying down the pivot row (2nd row) and clearing the pivot column (2nd column).
Previously cleared columns will remain cleared.

  x y z rh  
s
    0      
  0 *5 17 39  
  0 0      

Here are the calculations to find the next interation. Pay careful attention to the 3rd row where
we're subtracting -19 times a value. Since we're subtracting a negative, I went ahead and wrote it
as plus 19.

  x y z rhs  
  5(3) - 2(0) pivot column, 5(-4) - 2(17) 5(3) - 2(39)  
15 clear -54 -63
0
  pivot row, copy pivot row, copy pivot row, pivot row,  
0 5 copy copy
17 39
  previously pivot column, 5(23) + 19(17) 5(27) + 19(39)  
cleared clear 438 876
0 0

And the resulting matrix.

  x y z rhs  
  15 0 -54 -63  
  0 5 17 39  
  0 0 438 876  

Notice that all the elements in the first row are multiples of 3 and all the elements in the last row
are multiples of 438. We'll divide to reduce the rows.

  x y z rh  
 s
  5 0 - -21  
18
  0 5 17 39  
  0 0 1 2  

That had the added benefit of giving us a 1, exactly where we want it to be to pivot. So, we'll
pivot on the 1 in R3C3 and clear out the -18 and 17. Circle your pivot and box the other numbers
in that column to clear.

  x y z rh  
s
  5 0 - -21  
18
  0 5 17 39  
  0 0 *1 2  

Copy down the pivot row and clear the pivot column. Previously cleared columns will remain
cleared as long as you don't pivot in a row or column twice.

  x y z rh  
s
    0 0    
  0   0    
  0 0 *1 2  

Notice that each time, there are fewer calculations to perform. Here are the calculations for this
pivot. Again, since the value in the pivot column in the first row is -18 and we're subtracting, I
wrote it as + 18.

  x y z rhs  
  1(5) +18(0) previously pivot column, 1(-21) + 18(2)  
5 cleared clear 15
0 0
  previously 1(5) - 17(0) pivot column, 1(39) - 17(2)  
cleared 5 clear 5
0 0
  pivot row, copy pivot row, copy pivot row, copy pivot row,  
0 0 1 copy
 2

And the resulting matrix.

  x y z rh  
s
  5 0 0 15  
  0 5 0 5  
  0 0 1 2  

Notice that the first and second rows are multiples of 5, so we can reduce those rows.

  x y z rh  
s
  1 0 0 3  
  0 1 0 1  
  0 0 1 2  

And the final answer is x = 3, y = 1, and z = 2. You can also write that as an ordered triplet
{(3,1,2)}.

Iterative Methods of Solving System of Linear Equations