Practice Problems

Logistic Regression/Classification:

You are given a logistic regression model trained on a binary classification problem where Y=1
indicates success and Y=0 indicates failure. The model uses four regressors: X 1, X2, X3, and X4 .
The estimated logistic regression coefficients (betas) are as follows:

β 0=1.5 , β 1=0.7 , β 2=−0.4 , β 3=0.9 , and β 4 =−0 .5

Data Point X1 X2 X3 X4 Y
1 0.8 1.2 0.5 0.9 1
2 1.5 0.6 1.0 1.2 0
3 1.0 0.7 1.2 1.1 1
4 0.6 1.1 0.9 0.8 0
5 1.2 0.9 1.1 0.7 1
6 0.5 1.3 0.8 0.6 0
7 1.4 0.8 1.3 1.0 1
8 0.9 1.0 0.6 1.1 0
9 1.3 0.5 1.4 0.9 1
10 0.7 1.1 0.7 0.5 0

 For each data point, compute the linear combination z=β 0 + β 1 X 1+ β 2 X 2 + β 3 X 3+ β 4 X 4

 Calculate the predicted probability P(Y=1) for each data point
 Based on P(Y=1), calculate predicted Y ¿) for each data point
 Build the confusion matrix based on the actual and predicted values.

 Calculate accuracy of the model

 Calculate precision, recall, and F-score for both class 1 and class 0

*You can generate more questions by changing coefficient/dataset

Multiple Linear Regression:

We need to build linear regression model to predict Y using X 1, X2, X3, and X4 regressors for
following dataset:

Data Point X1 X2 X3 X4 Y
1 2 3 1 4 10
2 1 2 2 3 8
3 3 1 3 2 12
4 4 2 4 1 14
5 2 3 2 2 11
  Estimate the coefficients β 0, β 1 , β 2 , β 3 ,∧β 4 for linear regression model
 Using the estimated coefficients, calculate the predicted Y ¿) for each data point
 Compute the residuals for each data point

 Calculate SST, SSR, SSE, MST,MSR, MSE, R2 and F-statistics

 Determine the significance of regression for each coefficient using hypothesis testing.
Take standard errors as:
SE(β ¿¿ 0)=0.5 , SE(β ¿¿ 1)=0. 3 , SE(β ¿¿ 2)=0. 2 , SE(β ¿¿ 3)=0. 4 ,∧SE(β ¿¿ 4)=0. 2 5¿ ¿ ¿ ¿ ¿
 Calculate the 95% confidence interval for each coefficient using t-distribution. Use
standard error same as previous part
 Compute the correlation matrix between the regressors as well as Y and all regressors
 Calculate the hat matrix and compute the leverage for each data point
 Calculate the Cook’s distance for each data point

*You can generate more questions by changing dataset

Taylor Series Expansion

Consider the function f(x)=ln(1+x)

 Find the Taylor series expansion of f(x) around x=0 up to the 6th degree term.
 Use the remainder term to estimate the error when x=0.5 approximating ln(1.5) using the
Taylor series up to the 6th degree.
 Verify the result numerically by comparing the Taylor series approximation and the true
value of ln(1.5)

sin( x)
Let f ( x )=
1+ x 2

 Find the Taylor series expansion of f(x) around x=0 up to the 5th degree term.
 Approximate f(0.2) using the Taylor series expansion. What is the error as compared to
actual value?
 Approximate f(1) using the Taylor series expansion. What is the error as compared to
actual value?

Solution of nonlinear equations

Consider the nonlinear equation:

3 2
f ( x )=x −4 x + x−2

Solve for x in the interval [0,3] using the relaxation method, bisection method, and Newton-
Raphson method. Answer the following sub-questions

Relaxation Method:
  Derive the iterative formula for the relaxation method by rewriting the given equation in
the form:

x=g ( x )

Provide two different forms of g(x) that can be used for the iteration and discuss which
one is expected to converge more rapidly

 Choose an appropriate initial guess X0 based on the graph of f(x). Justify your choice.
 Implement the relaxation method using both iterative formulas. Continue iterating until
the absolute difference between successive approximations is less than 0.001
o For each iteration, provide the values of x n and f (x ¿¿ n)¿
o Compare the convergence rate for the two forms of g ( x )

Bisection Method

 Show that f(0) and f(3) have opposite signs

 Perform iterations of the bisection method on the initial interval [0,3]. Continue iterating
until the absolute difference between successive approximations is less than 0.001. For
each iteration, provide:

o The midpoint X m
o The function value f( X m)
o The length of the interval after each iteration

Newton-Raphson

 Perform the iterations of the Newton-Raphson method with initial guess of x 0=1.5 .
Continue iterating until the absolute difference between successive approximations is less
than 0.001. For each iteration, provide:

o The value of x n
o The function value f (x ¿¿ n)¿
o The value of f ' (x ¿¿ n)¿
o The difference | x n+1−x n |

 Use the Newton-Raphson method to identify another root, starting from x 0=2.8

 Compare the convergence speeds of the three methods (relaxation, bisection, and Newton-
Raphson) based on the number of iterations required to reach an error tolerance of 0.001

*You can generate more questions by changing f(x)

Solution of linear equations

You are given the following system of linear equations:
3 x 1+2 x 2 + x3 + 4 x 4 =10
2 x1 +3 x 2 +4 x 3 + x 4 =8
x 1+ 4 x 2 +3 x3 +2 x 4 =11
4 x1 + x 2 +2 x 3 +2 x 4=7

 Out of three methods you learned in the class, which one is best suited here and why?

 Gauss Elimination and Back Substitution:

 Perform the forward elimination step of the Gauss elimination method to reduce the
system to an upper triangular matrix.
 Perform back-substitution to find the values of x 1, x 2, x 3, and x 4
 Verify your solution by substituting the values of x 1, x 2, x 3, and x 4 into the original
equations

 Jacobi Method:
 Is it possible to arrange the equations in diagonally dominant way?
 Using an initial guess of x 1=0, x 2=0 , x 3=0 , and x 4 =0, perform the first three
iteration of Jacobi method.
 Are the solutions converging or not?
 Rearrange the equations in best possible way (diagonal coefficient is the highest in
particular equation), repeat the first five iterations of Jacobi method. Are the solutions
converging or not?

 Gauss-Seidel Method:
 Using an initial guess of x 1=0, x 2=0 , x 3=0 , and x 4 =0, perform the first three
iteration of Gauss-Seidel method.
 Are the solutions converging or not?
 Rearrange the equations in best possible way (diagonal coefficient is the highest in
particular equation), repeat the first five iterations of Gauss Seidel method. Are the
solutions converging or not?

You are given the following system of linear equations:

2 x1 +1 2 x2 +2 x 3+ 3 x 4 =1 8
x 1+ 2 x 2 +15 x3 + 4 x 4 =25
10 x 1+2 x 2+ x3 + x 4 =1 2
x 1+ 3 x 2 +4 x3 +2 0 x 4 =30

 Out of three methods you learned in the class, which one is best suited here and why?

 Gauss Elimination and Back Substitution:

  Perform the forward elimination step of the Gauss elimination method to reduce the
system to an upper triangular matrix.
 Perform back-substitution to find the values of x 1, x 2, x 3, and x 4
 Verify your solution by substituting the values of x 1, x 2, x 3, and x 4 into the original
equations

 Jacobi Method:
 Is it possible to arrange the equations in diagonally dominant way?
 Using an initial guess of x 1=0, x 2=0 , x 3=0 , and x 4 =0, perform the iteration of
Jacobi method until you reach the convergence of 0.005.

 Gauss-Seidel Method:
 Using an initial guess of x 1=0, x 2=0 , x 3=0 , and x 4 =0, perform iteration of Gauss-
Seidel method until you reach the convergence of 0.005.

 Which is better for convergence Jacobi or Gauss-Seidel?

*You can generate more questions by changing the coefficient and b values in equations

Numerical Integration
π
I 1=∫ sin ( x ) dx
0
1
I 2=∫ e
2
−x
dx
0

π
I 3=∫ x sin ( x ) dx
2

1 x
e
I 4=∫ dx
0 x +0.01

Find I 1, I 2, I 3and I 4 using rectangle, trapezoidal and Gauss Integration method. Use the number
of appropriate bins for the convergence criteria of 0.1 (for all), 0.05 (for all) and 0.01 (for all).