Scribd
Upload
20 views2 pages

ML Hw1

Uploaded by

xuando272004
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
20 views2 pages

ML Hw1

Uploaded by

xuando272004
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Homework 1

Instructions:

• Due Date: November 25th, 2024


• Submission:
– Submit your solutions as a printed document.
– Submit directly to the lecturer during your tutorial session on the due date

Grading Criteria:

• Clarity and Completeness: Clear explanation of steps and reasoning.


• Accuracy: Correct calculations and results.
• Code Quality: For programming tasks, code should be well-documented and efficiently
implemented.
• Insight: Demonstration of understanding of underlying concepts through discussion and
analysis.

Problem 1: Machine Learning as an Optimization Problem

(a) Explain why training a machine learning model can be formulated as an optimization
problem. What are the objectives and constraints involved?
(b) Provide examples of how optimization techniques are applied in the training of models
such as linear regression and logistic regression.
(c) Discuss the role of the loss (or cost) function in this context and how it guides the
optimization process.

Problem 2: Maximum Likelihood Estimation (MLE) and Maximum A Posteriori


(MAP)
Given a dataset of independent and identically distributed observations X = {x1 , x2 , . . . , xn }
drawn from a normal distribution with unknown mean µ and known variance σ 2 .

(a) Derive the Maximum Likelihood Estimator (MLE) for the mean µ.
(b) Assume a prior distribution for µ that is also normally distributed with mean µ0 and
variance τ 2 . Derive the Maximum A Posteriori (MAP) estimator for µ.
(c) Compare the MLE and MAP estimators. Discuss how the choice of µ0 and τ 2 affects the
MAP estimator.

Problem 3: Naive Bayes Classification


You are provided with a simplified dataset of text documents classified into two categories:
Sports and Politics. The vocabulary consists of the words: win, team, election, and vote.

1
Word Sports Count Politics Count
win 50 10
team 60 5
election 15 70
vote 10 80

(a) Explain the Naive Bayes assumption and how it applies to text classification.

(b) Using the data above, calculate the probability that a document containing the words win
and vote belongs to the Sports category versus the Politics category. Assume uniform
class priors and apply Laplace smoothing with α = 1.

(c) Interpret the results and discuss any limitations of the Naive Bayes classifier in this
context.

Problem 4: Logistic Regression


Consider a binary classification problem where the goal is to predict whether a student will
pass or fail an exam based on the numbers of hours spent fo studying and sleeping. Formulate
the logistic regression model for this problem.

Problem 5: Linear Regression and Overfitting


You are given a dataset where the input variable x ranges from 0 to 10, and the target
variable y is generated by y = 2x + ϵ, where ϵ is Gaussian noise with mean 0 and variance 4.

(a) Fit a linear regression model to the data and report the estimated parameters.

(b) Fit a 9th-degree polynomial regression model to the same data.

(c) Compare the training error and discuss which model is likely overfitting the data. Provide
visualizations to support your answer.

Problem 6: Regularization Techniques


Regularization is a technique used to prevent overfitting in machine learning models.

(a) Explain the difference between L1 (Lasso) and L2 (Ridge) regularization in the context
of linear regression.

(b) Given a dataset with multiple features are highly correlated, discuss which regularization
method would be more appropriate and why.

You might also like