hw3
Uploaded by
planck7737hw3
Uploaded by
planck7737HW 2
WANG KUIYU 2021310783
May 17, 2022
1 Problem 1
Assume input X ∈ Rn , output Y ∈ R, assume regression method to be linear regression, then
Z =X ×R (1)
H = f : f (x) = W X + b (2)
2
l(h, (x, y)) = (h(x) − y) (3)
2 Problem 2
l(h, (x, y)) = Ih(x)̸=y (4)
1
p(l(h(x, y)) = 1) = p(h(x) ̸= y) = (5)
2
1
R(h, D) = E(l) = 1 · p(l = 1) + 0 · p(l = 0) = (6)
2
3 Problem 3
3.1
let h∗ = argminR(h, D)
R(hERM
S , D) − R∗ ≤ R̂(h, S) − R̂(h∗ , S) + R(hERM
S , D) − R∗ ≤ 2 sup |R(h, D) − R̂(h, S)| (7)
h∈H
3.2
R(hERM
S , D) − R∗ ≤ 2 sup |R(h, D) − R̂(h, S)| (8)
h∈H
P (R(hERM
S , D) − R∗ > ϵ)
ϵ
≤ P (2 sup |R(h, D) − R̂(h, S)| > ϵ) = P (|R(hi , D) − R̂(hi , S)| > , hi ∈ H)
h∈H 2
|H|
X ϵ (9)
≤ P (|R(hi , D) − R̂(hi , S)| > )
i=1
2
mϵ2
≤ 2|H|exp(− )
2M 2
1
q
2M 2 (ln 2|H|+ln δ −1 )
let ϵ = m
r
∗ ∗ 2M 2 (ln 2|H| + ln δ −1 )
P (R(hERM
S , D) − R > ϵ) = P (R(hERM
S , D) >R + )
m
m 2M 2 (ln 2|H| + ln δ −1 ) (10)
≤ 2|H|exp(− )
2M 2 m
1
= 2|H|exp(−ln2|H| + lnδ) = 2|H| δ=δ
2|H|
So
r
2M 2 (ln 2|H| + ln δ −1 )
P (R(hERM
S , D) ≤ R∗ + )≥1−δ (11)
m
4 Problem 4
Figure 1: Compressed centered images
2
Figure 2: Top 100 PCA components
