Scribd
Upload
3 views

HW2

Homework 2 for AMATH 563 is due on April 28, 2023, and requires submission via Canvas in PDF format. The assignment includes theoretical problems related to positive definite symmetric (PDS) kernels and computational tasks using PCA and kernel regression on the MNIST dataset. Students are encouraged to collaborate but must submit their own work, adhering to strict academic integrity policies.

Uploaded by

cepem13540
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views

HW2

Homework 2 for AMATH 563 is due on April 28, 2023, and requires submission via Canvas in PDF format. The assignment includes theoretical problems related to positive definite symmetric (PDS) kernels and computational tasks using PCA and kernel regression on the MNIST dataset. Students are encouraged to collaborate but must submit their own work, adhering to strict academic integrity policies.

Uploaded by

cepem13540
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Homework 2

AMATH 563, Spring 2023

Due on Apr 28, 2023 at midnight.

Directions, Reminders and Policies


• You must upload a pdf file of your HW to Canvas by the due date.
• Make sure your solutions are well-written, complete, and readable.
• I suggest you use LATEX(Overleaf is a great option) to prepare your HW and typeset your mathematical
equations.
• If you prefer to hand in a handwritten solution then simply scan and upload the pdf.
• Remember you have two extension tokens that you can use for a day extension for your HWs throughout
the quarter.
• I encourage collaborations and working with your colleagues to solve HW problems but you should only
hand in your own work. We have a zero tolerance policy when it comes to academic misconduct and
dishonesty including: Cheating; Falsification; Plagiarism; Engaging in prohibited behavior; Submitting
the same work for separate courses without the permission of the instructor(s); Taking deliberate action
to destroy or damage another person’s academic work. Such behavior will be reported to the
UW Academic Misconduct office without warning.

1
Problems

Theory
1. Suppsoe Γ : 𝒳 ×𝒳 → R is a PDS kernel. Prove that ∀𝑥, 𝑥′ ∈ 𝒳 it holds that |Γ(𝑥, 𝑥′ )|2 ≤ Γ(𝑥, 𝑥)Γ(𝑥′ , 𝑥′ ).
2. Given a kernel 𝐾 on 𝒳 define its normalized version as

if 𝐾(𝑥, 𝑥) = 0 or 𝐾(𝑥′ , 𝑥′ ) = 0
⎧
⎨0
⎪
¯
𝐾(𝑥, 𝑥′ ) = 𝐾(𝑥, 𝑥′ )
⎪
⎩ √︀ √︀ Otherwise.
𝐾(𝑥, 𝑥) 𝐾(𝑥′ , 𝑥′ )
¯
Show that if 𝐾 is PDS then so is 𝐾.
3. Show that the following kernels on R𝑑 are PDS:
(︁ )︁𝛼
• Polynomial kernel: 𝐾(𝑥, 𝑥′ ) = 𝑥𝑇 𝑥′ + 𝑐 for 𝑐 > 0 and 𝛼 ∈ N.
• Exponential kernel: 𝐾(𝑥, 𝑥′ ) = exp(𝑥𝑇 𝑥′ ).
• RBF kernel: 𝐾(𝑥, 𝑥′ ) = exp(−𝛾 2 ‖𝑥 − 𝑥′ ‖22 ).

4. Let Ω ⊆ R𝑑 and let {𝜓𝑗 }𝑛 continuous functions on Ω and {𝜆𝑗 }𝑛𝑗=1 a sequence of
𝑗=1 be a sequence of∑︀
′ 𝑛 ′
non-negative numbers. Show that 𝐾(𝑥, 𝑥 ) = 𝑗=1 𝜆𝑗 𝜓𝑗 (𝑥)𝜓𝑗 (𝑥 ) is a PDS kernel on Ω.
5. Show that: (i) if 𝐾 and 𝐾′
are two reproducing kernels for an RKHS ℋ, then they have to be the
same. (ii) the RKHS of a PDS kernel 𝐾 is unique.

Computation
Download the MNIST training and test .csv files from Canvas and load them on your computer. I suggest
you use Python or MATLAB for this excercise.

• Use Principle Component Analysis (PCA) on the training set to reduce the dimension of your input.
How many modes do you need to preserve 95% of the variance in the training set?
• Extract the digits 1 and 9 from the training set. Use kernel regression to design and train a classifier
to distinguish these digits using three different kernels of your choosing (I suggest RBF, Polynomila,
and linear). It is a good idea to use PCA to reduce your input dimensions here. Also, you may use
cross validation to tune your kernel/regularization/nugget parameters if you need them. Present the
training and test error of your method.
• Repeat the above experiment for the digits (3, 8), (1, 7), and (5, 2).
• Write a report of a maximum of four pages, outlining your results and findings.

You might also like