Georgia Institute of Technology

Course Syllabus: CS7643 Deep Learning

Spring 2024
Delivery: 100% Web-Based on Canvas, with submissions on Canvas/Gradescope
Dates course will run: Jan 8th, 2024 – May 6th 2024

Instructor Information
Dr. Kira Zsolt
Email: [email protected]

General Course Information

Description
Deep learning is a sub-field of machine learning that focuses on learning complex, hierarchical
feature representations from raw data. The dominant method for achieving this, artificial neural
networks, has revolutionized the processing of data (e.g. images, videos, text, and audio) as well as
decision-making tasks (e.g. game-playing). Its success has enabled a tremendous amount of
practical commercial applications and has had a significant impact on society.
In this course, students will learn the fundamental principles, underlying mathematics, and
implementation details of deep learning. This includes the concepts and methods used to optimize
these highly parameterized models (gradient descent and backpropagation, and more generally
computation graphs), the modules that make them up (linear, convolution, and pooling layers,
activation functions, etc.), and common neural network architectures (convolutional neural networks,
recurrent neural networks, etc.). Applications ranging from computer vision to natural language
processing and decision-making (reinforcement learning) will be demonstrated. Through in-depth
programming assignments, students will learn how to implement these fundamental building blocks
as well as how to put them together using a popular deep learning library, PyTorch. In the final
project, students will apply what they have learned to real-world scenarios by exploring these
concepts with a problem that they are passionate about.

Pre- &/or Co-Requisites

Students must have a strong mathematical background (linear algebra, calculus especially taking
partial derivatives, and probabilities & statistics) and at least an introductory course in Machine
Learning (e.g. equivalent to CS 7641). This is not a soft requirement: This should NOT be your first
ML class, and self-study (e.g. online Coursera/Udacity courses) do not count. While you may be able
to do OK on some parts of the class, other parts such as quizzes will be difficult to do if you do not
have any background in ML. Strong programming skills (specifically Python) are necessary to
complete the assignments.

Course Objectives
● Describe the major differences between deep learning and other types of machine learning
algorithms.

v5.1 1
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning
● Explain the fundamental methods involved in deep learning, including the underlying
optimization concepts (gradient descent and backpropagation), typical modules they consist of,
and how they can be combined to solve real-world problems.
● Differentiate between the major types of neural network architectures (multi-layered perceptrons,
convolutional neural networks, recurrent neural networks, etc.) and what types of problems each
is appropriate for.
● Select or design neural network architectures for new data problems based on their
requirements and problem characteristics and analyze their performance.
● Describe some of the latest research being conducted in the field and open problems that are
yet to be solved.

Course Materials

Course Text
Deep Learning, by Ian Goodfellow and Yoshua Bengio and Aaron Courville, MIT Press.
Available online.

Additional Materials/Resources
All additional reading materials will be available via PDF on Canvas.

Course Website and Other Classroom Management Tools

All course materials and videos are located on Canvas.

Course Requirements, Assignments & Grading

Assignment Distribution and Grading Scale

Assignments Weight

On-Boarding Quiz (Required to verify identity using

proctoring software)

Pre-requisites problem set (PS0) and 1.25%

Quiz 0

Assignments (4) 60%

Quizzes (equally weighted) 18.75%

Final Project (including proposal) 20%

Ed Participation Bonus 1% extra credit for top 3 endorsed posters

v5.1 2
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning

Grading Scale
Your final grade will be assigned as a letter grade, with at least the following grades (i.e., 90 or
greater will definitely be an A).

A 90-100%

B 80-89%

C 70-79%

D 60-69%

F 0-59%

Assignment Due Dates

All assignments are due at 08:00:00 AM Eastern Time (ET) on the day the assignment is due,
unless otherwise noted. If you are located outside of ET, Canvas will display the due dates in your
local time (which can be changed by editing your personal Canvas settings). We will not accept
assignments submitted late due to time zone issues, so do verify your desired settings as there are
no exceptions.
Every homework deliverable and project deliverable will have a 48-hour grace period during which
no penalty will apply. This is intended to allow you time to verify that your submission has been
submitted (we recommend you re-download it and look it over to make sure all
questions/deliverables have been answered). Canvas will show your submission as late, but you do
not have to ask for this grace period. Deliverables after the grace period will receive a grade of
0. There is no grace period for taking quizzes.

Project
The project is a group assignment comprising of 2-4 members (instructor permission is required for
other group arrangements; strong justification will be needed for approval). The class project is
meant for students to (1) gain experience implementing deep models and (2) try Deep Learning on
problems that interest them. The amount of effort should be at the level of one homework
assignment per group member. A PDF write-up describing the project in a self-contained manner will
be the sole deliverable. Your final write-up will be structured like a paper from a computer vision
conference (CVPR, ECCV, ICCV, etc.). We will release this template as well as rubric. Additionally,
we will allow people to upload additional code, videos, and other supplementary material like code
upload for assignments. While the PDF may link to supplementary material, external documents and
code, such resources may or may not be used to evaluate the project. The final PDF should
completely address all the points in the rubric that will be released.

Technology Requirements and Skills

Computer Hardware and Software
 High-speed Internet connection
 Laptop or desktop computer with a minimum of a 2 GHz processor and 8 GB of RAM
v5.1 3
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning

 CUDA compatible GPU is helpful for assignments but not necessary.

 UNIX-like OS experience is recommended (Linux/macOS)
 Windows/Linux for PC computers OR Mac iOS for Apple computers.
 Complete Microsoft Office Suite or comparable and ability to use Adobe PDF software (install,
download, open and convert)
 Mozilla Firefox, Chrome browser, and/or Safari browsers (Chrome required for on-boarding quiz)

Canvas
This class will use Canvas to deliver course materials to online students. ALL course materials and
quiz assessments will take place on this platform. Gradescope will be used for submission of
assignments and the project.

Proctoring Information
To verify the identity of all GT online students, all online students are required to complete the
onboarding quiz that uses Honorlock. Honorlock is utilized for student identity verification and to
ensure academic integrity. Honorlock provides student identity verification via facial and ID photos.
You may also be asked to scan the room around you. The onboarding quiz will be a practice quiz
that will not affect your grade in the course. You can take the onboarding quiz as many times as you
want. All potential violations are reviewed by a human. The Honorlock support team is available
24/7. While Honorlock will not require you to create an account, download software, or schedule an
appointment in advance, you will need Google Chrome and download the Honorlock Chrome
Extension. Information on how to access Honorlock and additional resources are provided below.
You can also access Honorlock support at https://honorlock.com/support/.

Course Policies, Expectations & Guidelines

Communication Policy
You are responsible for knowing the following information:
1. Anything posted to this syllabus.
2. Anything emailed directly to you by the teaching team (including announcements via Ed
Discussion), 24 hours after receiving such an email or post.
Because Ed announcements are emailed to you as well, you need only to check your Georgia Tech
email once every 24 hours to remain up to date on new information during the semester. Georgia
Tech generally recommends students to check their Georgia Tech email once every 24 hours. So, if
an announcement or message is time sensitive, you will not be responsible for the contents of the
announcement until 24 hours after it has been sent.

Late and Make-up Work Policy

There will be no make-up work provided for missed assignments. Of course, emergencies (illness,
family emergencies) will happen. In those instances, please contact the Dean of Students office. The
Dean of Students is equipped to verify emergencies and pass confirmation on to all your classes.
For consistency, we ask all students to do this in the event of an emergency. Do not send any

v5.1 4
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning
personal/medical information to the instructor or TAs; all such information should go through the
Dean of Students.

Online Student Conduct and (N)etiquette

Communicating appropriately in the online classroom can be challenging. All communication,
whether by email, Ed, Canvas, or otherwise, must be professional and respectful. In order to
minimize this challenge, it is important to remember several points of “internet etiquette” that will
smooth communication for both students and instructors.
1. Read first, Write later. Read the ENTIRE set of posts/comments on a discussion board
before posting your reply, to prevent repeating commentary or asking questions that have
already been answered.
2. Avoid language that may come across as strong or offensive. Language can be easily
misinterpreted in written electronic communication. Review email and discussion board posts
BEFORE submitting. Humor and sarcasm may be easily misinterpreted by your reader(s).
Try to be as matter of fact and as professional as possible.
3. Follow the language rules of the Internet. Do not write using all capital letters, because it will
appear as shouting. Also, the use of emoticons can be helpful when used to convey
nonverbal feelings. ☺
4. Consider the privacy of others. Ask permission prior to giving out a classmate's email
address or other information.
5. Keep attachments small. If it is necessary to send pictures, change the size to an acceptable
250kb or less (one free, web-based tool to try is picresize.com).
6. No inappropriate material. Do not forward virus warnings, chain letters, jokes, etc. to
classmates or instructors. The sharing of pornographic material is forbidden.
NOTE: The instructor reserves the right to remove posts that are not collegial in nature and/or do not
meet the Online Student Conduct and Etiquette guidelines listed above.

University Use of Electronic Email

A university-assigned student e-mail account is the official university means of communication with
all students at Georgia Institute of Technology. Students are responsible for all information sent to
them via their university-assigned e-mail account. If a student chooses to forward information in their
university e-mail account, he or she is responsible for all information, including attachments, sent to
any other e-mail account. To stay current with university information, students are expected to check
their official university e-mail account and other electronic communications on a frequent and
consistent basis. Recognizing that some communications may be time-critical, the university
recommends that electronic communications be checked minimally twice a week.

Plagiarism & Academic Integrity

Georgia Tech aims to cultivate a community based on trust, academic integrity, and honor. Students
are expected to act according to the highest ethical standards. All students enrolled at Georgia Tech,
and all its campuses, are to perform their academic work according to standards set by faculty
members, departments, schools and colleges of the university; and cheating and plagiarism
constitute fraudulent misrepresentation for which no credit can be given and for which appropriate
sanctions are warranted and will be applied. For information on Georgia Tech's Academic Honor

v5.1 5
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning
Code, please visit http://www.catalog.gatech.edu/policies/honor-code/ or
http://www.catalog.gatech.edu/rules/18/.
You are encouraged to discuss problems and papers with others as long as this does not involve the
copying of code or solutions. After discussions, all materials that are part of a submission should be
wholly your own. Any public material that you use to gain an understanding of the materials (open-
source software, help from a textbook, or substantial help from a friend, etc.) should be
acknowledged explicitly in anything you submit to us. To re-emphasize, no matter what the source
you cannot copy any existing code, from other students, online, or otherwise, and all code must be
wholly your own code that you wrote by yourself. Our current policy is that you CANNOT use any AI-
assisted (e.g. based on large language models such as Github Copilot) text or coding responses in
part or whole; again, every piece of text or code you submit must be wholly your own. If you have
any doubts about whether something is legal or not, please do check with the class Instructor or the
TA.
We will actively check for cheating, and any act of dishonesty will result in a Fail grade. Any student
suspected of cheating or plagiarizing on a quiz, exam, or assignment will be reported to the Office of
Student Integrity, who will investigate the incident and identify the appropriate penalty for violations.

AI-based assistance
We will use the AI-based assistance policy developed by David Joyner. We treat AI-based
assistance, such as ChatGPT and Copilot, the same way we treat collaboration with other people:
you are welcome to talk about your ideas and work with other people, both inside and outside the
class, as well as with AI-based assistants.
However, all work you submit must be your own. You should never include in your assignment
anything that was not written directly by you without proper citation (including quotation marks and
in-line citation for direct quotes).
Including anything you did not write in your assignment without proper citation will be treated as an
academic misconduct case.
If you are unsure where the line is between collaborating with AI and copying from AI, we
recommend the following heuristics:
Heuristic 1: Never hit "Copy" within your conversation with an AI assistant. You can copy your own
work into your conversation, but do not copy anything from the conversation back into your
assignment.
Instead, use your interaction with the AI assistant as a learning experience, then let your assignment
reflect your improved understanding.
Heuristic 2: Do not have your assignment and the AI agent open at the same time. Similar to above,
use your conversation with the AI as a learning experience, then close the interaction down, open
your assignment, and let your assignment reflect your revised knowledge.
This heuristic includes avoiding using AI directly integrated into your composition environment: just
as you should not let a classmate write content or code directly into your submission, so also you
should avoid using tools that directly add content to your submission.
Deviating from these heuristics does not automatically qualify as academic misconduct; however,
following these heuristics essentially guarantees your collaboration will not cross the line into
misconduct.

v5.1 6
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning

Students with Disabilities

If you are a student with learning needs that require special accommodation, contact the Office of
Disability Services at 404.894.2563 or http://disabilityservices.gatech.edu/, as soon as possible, to
make an appointment to discuss your special needs and to obtain an accommodations letter.
Please also e-mail me as soon as possible to set up a time to discuss your learning needs.

Illness and Other Ailments

If you are a student that is negatively impacted by a health-related matter, please contact the Office
of Disability Services or the Office of Dean of Students at 404.894.6367 or
[email protected]. Do NOT send us any personal health information. They will
provide you with an accommodation letter that will allow us to try to find a suitable schedule for
completing all assignments. You MUST submit this and inform us that you did so on Ed before the
due date for the deliverable.

Mental Health Resources and Support Services

Campus Resources

Georgia Tech Police Department Collegiate Recovery Stamps Psychiatry Center

Emergency: Call 911 | 404-894-2500 Program 404-894-1420
404-894-2575 |
Dean of Students Office VOICE
counseling.gatech.edu
404-894-2565 | studentlife.gatech.edu 404-385-4464 |
Afterhours Assistance Line & Dean on Counseling Center 404-385-4451
Call: 404-894-2204 404-894-2575 | 24/7 Info Line: 404-894-9000 |
counseling.gatech.edu voice.gatech.edu
Center for Assessment, Referral and
Education (CARE) Health Initiatives Women’s Resource Center
404-894-3498 | care.gatech.edu 404-894-9980 404-385-0230 |
healthinitiatives.gatech. womenscenter.gatech.edu
edu
Veterans Resource Center
LGBTQIA Resource 404-894-4953 |
Center veterans.gatech.edu
404-385-4780 |
lgtbqia.gatech.edu

v5.1 7
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning

Community Resources

Georgia Crisis and Access Line National Suicide Prevention Hotline

1-800-715-4225 1-800-273-8255
The crisis line is staffed with professional A national network of local crisis centers that provides
social workers and counselors 24 hours free and confidential emotional support to people in
per day, every day, to assist those with suicidal crisis or emotional distress 24/7.
urgent and emergency needs.
Georgia State Psychology Clinic
Trevor Project 404-413-2500
1-866-488-7386 The clinic offers high quality and affordable
Trained counselors are available to psychological services to adults, children, adolescents,
support anyone in need. families and couples from the greater Atlanta area.

Student-Faculty Expectations Agreement

At Georgia Tech we believe that it is important to strive for an atmosphere of mutual respect,
acknowledgement, and responsibility between faculty members and the student body. See
http://www.catalog.gatech.edu/rules/22/ for an articulation of some basic expectation that you can
have of me and that I have of you. In the end, simple respect for knowledge, hard work, and cordial
interactions will help build the environment we seek. Therefore, I encourage you to remain
committed to the ideals of Georgia Tech while in this class.

Subject to Change Statement

The syllabus and course schedule may be subject to change. Changes will be communicated via the
Canvas announcement tool. It is the responsibility of students to check Ed Discussions, email
messages, and course announcements to stay current in their online courses.

v5.1 8
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning

Course Schedule
Week/Dates Modules/Lessons Deliverables

1 Module 1: Introduction to Neural Networks Quiz #0: Course prereqs and

policies Due Jan 15th 8:00 AM
Jan 8th Go through Welcome/Getting Started
ET (NO GRACE PERIOD)
Lesson 1: Linear Classifiers and Gradient
Descent
Pre-req Problem set (PS0) Jan
Readings: 15th 8:00 AM ET (NO GRACE
PERIOD)
 DL book: Linear Algebra background
 DL book: Probability background
 DL book: ML Background
 LeCun et al., Nature '15
 Shannon, 1956

2 Lesson 2: Neural Networks Quiz #1: Linear Classifiers and

Gradient Descent (Lesson 1)
Jan 15th Readings:
and Neural Networks (Lesson 2)
 DL book: Deep Feedforward Nets Due Jan 22nd 8:00 AM ET (NO
 Matrix calculus for deep learning GRACE PERIOD)
 Automatic Differentiation Survey, Baydin et
al.

3 Lesson 3: Optimization of Deep Neural Assignment 1 Due Jan 29th

Networks 8:00 AM ET (grace period until
Jan 22nd
Jan 31st)
Readings:
 DL book: Regularization for DL
 DL book: Optimization for Training Deep
Models

4 Module 2: Convolutional Neural Networks Quiz #2: Optimization of Deep

Neural Networks (Lesson 3),
Jan 29th (OPTIONAL) Lesson 6: Data Wrangling
and Convolution and Pooling
Lesson 5: Convolution and Pooling Layers Layers (Lesson 5) Due Feb 4th
8:00 AM ET (NO GRACE
Readings:
PERIOD)
 Preprocessing for deep learning: from
covariance matrix to image whitening
 cs231n on preprocessing
 DL book: Convolutional Networks
 Optional: Khetarpal, Khimya, et al. Re-
evaluate: Reproducibility in evaluating
reinforcement learning algorithms." (2018).
See related blog post

v5.1 9
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning

Week/Dates Modules/Lessons Deliverables

5 Lesson 6: Convolutional Neural Network

Architectures
Feb 5th

6 Lesson 7: Visualization Assignment 2 due Feb 19th

th 8:00 AM ET (grace period until
Feb 12 Lesson 8: PyTorch and Scalable Training
Feb 21st)
Readings:
 Understanding Neural Networks Through
Deep Visualization
 Grad-CAM: Visual Explanations from Deep
Networks via Gradient-based Localization

7 Lesson 9: Advanced Computer Vision Quiz #3: Convolutional Neural

Architectures Network Architectures (Lesson
Feb 19th
6), Visualization (Lesson 7),
Lesson 10: Bias and Fairness
Advanced Computer Vision
Readings: Architectures (Lesson 9) Due
Feb 26th 8:00 AM ET (NO
 Fully Convolutional Networks for Semantic
GRACE PERIOD)
Segmentation

8 Module 3: Structured Neural Representations Assignment 3 Due March 4th

8:00 AM ET (grace period until
Feb 26th Lesson 11: Introduction to Structured
March 6th)
Representations
Lesson 12: Language Models
Readings:
 DL Book: Sequential Modeling and
Recurrent Neural Networks (RNNs)

9 Lesson 13: Embeddings Project Proposal Due March

17th 8:00 AM ET (grace period
March 4th
until March 19th)
Readings:
 word2vec tutorial
 word2vec paper
 StarSpace paper

10 Lesson 14: Neural Attention Models

(March 13th Withdrawal
March 11th Readings:
Deadline)
 Attention is all you need
 BERT Paper

v5.1 10
Georgia Institute of Technology
Course Syllabus: CS7643 Deep Learning

Week/Dates Modules/Lessons Deliverables

 The Illustrated Transformer

11 Lesson 15: Neural Machine Translation Quiz #4: Module 3 Due March
25th 8:00 AM ET (NO GRACE
March 18th Lesson 16: Automated Speech Recognition
PERIOD)
(ASR)

12 Module 4: Advanced Topics Assignment 4 Due March 25th

8:00 AM ET (grace period until
March 25th Lesson 17: Deep Reinforcement Learning
March 27th)
Readings:
 MDP Notes (courtesy Byron Boots)
 Notes on Q-learning (courtesy Byron Boots)
 Policy iteration notes (courtesy Byron
Boots)
 Policy gradient notes (courtesy Byron
Boots)

13 Lesson 18: Unsupervised and Semi-Supervised

Learning
April 1st

14 Lesson 19: Generative Models Quiz #5: Module 4 Due April

15th 8:00 AM ET (NO GRACE
April 8th Readings:
PERIOD)
 Tutorial on Variational Autoencoder
 NIPS 2016 Tutorial: Generative Adversarial
Networks

15 Wrap-Up Final Project Due April 29th

8:00 AM ET (grace period until
April 15th
May 1nd)

v5.1 11