Course Introduction

Artificial Intelligence

Eun-Sol Kim (김은솔)

High-level Reasoning with Videos
- Planning
Inpainting

Blue cone
-> yellow cone

https://huggingface.co/spaces/nielsr/text-based-inpainting
Inpainting

Should define
the target region
with specific description
Protein Structure Prediction
Protein Structure Prediction
AlphaFold
Video Understanding

• 이 비디오의 주제는?
• 이 비디오의 태그는?
• Food
• Recipe
• Cooking
• Dish
• Dessert
• Cake
• Baking
• Cream
• Milk
• Pudding and Risotto

10
Video Inderstanding

주어 객체 관계 목적어 객체
(Subject Entity) (Relation) (Object Entity)

검은 셔츠 남자 앞에 있다 흰색 가스레인지

회색 셔츠 남자 옆에 있다 검은 셔츠 남자

흰색 가스레인지 옆에 있다 은색 전자레인지

회색 셔츠 남자 쓰고 있다 파란 모자

회색 셔츠 남자 입고 있다 흰 앞치마

11
Image Understanding

• 파란색 티셔츠를 입고 있는 사람 왼
쪽에 냉장고가 있어?
• 네

• 이 사진이 찍힌 장소가 어디야?

• 주방

• 사람이 몇 명 있어?
• 한명

12
Image Understanding

오른쪽에
흰색 컵 잡고 있다 이미지
있다

여성

파란 색 우산 잡고 있다 길가
있다

13
Course Introduction
Introduction

q This course provides a variety of fundamental mathematical

concepts and basic techniques used in the applications of artificial
intelligence.
q It is aimed at giving an understanding of how to build a
mathematical formulation for the concept of learning-from-data.
Lectures

q Lectures
q Tue, 14:30 ~ 16:00
q Thr, 13:00 ~ 14:30
Course Instructor

q Eun-Sol Kim
q IT.BT Bldg., 504-2
q [email protected]
Books
q Basically, class notes are mainly used.

q Christopher M. Bishop, Pattern Recognition and Machine Learning

q Kevin. P. Murphy, Machine Learning: A Probabilistic Perspective
q 스기야마 마사시, 통계적 기계학습
Prerequisites

q Linear algebra
q Probability
q Python Language (for project)
q Students should have a good understanding of the differentiation
and integration and the basic operations using vectors and matrices.
Topics (1/2)

q Vector differentiation, matrix differentiation, obtaining analytic

optimal solution
q Logistic regression, Fisher discriminant analysis
q Neural Network – Motivation, Architecture
q Neural Network – Backpropagation, Gradient descent, Numerical
Optimization
q Nearest neighbor methods
q Large margin methods, Support vector Machines
q Kernel Methods
q Midterm (10/26)
Topics (2/2)

q Probability model, Parameter estimation

q Graphical Model
q Gaussian Model, Multinomial Model
q Bayesian Inference
q Applications of Graphical Models
q Advanced Topics on Artificial Intelligence
q Final Exam (12/14)
Grading

q Attendance : 5%
q Should attend > 10 classes
q Participation : 5%
q Homework, Project: 20%
q Two exams: 70% (35% each)
Late Assignment Penalty

q Before the due date: 100%

q < 24 hours: 80%
q < 48 hours: 60%
q >= 48 hours: no score
Homework, Project

q One assignment, One final project

q Assignment
q Solving problems
q Project
q Learning machine learning algorithms given a benchmark dataset
q Python
q Analysing the results
q Writing a short paper
Plagiarism

q Any type of plagiarism will directly result in the failure of this class
(F) once detected!
q Although active discussion between classmates is encouraged, you
should write your original answers in your own words.
Artificial Intelligence
Artificial Intelligence
Birth of AI

Slide from stanford cs221

Early successes

Slide from stanford cs221

Knowledge-based Systems (70-80s)

Slide from stanford cs221

History of Neural Network Research
Neural network Deep belief net
Back propagation Science Speech

1986 2006 2011 2012

deep learning results
• • Unsupervised & Layer-wised
Loose tie with biological systems pre-training
• SVM
… …Rank Name Errorfor modelingDescription
• • Better
Shallowdesigns
model
• Solve general andproblems
learning training
• Boosting rate nonlinearity, dropout)
… …1 • (normalization,
Specific• methods
Tied with for biological
specific taskssystem
• Decision tree U. Toronto
But –it isHand 0.15315
givencrafted
up… featuresDeep Conv Net
• Feature learning (GMM-HMM, SIFT, LBP, HOG)
• … …2
KNN U. Tokyo
• • New Hard 0.26172 Hand-crafted
to train
development offeatures
computerand architectures
• …… …3 U. Oxford
• –Insufficient
GPU
0.26979
computational resources
learning models.
4 Xerox/INRIA 0.27058
• Small training sets
– Multi-core computer Bottleneck.
systems
• Does not work well
• Large scale databases
ObjectComputers
How Many recognition over 1,000,000
to Identify images
a Cat? and CPU
16000 1,000cores
categories
Kruger et al. TPAMI’13
(2 GPU)
Slides from Wanli Ouyang [email protected]
Machine Learning

q Definition
q a branch of artificial intelligence, a scientific discipline concerned with the
design and development of algorithms that allow computers to evolve
behaviors based on empirical data, such as from sensor data, web data or
databases

q Related Areas
q probability theory, statistics, pattern recognition, cognitive science, data
mining, adaptive control, computational neuroscience, theoretical computer
science, ...
Machine Learning: Applications

• Machine perception
• Computer vision
• Natural language processing
• Search engines
• Medical diagnosis
• Bioinformatics
• Structural health monitoring
• Speech and handwriting recognition
• Object recognition in computer vision
• Brain-machine interfaces and cheminformatics
• ...
What deep
learning can do?
What deep learning can do?

1. Automatic Colorization of Black and White images

http://richzhang.github.io/colorization/
What deep learning can do?

2. Automatically adding sounds to silent movies

http://vis.csail.mit.edu/
What deep learning can do?

3. Automatic Machine Translation

 What deep learning can do?

4. Object Classification and Detection in Photographs

ImageNet Classification with Deep Convolutional

Neural Networks
What deep learning can do?

5. Automatic Image Generation

DALLE, OpenAI
 What deep learning can do?

6. Automatic Text Generation

http://karpathy.github.io/2015/05/21/rnn-effectiveness/
 What deep learning can do?

7. Automatic Image Caption Generation

http://cs.stanford.edu/people/karpathy/deepimagesent/
 What deep learning can do?

8. Automatic Game Playing

https://youtu.be/TmPfTpjtdgg
Introduction to Machine Learning

Eun-Sol Kim (김은솔)

Artificial Intelligence
Training & Testing

https://www.youtube.com/watch?v=wymhAN93X9I
Training and Testing

q The main purpose of the ML algorithm is to make suitable

predictions for unseen inputs.
q Training with given data
q Testing with unseen and new data
An Example
q Housing Price Prediction

Size of # of Size of
No Age of house Price
house bedrooms kitchen

1 20 2 20 5 700

2 20 3 10 3 900

3 30 3 5 5 1400

4 35 3 10 8 1400

5 35 4 10 8 1450

6 40 4 5 4 1800

7 37 4 5 4 1700
Housing Price Prediction

Price

Size
Housing Price Prediction

Price

?
Size
Housing Price Prediction

Price

Size
Classification Example

건강한
x2 사람
건강하지
않은 사람

x1
Classification Example

건강한
x2 사람
건강하지
않은 사람

x1
Classification Example

건강한
x2 사람
건강하지
않은 사람

x1
Classification Example

건강한
x2 사람
건강하지
않은 사람

x1
Machine Learning
q Study of algorithms that
qImprove their performance P
qAt some task T
qWith experience E
q Learning = improving P at T with E

Tom Mitchell, Machine Learning https://www.youtube.com/watch?v=EfNfNhWnCfs

 Learning = improving P at T with E

T (Task) = Something Done (Output)

q Classification
q Regression
q Diagnosing
q Diabetes
q Cancer
q Driving
q Car
q Forecasting
q Weather
q Stock price
 Learning = improving P at T with E

E (Experience) = (Training) Data

q Supervised learning
q Learn w/ answers (class)
q Classification
q Yes/No, A/B/C/D, ...
q Regression
q Real number

q Unsupervised learning
q Learn w/o answers
q (no class)
q Clustering
q Feature selection

q Semi-supervised learning
q Web image auto-tagging
q Reinforcement learning
q AlphaGo, AlphaStar, Chess
 Learning = improving P at T with E

P (Performance Measure)
= Target (or Loss) Function
“If you cannot measure it, you can not improve it.”
Kelvin, Lord William Thomson (1824-1907)

error
error
 Learning = improving P at T with E

P (Performance Measure)
= Target (or Loss) Function
“If you cannot measure it, you can not improve it.”
Kelvin, Lord William Thomson (1824-1907)

• Error rate

• Euclidean distance

• Log Probability

• Information theoretical measures

o Mutual information
o KL