SlideShare a Scribd company logo
vscentrum.be
Introduction to machine
learning/AI
Geert Jan Bex, Jan Ooghe, Ehsan Moravveji
Material
• All material available on GitHub
• this presentation
• conda environments
• Jupyter notebooks
2
https://github.com/gjbex/PRACE_ML
or
https://bit.ly/prace2019_ml
Introduction
• Machine learning is making great strides
• Large, good data sets
• Compute power
• Progress in algorithms
• Many interesting applications
• commericial
• scientific
• Links with artificial intelligence
• However, AI  machine learning
3
Machine learning tasks
• Supervised learning
• regression: predict numerical values
• classification: predict categorical values, i.e., labels
• Unsupervised learning
• clustering: group data according to "distance"
• association: find frequent co-occurrences
• link prediction: discover relationships in data
• data reduction: project features to fewer features
• Reinforcement learning
4
Regression
Colorize B&W images automatically
https://tinyclouds.org/colorize/
5
Classification
6
Object recognition
https://ai.googleblog.com/2014/09/buildin
g-deeper-understanding-of-images.html
Reinforcement
learning
Learning to play Break Out
https://www.youtube.com/watch?v=V1eY
niJ0Rnk
7
Clustering
Crime prediction using k-means
clustering
http://www.grdjournals.com/uploads/articl
e/GRDJE/V02/I05/0176/GRDJEV02I0501
76.pdf
8
Applications in
science
9
Machine learning algorithms
• Regression:
Ridge regression, Support Vector Machines, Random Forest,
Multilayer Neural Networks, Deep Neural Networks, ...
• Classification:
Naive Base, , Support Vector Machines,
Random Forest, Multilayer Neural Networks,
Deep Neural Networks, ...
• Clustering:
k-Means, Hierarchical Clustering, ...
10
Issues
• Many machine learning/AI projects fail
(Gartner claims 85 %)
• Ethics, e.g., Amazon has/had
sub-par employees fired by an AI
automatically
11
Reasons for failure
• Asking the wrong question
• Trying to solve the wrong problem
• Not having enough data
• Not having the right data
• Having too much data
• Hiring the wrong people
• Using the wrong tools
• Not having the right model
• Not having the right yardstick
12
Frameworks
• Programming languages
• Python
• R
• C++
• ...
• Many libraries
• scikit-learn
• PyTorch
• TensorFlow
• Keras
• …
13
classic machine learning
deep learning frameworks
Fast-evolving ecosystem!
scikit-learn
• Nice end-to-end framework
• data exploration (+ pandas + holoviews)
• data preprocessing (+ pandas)
• cleaning/missing values
• normalization
• training
• testing
• application
• "Classic" machine learning only
• https://scikit-learn.org/stable/
14
Keras
• High-level framework for deep learning
• TensorFlow backend
• Layer types
• dense
• convolutional
• pooling
• embedding
• recurrent
• activation
• …
• https://keras.io/
15
Data pipelines
• Data ingestion
• CSV/JSON/XML/H5 files, RDBMS, NoSQL, HTTP,...
• Data cleaning
• outliers/invalid values?  filter
• missing values?  impute
• Data transformation
• scaling/normalization
16
Must be done systematically
Supervised learning: methodology
• Select model, e.g., random forest, (deep) neural network, ...
• Train model, i.e., determine parameters
• Data: input + output
• training data  determine model parameters
• validation data  yardstick to avoid overfitting
• Test model
• Data: input + output
• testing data  final scoring of the model
• Production
• Data: input  predict output
17
Experiment with underfitting and overfitting:
010_underfitting_overfitting.ipynb
From neurons to ANNs
18
𝑦 = 𝜎
𝑖=1
𝑁
𝑤𝑖𝑥𝑖 + 𝑏
𝑥
𝜎 𝑥
activation function
𝑤1
𝑤2
𝑤3
𝑤𝑁
𝑥1
𝑥2
𝑥3
𝑥𝑁
...
𝑏
𝑦
+1
inspiration
Multilayer network
19
How to determine
weights?
Training: backpropagation
• Initialize weights "randomly"
• For all training epochs
• for all input-output in training set
• using input, compute output (forward)
• compare computed output with training output
• adapt weights (backward) to improve output
• if accuracy is good enough, stop
20
Task: handwritten digit recognition
• Input data
• grayscale image
• Output data
• digit 0, 1, ..., 9
• Training examples
• Test examples
21
Explore the data: 020_mnist_data_exploration.ipynb
First approach
• Data preprocessing
• Input data as 1D array
• output data as array with
one-hot encoding
• Model: multilayer perceptron
• 758 inputs
• dense hidden layer with 512 units
• ReLU activation function
• dense layer with 512 units
• ReLU activation function
• dense layer with 10 units
• SoftMax activation function
22
array([ 0.0, 0.0,..., 0.951, 0.533,..., 0.0, 0.0], dtype=f
5
array([ 0, 0, 0, 0, 0, 1, 0, 0, 0, 0], dtype=ui
Activation functions: 030_activation_functions.ipynb
Multilayer perceptron: 040_mnist_mlp.ipynb
Deep neural networks
• Many layers
• Features are learned, not given
• Low-level features combined into
high-level features
• Special types of layers
• convolutional
• drop-out
• recurrent
• ...
23
Convolutional neural networks
24
1 ⋯ 0
⋮ ⋱ ⋮
0 ⋯ 1

Convolution examples
25
1 ⋯ 0
⋮ ⋱ ⋮
0 ⋯ 1
0 ⋯ 1
⋮ ⋱ ⋮
1 ⋯ 0
1 ⋯ 0
⋮ ⋱ ⋮
0 ⋯ 1
0 ⋯ 1
⋮ ⋱ ⋮
1 ⋯ 0
Convolution: 050_convolution.ipynb
Second approach
• Data preprocessing
• Input data as 2D array
• output data as array with
one-hot encoding
• Model: convolutional neural
network (CNN)
• 28  28 inputs
• CNN layer with 32 filters 3  3
• ReLU activation function
• flatten layer
• dense layer with 10 units
• SoftMax activation function
26
array([[ 0.0, 0.0,..., 0.951, 0.533,..., 0.0, 0.0]], dtype
5
array([ 0, 0, 0, 0, 0, 1, 0, 0, 0, 0], dtype=ui
Convolutional neural network: 060_mnist_cnn.ipynb
Task: sentiment classification
• Input data
• movie review (English)
• Output data
• Training examples
• Test examples
27
Explore the data: 070_imdb_data_exploration.ipynb
/
<start> this film was just brilliant casting
location
scenery story direction everyone's really suited the
part
they played and you could just imagine being there
Robert
redford's is an amazing actor and now the same being
director
norman's father came from the same scottish island
as myself
so i loved the fact there was a real connection with
this
film the witty remarks throughout the film were
great it was
just brilliant so much that i bought the film as
soon as it

Word embedding
• Represent words as one-hot vectors
length = vocabulary size
• Word embeddings
• dense vector
• vector distance  semantic distance
• Training
• use context
• discover relations with surrounding
words
28
Issues:
• unwieldy
• no semantics
How to remember?
Manage history, network learns
• what to remember
• what to forget
Long-term correlations!
Use, e.g.,
• LSTM (Long Short-Term Memory
• GRU (Gated Recurrent Unit)
Deal with variable length input and/or
output
29
Gated Recurrent
Unit (GRU)
• Update gate
• Reset gate
• Current memory content
• Final memory/output
30
𝑧𝑡 = 𝜎 𝑊
𝑧𝑥𝑡 + 𝑈𝑧ℎ𝑡−1
𝑟𝑡 = 𝜎 𝑊
𝑟𝑥𝑡 + 𝑈𝑟ℎ𝑡−1
ℎ′𝑡 = tanh 𝑊𝑥𝑡 + 𝑟𝑡 ⊙ 𝑈ℎ𝑡−1
ℎ𝑡 = 𝑧𝑡 ⊙ ℎ𝑡−1 + 1 − 𝑧𝑡 ⊙ ℎ′𝑡
Approach
• Data preprocessing
• Input data as padded array
• output data as 0 or 1
• Model: recurrent neural network
(GRU)
• 100 inputs
• embedding layer, 5,000 words, 64
element representation length
• GRU layer, 64 units
• dropout layer, rate = 0.5
• dense layer, 1 output
• sigmoid activation function
31
Recurrent neural network: 080_imdb_rnn.pynb
Caveat
• InspiroBot (http://inspirobot.me/)
• "I am an artificial intelligence dedicated to generating unlimited amounts of unique inspirational quotes for endless
enrichment of pointless human existence".
32

More Related Content

PPTX
Artificial Intelligence, Machine Learning and Deep Learning
PDF
Apache MXNet ODSC West 2018
PDF
Main principles of Data Science and Machine Learning
PDF
Scaling Deep Learning with MXNet
PPTX
Deep Learning with Python (PyData Seattle 2015)
PDF
Hardware Acceleration for Machine Learning
PPTX
Deep learning introduction
PPTX
Artificial Intelligence (AI) INTERNSHIP.pptx
Artificial Intelligence, Machine Learning and Deep Learning
Apache MXNet ODSC West 2018
Main principles of Data Science and Machine Learning
Scaling Deep Learning with MXNet
Deep Learning with Python (PyData Seattle 2015)
Hardware Acceleration for Machine Learning
Deep learning introduction
Artificial Intelligence (AI) INTERNSHIP.pptx

Similar to prace_days_ml_2019.pptx (20)

PPTX
Deep learning
PDF
Introduction to Deep learning Models.pdf
PDF
Deep Learning & Tensor flow: An Intro
PDF
Deep learning
PPTX
Deep learning: the future of recommendations
PPTX
GDSC BPIT ML Campaign.pptx
PDF
GDSC Machine Learning Session Presentation
PPTX
Automatic Attendace using convolutional neural network Face Recognition
PDF
From neural networks to deep learning
PPTX
Deep_Learning_Introduction for newbe.pptx
PPTX
Deep Learning: Towards General Artificial Intelligence
PDF
深度学习639页PPT/////////////////////////////
PDF
Machine learning in science and industry — day 4
PDF
Tutorial on Deep Learning
PPTX
Introduction to deep learning
PPTX
Introduction to deep learning
PDF
Deep Learning for Personalized Search and Recommender Systems
PPTX
Deep Learning Jump Start
PPTX
Deep learning with TensorFlow
Deep learning
Introduction to Deep learning Models.pdf
Deep Learning & Tensor flow: An Intro
Deep learning
Deep learning: the future of recommendations
GDSC BPIT ML Campaign.pptx
GDSC Machine Learning Session Presentation
Automatic Attendace using convolutional neural network Face Recognition
From neural networks to deep learning
Deep_Learning_Introduction for newbe.pptx
Deep Learning: Towards General Artificial Intelligence
深度学习639页PPT/////////////////////////////
Machine learning in science and industry — day 4
Tutorial on Deep Learning
Introduction to deep learning
Introduction to deep learning
Deep Learning for Personalized Search and Recommender Systems
Deep Learning Jump Start
Deep learning with TensorFlow
Ad

More from RohanBorgalli (14)

PPT
Genetic Algorithms.ppt
PPT
SHARP4_cNLP_Jun11.ppt
PPT
Using Artificial Intelligence in the field of Diagnostics_Case Studies.ppt
PPT
02_Architectures_In_Context.ppt
PPTX
Automobile-pathway.pptx
PPTX
Autoregressive Model.pptx
PPT
IntrotoArduino.ppt
PPTX
Dimension Reduction Introduction & PCA.pptx
PPT
Telecom1.ppt
PPT
FactorAnalysis.ppt
PDF
Time Series Analysis_slides.pdf
PPTX
Image captions.pptx
PDF
NNAF_DRK.pdf
PDF
R Programming - part 1.pdf
Genetic Algorithms.ppt
SHARP4_cNLP_Jun11.ppt
Using Artificial Intelligence in the field of Diagnostics_Case Studies.ppt
02_Architectures_In_Context.ppt
Automobile-pathway.pptx
Autoregressive Model.pptx
IntrotoArduino.ppt
Dimension Reduction Introduction & PCA.pptx
Telecom1.ppt
FactorAnalysis.ppt
Time Series Analysis_slides.pdf
Image captions.pptx
NNAF_DRK.pdf
R Programming - part 1.pdf
Ad

Recently uploaded (20)

PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
PPTX
Foundation to blockchain - A guide to Blockchain Tech
PDF
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
PDF
Unit I ESSENTIAL OF DIGITAL MARKETING.pdf
PPTX
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
PPTX
Fundamentals of safety and accident prevention -final (1).pptx
PPT
Total quality management ppt for engineering students
PPTX
Fundamentals of Mechanical Engineering.pptx
PDF
BIO-INSPIRED HORMONAL MODULATION AND ADAPTIVE ORCHESTRATION IN S-AI-GPT
DOCX
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
PDF
R24 SURVEYING LAB MANUAL for civil enggi
PPTX
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
PPTX
Internet of Things (IOT) - A guide to understanding
PDF
Human-AI Collaboration: Balancing Agentic AI and Autonomy in Hybrid Systems
PDF
A SYSTEMATIC REVIEW OF APPLICATIONS IN FRAUD DETECTION
PDF
PREDICTION OF DIABETES FROM ELECTRONIC HEALTH RECORDS
PPTX
UNIT 4 Total Quality Management .pptx
PPTX
Current and future trends in Computer Vision.pptx
PDF
Level 2 – IBM Data and AI Fundamentals (1)_v1.1.PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
Foundation to blockchain - A guide to Blockchain Tech
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
Unit I ESSENTIAL OF DIGITAL MARKETING.pdf
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
Fundamentals of safety and accident prevention -final (1).pptx
Total quality management ppt for engineering students
Fundamentals of Mechanical Engineering.pptx
BIO-INSPIRED HORMONAL MODULATION AND ADAPTIVE ORCHESTRATION IN S-AI-GPT
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
Embodied AI: Ushering in the Next Era of Intelligent Systems
R24 SURVEYING LAB MANUAL for civil enggi
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
Internet of Things (IOT) - A guide to understanding
Human-AI Collaboration: Balancing Agentic AI and Autonomy in Hybrid Systems
A SYSTEMATIC REVIEW OF APPLICATIONS IN FRAUD DETECTION
PREDICTION OF DIABETES FROM ELECTRONIC HEALTH RECORDS
UNIT 4 Total Quality Management .pptx
Current and future trends in Computer Vision.pptx
Level 2 – IBM Data and AI Fundamentals (1)_v1.1.PDF

prace_days_ml_2019.pptx

  • 1. vscentrum.be Introduction to machine learning/AI Geert Jan Bex, Jan Ooghe, Ehsan Moravveji
  • 2. Material • All material available on GitHub • this presentation • conda environments • Jupyter notebooks 2 https://github.com/gjbex/PRACE_ML or https://bit.ly/prace2019_ml
  • 3. Introduction • Machine learning is making great strides • Large, good data sets • Compute power • Progress in algorithms • Many interesting applications • commericial • scientific • Links with artificial intelligence • However, AI  machine learning 3
  • 4. Machine learning tasks • Supervised learning • regression: predict numerical values • classification: predict categorical values, i.e., labels • Unsupervised learning • clustering: group data according to "distance" • association: find frequent co-occurrences • link prediction: discover relationships in data • data reduction: project features to fewer features • Reinforcement learning 4
  • 5. Regression Colorize B&W images automatically https://tinyclouds.org/colorize/ 5
  • 7. Reinforcement learning Learning to play Break Out https://www.youtube.com/watch?v=V1eY niJ0Rnk 7
  • 8. Clustering Crime prediction using k-means clustering http://www.grdjournals.com/uploads/articl e/GRDJE/V02/I05/0176/GRDJEV02I0501 76.pdf 8
  • 10. Machine learning algorithms • Regression: Ridge regression, Support Vector Machines, Random Forest, Multilayer Neural Networks, Deep Neural Networks, ... • Classification: Naive Base, , Support Vector Machines, Random Forest, Multilayer Neural Networks, Deep Neural Networks, ... • Clustering: k-Means, Hierarchical Clustering, ... 10
  • 11. Issues • Many machine learning/AI projects fail (Gartner claims 85 %) • Ethics, e.g., Amazon has/had sub-par employees fired by an AI automatically 11
  • 12. Reasons for failure • Asking the wrong question • Trying to solve the wrong problem • Not having enough data • Not having the right data • Having too much data • Hiring the wrong people • Using the wrong tools • Not having the right model • Not having the right yardstick 12
  • 13. Frameworks • Programming languages • Python • R • C++ • ... • Many libraries • scikit-learn • PyTorch • TensorFlow • Keras • … 13 classic machine learning deep learning frameworks Fast-evolving ecosystem!
  • 14. scikit-learn • Nice end-to-end framework • data exploration (+ pandas + holoviews) • data preprocessing (+ pandas) • cleaning/missing values • normalization • training • testing • application • "Classic" machine learning only • https://scikit-learn.org/stable/ 14
  • 15. Keras • High-level framework for deep learning • TensorFlow backend • Layer types • dense • convolutional • pooling • embedding • recurrent • activation • … • https://keras.io/ 15
  • 16. Data pipelines • Data ingestion • CSV/JSON/XML/H5 files, RDBMS, NoSQL, HTTP,... • Data cleaning • outliers/invalid values?  filter • missing values?  impute • Data transformation • scaling/normalization 16 Must be done systematically
  • 17. Supervised learning: methodology • Select model, e.g., random forest, (deep) neural network, ... • Train model, i.e., determine parameters • Data: input + output • training data  determine model parameters • validation data  yardstick to avoid overfitting • Test model • Data: input + output • testing data  final scoring of the model • Production • Data: input  predict output 17 Experiment with underfitting and overfitting: 010_underfitting_overfitting.ipynb
  • 18. From neurons to ANNs 18 𝑦 = 𝜎 𝑖=1 𝑁 𝑤𝑖𝑥𝑖 + 𝑏 𝑥 𝜎 𝑥 activation function 𝑤1 𝑤2 𝑤3 𝑤𝑁 𝑥1 𝑥2 𝑥3 𝑥𝑁 ... 𝑏 𝑦 +1 inspiration
  • 19. Multilayer network 19 How to determine weights?
  • 20. Training: backpropagation • Initialize weights "randomly" • For all training epochs • for all input-output in training set • using input, compute output (forward) • compare computed output with training output • adapt weights (backward) to improve output • if accuracy is good enough, stop 20
  • 21. Task: handwritten digit recognition • Input data • grayscale image • Output data • digit 0, 1, ..., 9 • Training examples • Test examples 21 Explore the data: 020_mnist_data_exploration.ipynb
  • 22. First approach • Data preprocessing • Input data as 1D array • output data as array with one-hot encoding • Model: multilayer perceptron • 758 inputs • dense hidden layer with 512 units • ReLU activation function • dense layer with 512 units • ReLU activation function • dense layer with 10 units • SoftMax activation function 22 array([ 0.0, 0.0,..., 0.951, 0.533,..., 0.0, 0.0], dtype=f 5 array([ 0, 0, 0, 0, 0, 1, 0, 0, 0, 0], dtype=ui Activation functions: 030_activation_functions.ipynb Multilayer perceptron: 040_mnist_mlp.ipynb
  • 23. Deep neural networks • Many layers • Features are learned, not given • Low-level features combined into high-level features • Special types of layers • convolutional • drop-out • recurrent • ... 23
  • 24. Convolutional neural networks 24 1 ⋯ 0 ⋮ ⋱ ⋮ 0 ⋯ 1 
  • 25. Convolution examples 25 1 ⋯ 0 ⋮ ⋱ ⋮ 0 ⋯ 1 0 ⋯ 1 ⋮ ⋱ ⋮ 1 ⋯ 0 1 ⋯ 0 ⋮ ⋱ ⋮ 0 ⋯ 1 0 ⋯ 1 ⋮ ⋱ ⋮ 1 ⋯ 0 Convolution: 050_convolution.ipynb
  • 26. Second approach • Data preprocessing • Input data as 2D array • output data as array with one-hot encoding • Model: convolutional neural network (CNN) • 28  28 inputs • CNN layer with 32 filters 3  3 • ReLU activation function • flatten layer • dense layer with 10 units • SoftMax activation function 26 array([[ 0.0, 0.0,..., 0.951, 0.533,..., 0.0, 0.0]], dtype 5 array([ 0, 0, 0, 0, 0, 1, 0, 0, 0, 0], dtype=ui Convolutional neural network: 060_mnist_cnn.ipynb
  • 27. Task: sentiment classification • Input data • movie review (English) • Output data • Training examples • Test examples 27 Explore the data: 070_imdb_data_exploration.ipynb / <start> this film was just brilliant casting location scenery story direction everyone's really suited the part they played and you could just imagine being there Robert redford's is an amazing actor and now the same being director norman's father came from the same scottish island as myself so i loved the fact there was a real connection with this film the witty remarks throughout the film were great it was just brilliant so much that i bought the film as soon as it 
  • 28. Word embedding • Represent words as one-hot vectors length = vocabulary size • Word embeddings • dense vector • vector distance  semantic distance • Training • use context • discover relations with surrounding words 28 Issues: • unwieldy • no semantics
  • 29. How to remember? Manage history, network learns • what to remember • what to forget Long-term correlations! Use, e.g., • LSTM (Long Short-Term Memory • GRU (Gated Recurrent Unit) Deal with variable length input and/or output 29
  • 30. Gated Recurrent Unit (GRU) • Update gate • Reset gate • Current memory content • Final memory/output 30 𝑧𝑡 = 𝜎 𝑊 𝑧𝑥𝑡 + 𝑈𝑧ℎ𝑡−1 𝑟𝑡 = 𝜎 𝑊 𝑟𝑥𝑡 + 𝑈𝑟ℎ𝑡−1 ℎ′𝑡 = tanh 𝑊𝑥𝑡 + 𝑟𝑡 ⊙ 𝑈ℎ𝑡−1 ℎ𝑡 = 𝑧𝑡 ⊙ ℎ𝑡−1 + 1 − 𝑧𝑡 ⊙ ℎ′𝑡
  • 31. Approach • Data preprocessing • Input data as padded array • output data as 0 or 1 • Model: recurrent neural network (GRU) • 100 inputs • embedding layer, 5,000 words, 64 element representation length • GRU layer, 64 units • dropout layer, rate = 0.5 • dense layer, 1 output • sigmoid activation function 31 Recurrent neural network: 080_imdb_rnn.pynb
  • 32. Caveat • InspiroBot (http://inspirobot.me/) • "I am an artificial intelligence dedicated to generating unlimited amounts of unique inspirational quotes for endless enrichment of pointless human existence". 32

Editor's Notes

  • #13: https://www.kdnuggets.com/2018/07/why-machine-learning-project-fail.html