CSC668 - PCS716

Machine Learning
Dr Muhammad Sharjeel
 01
Introduction and
Basic Concepts
 Who are you?
 Why are you here?
 How well do you know me?

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Dr Muhammad Sharjeel (Assistant Professor)
 What have I done?
 PhD, Computational Linguistics, Natural Language Processing, Machine Learning
 Lancaster University, Lancaster, United Kingdom
 MS, Computer Science, Network Systems
 Swinburne University of Technology, Melbourne, Australia
 BS, Computer Science
 Peshawar University, Peshawar, Pakistan

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Dr Muhammad Sharjeel (Assistant Professor)
 What I teach?
 15 years of teaching experience

 Design and Analysis of Algorithms  Introduction to Computing

 Data Security and Encryption  Computing for Management
 Information Security  Introduction to Computer Programming
 Introduction to Data Science  Data Structures and Algorithms
 Programming Fundamentals  Wireless and Mobile Computing
 Advanced Algorithm Analysis  Network Security
 Machine Learning  Introduction to Information and
Communications Technologies

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Dr Muhammad Sharjeel (Assistant Professor)
 What kind of research do I do?
 Computational Linguistics, Natural Language Processing, Machine Learning
 Natural Language Processing and Machine Learning
 Paraphrase Detection, Paraphrase Types Detection, Paraphrase Generation
 Short Story Generation, Story Conclusion Generation
 Natural Language Processing for Cyber Security
 Forensic linguistics, Vulnerability detection, Abusive language detection
 Natural Language Processing for HealthCare
 Autism Spectrum Disorder detection, Diabetes detection, Clinical text analyses
 Natural Language Processing for Requirements Engineering
 F/NF Rs categorization, Requirements Documents Summarization

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 I like to
 interact with my students
 ask questions
 be asked questions
 give assignments and projects
 I like to learn

 I am very bad at memorizing names (not faces)

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Be punctual
 Don't be afraid to speak (not talk)
 Cell phones and other electronics are not prohibited in the lecture
 However,
 recording or taking pictures in the class is strictly prohibited
 Please ask before doing!

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Email: [email protected] Office: PhD Faculty Block - 116
 Office Hours: Thursday, 4:30-5:30 (before the start of weekly lecture)
 WhatsApp for course related / general discussion
 Email for appointments, meeting requests, or for research related discussion
 Course material available on Google Drive shared folder:
 http://tinyurl.com/fa24ml
 Assessment:
 Midterm = 25%
 Quiz/Assignments = 25%
 Terminal = 50%

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 A Course in Machine Learning (Hal Daumé III)
 Link: http://ciml.info/

 Introduction to Machine Learning with Python (Sarah Guido, Andreas C. Muller)

 Introduction to Machine Learning (Ethem Alpaydın)

 Machine Learning (Tom Mitchell)

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Please get in touch with me if you’re unsure of whether or not you’re at the right
level for this course
 My guess is that you are!
 Read about Docker and Jupyter on the web
 https://www.docker.com
 https://www.jupyter.org
 Make a GitHub account
 https://www.github.com/
 Download and install Anaconda (Python)
 https://www.anaconda.com/download
 https://www.python.org
 Start using Google Colab and get yourself familiarize with it
 https://colab.research.google.com/

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Gettings started with Python
 https://realpython.com/tutorials/machine-learning/

 Machine Learning with Python

 https://machine-learning-with-python.readthedocs.io/en/latest/

 Python Crash Course, 3rd Edition: A Hands-On, Project-Based Introduction to

Programming - 3rd Edition – by Eric Matthes
 https://ehmatthes.github.io/pcc_3e/

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Pakistan.AI
 https://www.facebook.com/groups/pakdotai/

 National Centre of Artificial Intelligence

 https://ncai.pk/

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 We will study the underlying concepts and algorithmic ideas in the field of ML
 What will you learn?
 A broad theoretical and practical understanding of ML algorithms
 Ability to implement and optimize the algorithms
 Identify where ML can be applied (in your research area) and make the most appropriate
decisions

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Artificial Intelligence (1956) aims to create intelligent machines that can replace or
exceed human intelligence
 Machine Learning (1997) is a subset of AI that enables machines to learn from
existing data to make decisions or predictions
 Deep Learning (2017) is a technique within ML which utilizes layers of neural
networks to process data and make decisions
 Generative AI (2021) uses DL models (e.g., Generative Adversarial Networks) to
generate high-quality text, images, and other content based on the data they were
trained on

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 A growing and fast moving field
 Open field, lots of room for new work
 Broad applicability
 Finance, entertainment, Natural Language Processing, Information Retrieval, Computer
Vision, robotics, healthcare, medicine, biology, etc.
 What can ML do?
 Product recommendations
 Language translation
 Monitor unusual behaviour
 Predict stock prices
 Traffic alerts
 Virtual personal assistants
 Fraud detection

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 “Intelligent” machines can even be “smarter” than humans!
 Beat the best Go player
 Solving Rubik’s Cube with a robot hand
 https://towardsdatascience.com/the-decade-of-artificial-intelligence-6fcaf2fae473

 What’s your favourite ML application?

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Can we predict how a viewer would rate a movie?
 Why would we do it in the first place?
 To make better movie recommendations (get more rentals)
 “In 2006, Netflix announced they would pay $1 million to anyone in the world who
could improve their recommendation algorithm by 10%”.

 https://strategybreakdowns.com/p/netflix-million-dollar-prize

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 8000 tickets for the Neural Information Processing Systems (NeurIPS 2018)
conference sold out in 11 minutes 38 seconds
 13,000 people attended the NeurIPS 2023

 Machine learning is everywhere!

 And you are probably already using it
 And you are probably already impacted by it

 https://www.forbes.com/sites/samshead/2018/09/05/ai-researchers-left-disappointed-as-nips-sells-out-in-under-12-
minutes/

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Is an email spam?
 Can you name all the people in a photo?
 If I like these three movies, what should I watch next?
 Will a stock price go up or down tomorrow? By how much?
 Could you find the best ads for a website?
 How to read that Dutch website in English?
 Does this genetic marker correspond to Alzheimer’s disease?
 What is the 3 dimensional structure of this protein?

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 The best way to understand Machine Learning is by simply doing it
 i.e., by discussing tasks and algorithms

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 The Names Game
 Each enrolee of the FA24 class of ML at CUI, Lahore Campus was labelled either
+ or –
 Only the instructor knows the method that generated the labels
 Hint: the function is depended only on the enrolee's name
 Task for the Enrolees
 Look at the examples (names) and find the hidden function

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 “The studies reported here have been concerned with the programming of a digital
computer to behave in a way which, if done by human beings or animals, would be
described as involving the process of learning”
A.L. Samuel
IBM Journal of Research and Development, 3 (3), 10–229, 1959

 “A computer program is said to learn from experience E with respect to some class of
tasks T and performance measure P if its performance at tasks in T, as measured by P,
improves with experience E ”
Tom Mitchell, Machine Learning, 1997

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Machine Learning is about predicting the future based on the past
 More formally, Machine Learning is about predicting the unseen based on the already
seen
 Even more formally, Machine Learning is a field of study that gives computers the
ability to learn without being explicitly programmed

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 How to solve a specific problem?
 Write a computer program that encodes a set of rules to solve the problem
 At times is difficult to specify those rules, e.g., given a picture determine whether there is a
cat in the image
 Learning systems are not directly programmed to solve a problem, instead develop
own program based on:
 Examples of how they should behave
 From trial-and-error experience trying to solve the problem
 Different than standard CS:
 Want to implement unknown function, only have access to input-output

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 In ML, a computer program is said to learn
 from experience E
 with respect to some class of tasks T, and
 performance measure P
 We evaluate if the program performance at tasks in T as measured by P improves
with experience E

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Example:
 Learn to drive a car
 Class of tasks (T)
 Starting a car, changing gear, control on accelerator and breaks, parking a car etc.
 Performance Measure (P)
 Efficiency
 Experience (E)
 Number of hours spent in driving the car
 A person (program) is said to have learned to drive a car, if his/her performance P on
task(s) T is improving with experience E

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Examples:
 Learn to play checkers
 Class of tasks (T) : playing checkers
 Performance measure (P) : percent of games won against opponents
 Experience (E) : playing practice game against itself
 Learn to recognise handwriting
 Class of tasks (T) : recognizing and classifying handwritten words within images
 Performance measure (P) : percent of words correctly classified
 Experience (E) : a database of handwritten words with given classifications
 Learn to drive a robot
 Class of tasks (T) : driving a robot on the public highways using vision sensors
 Performance measure (P) : average distance travelled before an error
 Experience (E) : a sequence of images and steering command recorded

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Learning is an adaptive change in behaviour caused by experience (Shepherd, 1988)
 To learn something is the one's ability to use previous knowledge to perform future
actions
 Suppose you took a new course this semester (e.g., Mathematics)
 You expect to “learn” something from that course
 What is a common way to judge how well you do?
 You did well at learning, if you do well on the exam

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 What makes a reasonable exam?
 If it has chemistry questions, it’s not representative of your learning
 Remember the course was mathematics
 If it only has questions that were already solved in the lectures, that’s a bad test of your
learning
 The best practice would be
 You study and understand the concepts with examples during the lectures
 The exam then have “new” but “related” questions
 The good exam would test your ability to “generalize”

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Learning is captured in a model learned from discovering patterns in data
 Patterns are discovered through model parameters
 Data is provided as training examples
 Trained model should be a good and useful approximation of the data
 The trained model is used later for prediction
 All of the above is achieved using a learning algorithm
 Each algorithm has a “bias”

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Let’s try out an example

Class A Class B

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Different learning algorithms/models have different hypotheses
 Inductive bias, some hypotheses are more probable than others

Flying?

Bird?

Mammal?

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Suppose that you moved to Mexico, the original home of the papaya
 You never encountered this wonderful fruit before
 You visit the local market and have to learn to buy the good ones
 You observe that their
 Color ranges from dark green through orange and red to deep brown
 Softness ranges from rock hard to mushy
 Once you start buying and noting whether or not you like them, you create your very
own dataset

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Based on your observations, you have the following rules to buy a papaya
 You never like dark green ones
 Orange and red ones are invariably tasty
 Deep brown ones are very good, provided they are not mush
 You have never encountered a rock hard papaya
 Congratulation, you have learned a rule based classifier

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 To learn such a rule list,
 Search for a one-feature condition that covers as many tasty papayas and as few not tasty
ones as possible
 conditions like Colour = green
 Refine this rule with extra conditions until you have a good rule
 Remove all covered examples and start again

 An example rule would be:

If Colour = Orange or Red Then Tasty
Else If Colour = Deep Brown And Softness ≠ Mushy Then Tasty
Else Not Tasty

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Although rule lists are easy to find, they are not necessarily easy to interpret
 At times
 the condition of a rule implicitly contains the negation of all rules above it
 By aggressively going for the best rule first the next rules may become unnecessarily
convoluted
 Better to take a more balanced approach

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Lables Training Data

A Example-1

B Example-2 Learning Algorithm

A Example-3

A Example-4

B Example-5

Lables

?
Test Data

Example-1
f
? Example-2

Predicted Lable

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 ML algorithms can be classified into three main groups
 Supervised (predictive) Learning: The goal is to learn a mapping from inputs x to
outputs y given the labelled set S = {(x1, t1), (x2, t2), . . . , (xN , tN)}
 xk is a feature vector
 When ti ∈ {1, 2, . . . , C}, it is called classification
 When ti ∈ R, it is called regression
 Unsupervised (descriptive) Learning: The goal is to find interesting pattern in data S =
{x1, x2, . . . , xN}
 More typical of human and animal learning
 Reinforcement Learning: The goal is to learn by interacting with the environment
 Learning agent learns from the consequences of its actions

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Supervised Learning:
 Instructor supplies a collection of examples with answers (labels)
 Learner has to learn (from the provided data) to label new unseen examples
 Unsupervised Learning:
 No instructor, learner has only unlabeled examples
 Learner has to find out a way to find patterns (assign labels) within the unlabeled examples
 Reinforcement Learning:
 No instructor, no data
 Learner learns by interacting with the environment

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Supervised Learning:
 Classification (Fraud Detection, Sentiment Analysis)
 Regression (Weather Forecasting, Population Growth Estimation)
 Unsupervised Learning:
 Dimensionality Reduction (Meaningful Compression, Big Data Visualisation)
 Clustering (Customer Segmentation, Recommender Systems)
 Reinforcement Learning:
 Real-time Decisions, Robot Navigation, Skill Acquisition

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel
 Training phase
 ML algorithm tries to learn from the training data and outputs a ML trained model which
can then be used to make predictions
 Training phase creates the “model” using the “data” (training data)
 Testing phase
 Performance of the trained model (created in the training phase) is evaluated (on the test
data) using the evaluation measure(s)
 Testing phase checks the “error” in the “model” using “data” (test data)
 Trained model (or model) is used to make predictions on new (unseen) data
 If a model produces an accuracy of 80% on a large test set, then presumably it will
correctly classify 80% of unseen data
 If a trained model performs well on large test data, it will perform well on real-time data

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 For both training and testing, we need data, therefore, we split the available data into
 Train data (or train set)
 Test data (or test set)
 Standard approach for data split
 2/3 train set
 1/3 test set
 Important: Train set and test set must be disjoint, i.e., example(s) in the train set should
not occur in the test set and vice versa
 Two main approaches to data split
 Random split
 Class balanced split

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Dataset = 600 instances (A = 300, B = 300) [balanced]
 Train/Test split ratio is 67%-33%
 Random split
 Train set = 400 instances (A = 175, B = 225)
 Test set = 200 instances (A = 125, B = 75)
 Class balanced split
 Train set = 400 instances (A = 200, B = 200)
 Test set = 200 instances (A = 100, B = 100)

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Dataset = 900 instances (A = 600, B = 300) [un-balanced]
 Train/Test split ratio is 67%-33%
 Random split
 Train set = 600 instances (A = 500, B = 100)
 Test set = 300 instances (A = 100, B = 200)
 Class balanced split
 Train set = 600 instances (A = 400, B = 200)
 Test set = 300 instances (A = 200, B = 100)

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Batch method
 All training examples are used at once to compute the hypothesis
 Incremental method
 All training examples are used iteratively to refine a current hypothesis, one at time and
randomly
 On-line method
 The training examples are used as they become available, one at a time

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Deductive Learning
 Information
 Pattern
 Tentative Hypothesis
 Theory

 Inductive Learning
 Theory
 Hypothesis
 Observation
 Confirmation

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Deductive Learning:
 Works on existing facts and knowledge
 Does not generate "new" knowledge at all
 Makes the reasoning system more efficient
 Example:
 Concept to be learned - Throwing a ball in air
 How Deductive Learning works?
 We know Newton's Law of Gravitation
 So, we conclude that if we let a ball go, it will certainly fall downwards

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Inductive Learning:
 Takes examples of a concept and generalizes rather than starting with existing knowledge
 Generates “new” knowledge
 Has “scope of error”

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

 Inductive Learning:
 Example:
 Concept to be learned - Throwing a ball in air
 How Inductive Learning works?
 Take examples of the concept to be learned
 1 example – 1 time we throw a ball in the air
 50 examples – 50 times we throw a ball in the air
 100 examples – 100 times we throw a ball in the air
 Learn from examples, we throw a ball 100 times in the air and learned that every time we
throw the ball in the air, it falls downward
 Generalize the concept learned from examples
 We conclude that if we let a ball go, it will certainly fall downwards

CSC668 - PCS716 – Machine Learning Dr Muhammad Sharjeel

Thanks