Framing a

Machine Learning Problem

Facilitators:
Rahman, Brian, Eva, Andrew, George,
Mark, Peter, Confred
 Today`s Agenda


Defining a ML problem and proposing a
solution;
 Identifying good ML problems
 Deciding on ML
 Formulating a problem as an ML problem

ML Bootcamp Sept 16 - Oct 7, 2023

Defining a ML problem and proposing a
solution

ML Bootcamp Sept 16 - Oct 7, 2023

 Defining a ML problem

ML – process of training a software (or model)
to make predictions by learning from data

Branches of ML
 Supervised learning
 Unsupervised / self-supervised learning
 Reinforcement learning

ML Bootcamp Sept 16 - Oct 7, 2023

 Kinds of ML problems

Supervised and unsupervised ML problems fall under
multiple categories

ML Problem Type Description Example

Classification Predict label for previously Identify image of dog from that of
unseen example cat, bicycle from motor bike
Regression Predict numerical values Predicting price of houses

Clustering Group similar examples Most relevant documents

(unsupervised)
Association rule Infer likely association If you buy a bed, you are likely to
learning patterns in data buy a mattress too (unsupervised)
Structured output Create complex output Image recognition bounding
boxes
Ranking Identify position on a scale Search result ranking in a search
or status ML Bootcamp Sept 16 - Oct 7,engine
2023
 
Check Your Understanding


https://developers.google.com/machine-
learning/problem-framing/cases#check-
your-understanding

ML Bootcamp Sept 16 - Oct 7, 2023

 The ML Mindset


"Machine Learning changes the way
you think about a problem. The focus
shifts from a mathematical science to a
natural science, running experiments
and using statistics, not logic, to
analyse its results." - Peter Norvig -
Google Research Director

ML Bootcamp Sept 16 - Oct 7, 2023

 Experimental Design


Scientific method
 It is helpful to think of the ML process as an
experiment where we run test after test after test
to converge on a workable model
 Like an experiment, the process can be exciting,
challenging, and ultimately worthwhile

ML Bootcamp Sept 16 - Oct 7, 2023

 Step Example

1. Set the research goal I want to predict how heavy traffic will be
on a given day.
2. Make a hypothesis I think the weather forecast is an
informative signal for traffic prediction!
3. Collect the required data Collect historical traffic data and weather
data on each day
4. Test your hypothesis Train a model using this data to predict
traffic.
5. Analyze the results you get Is this model better than existing systems
for traffic prediction?
6. Draw a conclusion I should (not) use this model to make
traffic predictions, because of X, Y, and Z.
7. Refine your hypothesis and Time of year could be a helpful signal for
repeat traffic
ML Bootcamp Sept prediction?
16 - Oct 7, 2023
 Identifying good problems for ML

Characteristics of a good ML problem
 Clear use case
 * Start with the problem, not the solution. Make sure you aren't treating ML as a
hammer for your problems
 Focus on problems that would be difficult to solve with traditional
programming e.g,

Smart Reply – automated email reply, saves user time

Google Photos – find a specific photo by keyword search without
manual tagging

* ML solves problems by examining patterns in data/adapting with them
 Ask yourself the following questions,

What is the problem being faced?

Would it be a good problem for ML?
ML Bootcamp Sept 16 - Oct 7, 2023
 Identifying good problems for ML

Characteristics of a good ML problem...
 Know the problem before focusing on the data
 * Be prepared to have your assumptions challenged
 Once you`ve clear understanding of problem, list potential
solutions to test in order to generate the best model

Understand that you`ll have to try out a few solutions before you
land on a good working model
 EDA helps you understand your data, but you can't yet
claim that patterns you find generalize until you check
those patterns against previously unseen data

Failure to check could lead you in the wrong direction or reinforce
stereotypes or bias
ML Bootcamp Sept 16 - Oct 7, 2023
 Identifying good problems for ML

Characteristics of a good ML problem...
 Data, data, more data
 * ML requires a lot of relevant data
 Data collected specifically for your task is most useful

In practice, secondary data is used in majority of applications
 How much is a lot? - depends on the ML problem

but more data will improve your model (e.g, robustness) and
it's predictive power. A good rule of thumb is to have at least
000`s of examples for basic linear models, and 100`s of
000`s for neural networks. If you have less data, consider a
non-ML solution first and/or transfer learning methods
ML Bootcamp Sept 16 - Oct 7, 2023
 Identifying good problems for ML

Characteristics of a good ML problem...
 Predictive Power
 * Your features should contain predictive power
 Ensure your data set contains relevant features that
correlate with the phenomenon being investigated

e.g, is bedroom count a good predictor for house prices?

Don`t try out features arbitrarily without a hypothesis
 Your goal is to build a model that generalizes well to
previously unseen samples and this is possible only
if you use the right features
ML Bootcamp Sept 16 - Oct 7, 2023
 Identifying good problems for ML

Characteristics of a good ML problem...
 Predictions vs. Decisions
 * Aim to make decisions, not just predictions
 Your product take action on output of ML model

ML better at making decisions than deriving insight from
data (for the latter, use statistical approaches)

Ensure predictions allow you to take a useful action e.g,
a model that predicts likelihood of clicking certain videos
could allow a system to pre-fetch the videos most likely to
be clicked

ML Bootcamp Sept 16 - Oct 7, 2023

Examples of prediction / decision pairs

Prediction Decision
What video the learner Show those videos in the
wants to watch next recommendation bar

Probability someone will If P(click) > 0.12, prefetch

click on a search result. the web page

What fraction of a video If a small fraction, don't

ad the user will watch show the user the ad

ML Bootcamp Sept 16 - Oct 7, 2023

 Hard ML problems

Clustering
 What does each cluster
mean in an unsupervised
learning problem? E.g, if
your model indicates that
the user is in the blue
cluster, you'll have to
determine what the blue
cluster represents
 Semi-supervised learning
may help

ML Bootcamp Sept 16 - Oct 7, 2023

 Hard ML problems...

Anomaly detection
 how do you decide what constitutes an anomaly
to get labeled data?

ML Bootcamp Sept 16 - Oct 7, 2023

 Hard ML problems...

Causation
 ML can identify correlations – mutual
relationships or connections between two or
more things. Determining causation (one event
or factor causing another) is harder. It is easy to
see that something happened, but much harder
to understand why it happened
 You can't determine causation from only
observational data – you need to run
experiments

ML Bootcamp Sept 16 - Oct 7, 2023

 Hard ML problems...

No data
 if you have no data to train a model, then ML
cannot help you. Without data, use a simple,
heuristic, rule-based system
 Some new products with no training data start
with a heuristic rule system, and obtain training
data only after users interact with it

ML Bootcamp Sept 16 - Oct 7, 2023

 Deciding to use ML

Set yourself up for success by thinking about these
things before trying to frame a problem for ML
 Start clearly / simply – what would you like the ML model to
do for you?

e.g. I want the ML model to predict the price of a house
 What is your ideal outcome?

e.g tourism recommendations – my ideal outcome is to suggest
tourism destinations that tourists find attractive and worth their
time and money
 Success and failure metrics

Quantify it, measurable, what output would you like the ML model
to produce (based on type of ML problem),
ML Bootcamp Sept 16 - Oct 7, 2023
Formulate problem as an ML problem
1) Suggested approach for framing ML problem
1) Articulate your problem
2) Start simple
3) Identify your data sources
4) Design your data for the model
5) Determine where data will comes from
6) Determine easily obtained inputs
7) Ability to Learn
8) Think about potential Bias
ML Bootcamp Sept 16 - Oct 7, 2023
 Articulate your problem

Is it a classification, regression, clustering,
anomaly detection problem?

ML Bootcamp Sept 16 - Oct 7, 2023

 Articulate your problem

Write down a succint problem statement
 e.g. Our problem is best framed as 3-class, single-
label classification, which predicts whether a video
will be in one of three classes—{very popular,
somewhat popular, not popular}—28 days after
being uploaded

ML Bootcamp Sept 16 - Oct 7, 2023

 Start simple

Simply the problem further if possible e.g,
 We will predict whether an uploaded video is likely
to become popular or not (binary classification)
 We will predict an uploaded video’s popularity in
terms of the number of views it will receive within a
28 day window (regression)

Start by using the simplest model (baseline) possible for
your ML problem

ML Bootcamp Sept 16 - Oct 7, 2023

 Identify your data sources

Provide answers to the following questions about your
labels:
 How much labeled data do you have?
 What is the source of your label?
 Is your label closely connected to the decision you will be
making?

Example
 Our data set consists of 100,000 examples about past
uploaded videos with popularity data and video descriptions.

ML Bootcamp Sept 16 - Oct 7, 2023

 Design your Data for the Model

Identify the data that your ML system should
use to make predictions (input -> output),
Title Channel Upload time Uploaders recent Output
videos (label)
My silly cat Alice 2018-03-21 08:00 Another cat video, Very popular
yet another cat
A snake video Bob 2018-04-03 12;00 None Not popular

ML Bootcamp Sept 16 - Oct 7, 2023

 Determine Where Data Comes From

Assess how much work it will take to develop a data
pipeline to construct each column for a row. When does
the example output become available for training
purposes?

Example
 We applied the labels {very popular, somewhat popular, not
popular} to each video that fell within a determined range of
views and "thumbs ups" and determined keyword descriptions
for each video. Hand-generating descriptions is not sustainable,
so we are considering adding a keyword description to the
upload form.

ML Bootcamp Sept 16 - Oct 7, 2023

 Determine Easily Obtained Inputs

Pick 1-3 inputs that are easy to obtain and that
you believe would produce a reasonable, initial
outcome
 Consider the engineering cost to develop a data
pipeline to prepare the inputs, and the expected
benefit of having each input in the model

ML Bootcamp Sept 16 - Oct 7, 2023

 Ability to Learn

Will the ML model be able to learn? List aspects
of your problem that might cause difficulty
learning. For example:
 The data set doesn't contain enough positive labels.
 The training data doesn't contain enough examples.
 The labels are too noisy.
 The system memorizes the training data, but has
difficulty generalizing to new cases.

ML Bootcamp Sept 16 - Oct 7, 2023

 Think About Potential Bias

Many datasets are biased in some way. These
biases may adversely affect training and the
predictions made e.g,
 A biased data source may not translate across
multiple contexts
 The training sets may not be representative of the
ultimate users of the models and may therefore
provide them with a negative experience

ML Bootcamp Sept 16 - Oct 7, 2023

 Conclusion

It is important to frame your problem properly
for ML


Not all problems require or need to be solved
using ML

ML Bootcamp Sept 16 - Oct 7, 2023

 Quiz

Complete the quiz at this link
https://elearning.umu.ac.ug/mod/quiz/attempt.
php?attempt=15240&cmid=17874

ML Bootcamp Sept 16 - Oct 7, 2023