Certificate in Data Science for

Investment Professionals
COMING SOON!

Understand the application of data science in the investment process

and act as a guide at the intersection of data scientists, investment
professionals and clients

Are you ready for the machine learning Course topics

revolution?
Course 1 Data and Statistics Foundation
The Certificate in Data Science for Investment
Professionals provides you with practical
Course 2 Statistics for Machine Learning
knowledge of machine learning fundamentals
and how they are used in the investment
Course 3 Machine Learning
process.
This certificate is designed to enable you Course 4 Natural Language Processing
to apply machine learning concepts to real-
world investment problems and explain them Course 5 Mitigating Biases in the
Data Science Pipeline
clearly to a non-expert audience and clients.
90-minute multiple-choice
Final Exam exam (online)
Master the language of data science to
better serve your clients
The certificate is composed of five courses
with practical application exercises and one
final exam. You will be able to: Key facts
• Describe and evaluate machine learning
• Launching Spring 2023
techniques
• Select and develop data visualizations • Beginner level certificate
using Python • Online, self-paced learning
• Apply machine learning to address • Time to complete: 90 – 100 hours
investment problems
• Content access: 12 months
• Explain machine learning techniques to a
non-expert audience • Recognition: Digital badge
• Use natural language processing to make
investment decisions
• Evaluate and mitigate machine learning
biases

© 2023 CFA Institute. All rights reserved.

1
Data Science for Investment Professionals Glossary

Course 1 – Data and Statistics Foundation Confidence Interval

For a statistic calculated from a sample, a confidence
Acceptance Region
interval is a range that would be expected to enclose
In a hypothesis test, the acceptance region is a range
the true population value most of the time (e.g., 90%
of test statistic results (sample outcomes) in which
of the time for a 90% confidence interval).
the null hypothesis is accepted (i.e., we cannot rule out
chance as an explanation for the observed results).
Contingency Table
A contingency table is a table with columns
Alternative Hypothesis
representing one categorical variable and rows
In a hypothesis test, the alternative hypothesis is
representing another. The cells reflect counts of
constructed so that it represents the conclusion
data falling into the two categories. For example, for
you reach if the null hypothesis is rejected. Together,
investor data, the rows might represent male/female
the null hypothesis and alternative hypothesis must
and the columns might represent owns stocks/
account for all possible outcomes to the study.
doesn’t own stocks. The cell in the “male” row and
the “doesn’t own stocks” column is the count of male
Arithmetic Mean
investors in the data who do not own stocks.
The sum of a set of values, divided by the number of
values in the set (also known as the average).
Continuous Data
Continuous data are numeric data that can be
Array Data
measured on an infinite scale (i.e., between any two
Array data is a collection of similar data types stored
points there are an infinite number of values a data
contiguously. A vector is an of array data, and a
point can assume).
two-dimensional matrix is also an array of data.
A spreadsheet range is an array.
Critical Value
In a hypothesis test, the critical value is a sample
Bar Chart
statistic that is just barely extreme enough (compared
A bar chart represents items or categories on one axis
with what you would expect under the null model)
and counts or values for those items on the other axis.
to be deemed improbable enough to reject the null
hypothesis.
Bias in Estimation
Estimation typically involves samples taken to
Cross-Sectional Data
measure some quantity or fit a model in a larger
Cross-sectional data are data from many units or
population or process, and bias is error in a consistent
records at one point in time. It is like taking a snapshot
direction across multiple samples.
of all details at a point in time.
Categorical Data
Cumulative Frequency Distribution
Data that indicate which category a record belongs
A cumulative frequency distribution can be visualized
to (e.g., for a resident’s home: condo, single-family
as a two-dimensional plot, with possible data values
dwelling, rental apartment). Ordered categorical data
(or ranges) arrayed in ascending order on the x-axis,
represents categories that can be ranked by magnitude
with the y-axis representing the proportion of the data
(e.g., no pain, mild pain, moderate pain, severe pain).
falling at or below the x-value (or range).
Central Limit Theorem
Deciles
The central limit theorem states that the distribution
Deciles are segments of the data that are delineated
of the sample mean approaches normality in its shape
by ranking the observations and then segmenting
as the sample size increases, even if the underlying
them into tenths.
data are not normally distributed.

Discrete Data
Coefficient of Variation
Discrete data assume values on a scale that is finite,
The coefficient of variation for a dataset is the ratio of
or countably infinite. Integer scales are a common
the standard deviation to the mean.
example, but the data need not be in integer form.

2
Data Science for Investment Professionals Glossary

Estimate in a chance model (the “null hypothesis” embodying

In statistics, an estimate is a statistic calculated from, the assumption of no treatment effect, or nothing
or a model derived from, a sample of data drawn from unusual). If the outcome is sufficiently extreme
a larger population. compared with the chance model, the null hypothesis
is rejected and the outcome is deemed “statistically
False Positive significant.” The purpose of a hypothesis test is to
With binary data in which one category is of special protect against being fooled by random chance.
interest (typically designated as a “1” or positive),
a false positive is a record that is estimated by Interval Estimate
a model or diagnostic procedure to be a “1” but is, An interval estimate is a range bracketing a
in reality, a “0” (or negative). measurement from a sample. You see interval estimates
often in opinion poll results: “35% favor the proposition,
Family-Wise Error Rate (FWER) with a margin of error of 5% in either direction.”
In a single hypothesis test of one question, the
probability of mistakenly concluding a sample result Joint Probability
is significant, given the null model, is set at some low In considering two events, the joint probability is the
level, often 0.05. When multiple questions are asked probability that both will happen.
and tested, all at that 0.05 level, the probability of
such a mistake rises with the number of tests. This Kurtosis
multiple test error probability is the FWER. Kurtosis is a measure of the extent to which a data
distribution is more or less spread out than a normal
Frequency Table distribution (i.e., has more or fewer values in its tails).
For a dataset, a frequency table tabulates the counts
of how many observations have the various values, or Kurtosis, Leptokurtic or Fat-Tailed
how many observations fall within the various ranges More extreme or distant observations occur than with
(bins). The values or bins are arranged contiguously to a normal distribution.
cover all the data, and zero-count values or bins are
included. Kurtosis, Mesokurtic
Extreme or distant observations occur to the same
Geometric Mean extent as in a normal distribution.
The geometric mean of a set of n values is found by
multiplying the values together and then taking the Kurtosis, Platykurtic or Thin-Tailed
nth root of the product. The geometric mean is often Fewer extreme or distant observations occur than
used to describe proportional growth. with a normal distribution.

Harmonic Mean Look-Ahead Bias

The harmonic mean of a set of values is calculated Bias that results when a simulation study or a model
by dividing the number of values by the sum of their uses information that will not be available at the time
reciprocals. The geometric mean is often used to of real-word deployment.
average rates (e.g., price multiples, speeds).
Mean Absolute Deviation (MAD)
Histogram Mean absolute deviation is the average absolute
A histogram is a plot of a frequency table. One axis deviation from the mean.
(usually the x-axis) has the data’s possible values
or bin ranges, and the other has the counts or Median
proportions of observations that take a given value or After a dataset has been ranked according to the
fall in a given bin range. magnitude of one variable, the median is the record
in the middle (i.e., half of the records lie above and
Hypothesis Testing half below).
A hypothesis test is a statistical procedure applied
to sample outcomes in a study or experiment to test Mode
how far an outcome departs from what might occur The mode is the most frequent value in a dataset.

3
Data Science for Investment Professionals Glossary

Monte Carlo Simulation Percentiles

A Monte Carlo simulation is the repeated execution of After a dataset has been ranked according to the
an algorithm that incorporates a random component; magnitude of one variable, a percentile, k (from 0 to
each time the trial is repeated, this component is 100), indicates what percentage of the records lie
rerandomized. below k. For example, below the 15th percentile lie the
lowest 15% of the records (according to the ranked
Mutually Exclusive Events variable).
Events (or, more precisely, outcomes to an event) are
mutually exclusive if only one of them can occur. Pie Chart
A pie chart is a circle, with pie-shaped segments
Nominal Data whose sizes indicate the portion of the data that falls
Nominal data are data, usually non-numeric, that in the category represented by that segment.
provide no quantitative information. For example, the
terms “transportation,” “communications,” and “tech” Point Estimate
to describe business sectors are nominal. A point estimate is a measurement based on a
sample. It is a single value, which distinguishes it
Null Hypothesis from an interval estimate.
In a statistical hypothesis test, the null hypothesis
embodies the proposition you are hoping to disprove: Population
the treatment has no effect on blood pressure, the In a study or modeling problem, a population is the
new web page does not increase click throughs. entire data of interest. Often we deal only with a
It is represented by a probability model reflecting specified sample of the data, particularly when the
this proposition, such as all the blood pressure population as a whole is difficult to define or reach in
readings grouped together in a single hat, or a single specific practical terms (e.g., “sports fans”).
click-through rate for both web pages.
Population Mean
Numerical Data The average value of a variable of interest for the
Numeric data are data that are expressed in numbers population; this is often a theoretical construct if all
that convey quantitative information; numbers that members of the population are not available.
simply represent categories are not included.
Population Parameter
Ordinal Data The value of some measure of a population (mean,
Ordinal data are rank data in which the values standard deviation, percentile); this is often a
(usually integers) represent rank order, but no further theoretical construct if all members of the population
quantitative information (a financial stability value of are not available.
4 does not imply 33% more stability than 3.)
Probability
Panel Data Most people have an intuitive sense of what
Panel data are data for a group of subjects over time probability means, such as “the chance of rain is
(e.g., vaccine recipients monitored monthly). 40%.” To a gambler, the probability of winning a game,
or of correctly specifying the outcome of an event
Pareto Chart such as tossing a coin, is the proportion of times the
A Pareto chart for a dataset is a bar chart in which the game is won, or the coin outcome happens. This is the
bars represent categories of some variable of interest, “frequentist” definition of probability: the outcome of
and the height of a bar represents the percent of interest as a proportion of all outcomes to an event,
records in the category for that bar. The bars are if the event could be repeated indefinitely.
arranged in descending order of height and are often
supplemented by a line plot above them representing Probability Density Function
cumulative percentages. Pareto charts are often used A probability density function is a function that can be
to attribute a condition to likely primary contributors used to evaluate the probability that a value will fall in
(e.g., 40% of aircraft incidents are due to pilot error, a certain range.
25% to air traffic control, 20% to maintenance issues).

4
Data Science for Investment Professionals Glossary

Probability Distribution they reduce bias. Stratified sampling operates

A probability distribution is a tabular or graphical hierarchically (e.g., select schools randomly from
display of the probability of values in a population; among schools, select classes from the selected
it is like an extension of the frequency distribution. schools, select students from the selected classes).
Common theoretical probability distributions are the In systematic sampling, a rule governs selection
uniform (all outcomes equally likely), the binomial (e.g., every 10th record).
(think of it as representing the number of heads in
n tosses of a coin that has a p probability of landing Sample Selection Bias
heads), and the bell-shaped normal (think of it as a Selection bias occurs when the sampling process
smoothed version of the binomial distribution where n yields samples that are not representative of the
is large). population you want to study. For example, in 1936,
the Literary Digest poll predicted Alf Landon would
P-Value beat Franklin Roosevelt for president, largely because
A hypothesis test estimates the probability that a the Digest relied on, among other things, lists of
chance “null hypothesis” model would produce an owners of telephones (a luxury good at the time).
outcome as extreme as an observed value (e.g., a value
such as the difference between a treatment sample Sample Statistic
and a control sample). This probability is the p-value. A sample statistic is a metric (e.g., mean, standard
deviation) calculated from a sample.
Quantiles
Quantiles are equal-size segments of a ranked Sampling Error
dataset, common quantiles being quartiles (fourths), Samples drawn from a population typically differ
quintiles (fifths), and deciles (tenths). The “second from one another, as do the metrics (e.g., means,
quintile,” for example, would span from the 20th to the proportions) calculated. Sampling error is a general
40th percentile. term referring to the variability in these metrics.

Random Variable Significance Level

A random variable is a function or rule that assigns In a hypothesis test, the significance level is a low
a real number to each element in a sample space. percentage (typically 5% or 1%) that sets a cutoff in
For example, in flipping a coin, the sample space the distribution of the test statistic under the null
elements are heads and tails, and the rule could be hypothesis. Beyond this cutoff point, the result is
“1 if heads, 0 if tails.” The number assigned could deemed sufficiently extreme so as not to be attributed
be discrete (from a finite or countably infinite set of to chance and is, therefore, statistically significant.
numbers) or continuous (any real value).
Skewness
Range Skewness is departure from symmetry in the
In describing a dataset, the range is the largest value distribution of data, or in the distribution of a statistic.
minus the smallest value. If the tail of the distribution extends more to the right,
the skew is positive. If the tail extends more to the
Rejection Region left, the skew is negative.
In a hypothesis test, the rejection region is the range
of sample statistics deemed too improbable under the Standard Deviation
null hypothesis to be attributable to chance. Typically, The standard deviation of a dataset is the square root
this means the 5% or 1% of values in the tail(s) of a of the variance.
sample statistic’s distribution.
Standard Error
Sample The standard error of a sample statistic (say, the
A sample is a set of observations or records drawn mean of a sample) is the standard deviation of that
from a larger population or process. Samples can statistic, which is calculated from many sample
be drawn by a variety of methods to well-represent results. If the larger population is not available to draw
the population; samples selected randomly (simple additional samples, bootstrap sampling of the original
random samples) are statistically useful because sample can be used to estimate the standard error.

5
Data Science for Investment Professionals Glossary

Standard Normal Distribution Tree Map

The standard normal distribution is a normally A tree map is a rectangular representation of
shaped data distribution with a mean of 0 and a categories and subcategories in a dataset. Interior
standard deviation of 1. Data are often normalized or blocks of different sizes represent, by their size,
standardized by subtracting the mean and dividing it the relative magnitudes of the categories and
by the standard deviation; this gives the data a mean subcategories they represent. Colors may also be
of 0 and a standard deviation of 1, but it does not used to group subcategory blocks into categories.
make the data normally distributed if they were not
normally distributed to begin with. Type I Error
In a hypothesis test, a type I error is mistakenly
Structured Data concluding an effect (e.g., the effect of an intervention
Structured data are data that are tabular and in an experiment) is real, as opposed to being the
quantitative (or categorical) in nature. Structured data result of random chance. A type I error is also known
are contrasted with unstructured data (e.g., a set of as a false positive, which happens when a null
Tweets, or doctor’s notes). hypothesis is rejected when it is actually true.

Survivor Bias Type II Error

Survivor bias occurs when the entire population This type of error, also known as false negative,
is of interest but only the surviving members of a occurs when a null hypothesis is accepted when it is
population are examined. For example, in studying actually not true.
an industry, looking only at firms that are currently in
operation can cause survivor bias. Unstructured Data
Unstructured data are data that are not tabular and
T-Distribution quantitative (e.g., a set of Tweets, or doctor’s notes).
The t-distribution is a bell-shaped distribution that is
used to describe, among other things, the distribution of Variance
a sample mean, or the difference between two sample For a set of values, the variance is the average of the
means. It is a family of distributions, whose shapes squared deviations from the mean.
(peaked versus flat) depend on the sizes of the samples.
Weighted Mean
Test Statistic For a set of values, a weighted mean is calculated
In a statistical study, the test statistic is what you by multiplying each value by a weight, and then by
measure (e.g., mean, correlation, difference in means, dividing by the sum of the weights. It is used when
ratio) in a sample or samples to answer the research some values are deemed more important than others.
question of the study.
Z-Score
Time-Period Bias The z-score for a value from a dataset is calculated by
Time-period bias occurs when the time period for an subtracting the mean of the dataset, and dividing it by
analysis is not representative of a longer period. The the standard deviation of the dataset. This process is
opportunity for bias is higher when the time period is sometimes called standardization or normalization.
relatively short and when it is chosen specifically to
illustrate a point.

Time-Series Data
Time-series data are measurements of one or more
variables over time, with the measurements equally
spaced in time.

6
Data Science for Investment Professionals Glossary

Course 2 – Statistics for Machine Learning Binomial Probability Distribution

The binomial probability distribution is the frequency
Adjusted R2
distribution of the “success” outcomes of a set of n
R2 is a measure of the proportion of variation in data
probabilistic trials (e.g., flipping a coin) each having
that is explained by a linear model. Adjusted R2 is a
constant success probability of p. Success, or 1 (with
modified version that penalizes added predictors, to
0/1 designations), is an arbitrary designation and is
encourage parsimony. It is:
usually the class of interest or the more unusual class.
1 − (1 − R2)((n − 1)/(n − p − 1)),
Black-Box Models
Black-box machine learning models generate
where n is the number of records, and p is the number
predictions without yielding usable information about
of predictors.
the relationships between predictor values and
outcomes. Neural networks and ensembles of trees
Analysis of Variance
are examples of black-box models.
Analysis of variance (ANOVA) is a statistical method
for assessing the differences among multiple groups,
Central Limit Theorem
where each group is summarized numerically
The central limit theorem states that the distribution
(typically with the mean). In particular, the method
of a sample mean gradually becomes normally shaped
evaluates whether the differences are statistically
as the size of the samples (all drawn from a common
significant.
population) increases, even if the observations from
the common population is not normally distributed.
Autoregressive Time Series Model
It was very important in the early days of statistics
An autoregressive time series model is one that
when it enabled formula alternatives to the
forecasts future values in a time series on the basis of
resampling and permutation procedures that made up
prior values.
the foundation of inference.
Bayes Theorem
Character Data
Bayes theorem is a formula for calculating the
Character data do not convey any quantitative
probability of an event, A, given that some other event,
information, such as letters, punctuation, or even
B has occurred, P(A|B), when you know the reverse:
numbers that are part of a text.
P(B|A). The formula is:

Coefficient of Determination
P(A|B) = P(B|A) * P(A) /P(B).
The coefficient of determination, or R2, is a measure of
the proportion of variation in data that is explained by
For example, if you know the prevalence of a disease,
a linear model.
and the probability of a positive diagnostic test if you
have the disease, and the general rate of positive
Collectively Exhaustive Events
tests, you would use Bayes formula to calculate
In probability theory, events are collectively
P(disease|positive test).
exhaustive if at least one of them must occur. In ice
hockey, for example, a team’s outcome must be one of
Bernoulli Distribution
three events: win, loss, or tie.
The Bernoulli distribution is a special case of the
binomial in which there is just one trial. For each
Conditional Probability
event, there is only two possible outcomes.
Conditional probability is the probability that some
event occurs given that some other event or condition
Bias–Variance Trade-off
has occurred. For example, the probability that a web
Statistical and machine learning prediction algorithms
visitor purchases a product during a session, given
can underperform in two ways: (1) stable predictions
that they have purchased a product in the past.
that fall short in accuracy (bias); or (2) accurate
predictions that are unstable, depending on the
data sample (high variance). Typically, there is
a trade-off: improve accuracy (minimize bias) and
decrease stability (increase variance).

7
Data Science for Investment Professionals Glossary

Continuous Values random behavior. Determining the volume of a sphere

Where measurements can take any numerical value in if you know its radius is a deterministic model. A model
a relevant range, those values are termed continuous might be effectively deterministic—for example,
values. Depending on the precision of a thermometer, determining stopping distance using full braking
for example, a temperature might be measured at power for a car traveling a certain speed, even though
12 degrees, or 12.5, or 12.39, or 12. 387. minor random elements might be involved. The
application of a deterministic model to a situation
Correlation that clearly involves random behavior (e.g., using a
Correlation between two variables exists when linear model to predict sales) imparts a false degree
they move in tandem: one goes up and the other of confidence.
goes up, and vice versa. Often, “correlation” is used
synonymously with “correlation coefficient,” which Discrete Values
is a measure of the degree of linear correlation that Where quantitative measurements can take specific,
ranges from +1 (perfect positive correlation) to −1 usually evenly spaced values, those values are
(perfect negative correlation). termed discrete values. The number of occupied seats
on an airplane is a discrete value (e.g., 76 seats might
Cost Function be occupied, or 77, but not 76.5).
See Loss Function.
Dummy Variables
Counterfactual Dummy variables are 0/1 variables created to record
A counterfactual in machine learning is a modification the presence or absence of a category. For example,
to data that is used to explore model behavior, for a stock might be small cap, mid cap, or large cap.
example “suppose the input had been x2 instead of Most machine learning algorithms require numeric
x1: how would the output change?” Counterfactuals inputs, so this single categorical variable might be
are used in interpretability methods for black-box recast as three binary dummies to indicate whether
models. that category applies: small cap (0/1), mid cap (0/1),
and large cap (0/1).
Counterfactual Thinking
In social psychology, counterfactual thinking is the Empirical Probability
human tendency to imagine or explore alternative An empirical probability is a probability estimate
paths of possible prior events. based on prior data. For example, the probability that
a company’s overall market cap will rise more than 1%
Covariance Stationary following a stock split is best estimated by what has
A time series is covariance stationary if it does not happened in similar circumstances in the past. This is
change over time. Specifically, its mean, variance, and distinguished from a theoretical probability based on
autocorrelation remain constant. physical knowledge and assumptions: the probability
that a coin flip will land heads can be estimated based
Data Drift on knowledge of the coin and the flipping process,
Machine learning models are trained on a given set without having to flip thousands of coins.
of data. After the model is deployed in a pipeline
with new data, the characteristics (e.g., the mean Entropy
or variance) of the input data may change, in which In information theory, entropy is randomness or,
case, the model may no longer be appropriate and particularly with respect to the transmission of
require retraining. information, confusion, or disorder. In machine learning,
entropy means heterogeneity or impurity in outcomes.
Dependent Variable It is a popular cost function (or loss function) in
In a machine learning algorithm, the dependent decision trees, where the more homogeneous the
variable is the variable to be predicted. Also termed classes in a node, the better (the tree’s rules are doing
the response, target, or outcome. a good job separating the classes). In a classification
tree, entropy ranges from 0 (completely homogeneous)
Deterministic Models to 1. It is calculated as the sum of -pkln(pk) across the
A deterministic model is one that has all the outcome classes k, where p is the proportion of records
information needed to explain or predict an outcome belonging to class k.
with certainty, and it does not explicitly incorporate

8
Data Science for Investment Professionals Glossary

Explained Variation Homoskedasticity

The effectiveness of statistical and machine learning Homoskedasticity is when the variance of a variable
modeling can be measured by the extent to which a changes little over different ranges of the data.
model, with its predictions, can match the variation
in the independent (outcome) variable. Metrics, such Independent Events
as R2, measure this effectiveness: R2 = 0 means Events are independent if knowing the outcome of
that none of the variation in the outcome variable is one event does not affect the probability of the other.
explained by the model, while R2 = 1 means that all of Quarterly advertising revenue for Meta is probably
the variation is explained by the model. independent of quarterly deaths from infectious
diseases. But it is not independent of quarterly
F-Distribution change in gross domestic product.
The F-distribution is a probability distribution of the
ratio of variances of two samples drawn from the Independent Variable
same (normal) population. It is used in testing null An independent variable is a variable (also called a
hypotheses about whether variances are equal. It is feature or attribute) that is included as a predictor
also used in analysis of variance (ANOVA) procedures in a statistical or machine learning model. The term
in which the goal is to determine whether two or more “independent variable” is used in statistics, and terms
samples of numerical data are statistically different like “feature” or “attribute” tend to be used more in
from one another. machine learning.

Features Labels
Features are measurements of characteristics in In machine learning, labels are another term for
a dataset that vary from record to record and are “outcome variable,” particularly in cases in which
useful in prediction. Synonyms for “features” are the value must be assigned by human review
as follows: predictor variables (from statistics), (e.g., pictures of cats or dogs in training data).
fields (from database management), columns, and
attributes. Some examples of numerical features are Linear Regression
as follows: sales, number of customers, and assets. Linear regression is a statistical procedure that
Some examples of categorical features (which are estimates a linear relationship between predictor
often binary) are as follows: purchase/don’t purchase, variables (also called features or independent variables)
survive/die, level of education, and low/mid/large cap. and a numerical outcome. It typically does so by
minimizing the squared errors or residuals (i.e., the
Gradient Descent differences between predicted and actual values).
Machine learning algorithms typically seek to
minimize predictive error or a related loss function, Logistic Regression
and neural networks do so through some form of The goal of a logistic regression is to predict the
iterative trial and error. Gradient descent is one probability of being a 1 in binary data. A linear model
popular form of this trial-and-error process. It can be is used, but the output must be scaled so that it
likened to hiking over hills and valleys in search of the lies between 0 and 1. This is done by using the logs
lowest point, following a rule that says to proceed of all terms and fitting a standard linear model.
in whatever direction takes you downhill from your After the model is fit, the coefficients can then
current position. This does not guarantee a global be exponentiated so that relationships between
minimum, but the algorithm can be adapted so that it predictors and the output can be interpreted. The
is not necessarily stopped by a local minimum. output is now in terms of odds, which can be
converted to probability.
Heteroskedasticity
Heteroskedasticity is when the variance of a variable Lognormal Distribution
changes over different ranges of the data. For The lognormal distribution is a distribution of data
example, wages may vary only a little for people with that is long-tailed to the right. When the logs of
a high school education but may vary considerably the data in a lognormal distribution are taken, the
more so for people with a graduate education. resulting transformed distribution becomes normally
distributed.

9
Data Science for Investment Professionals Glossary

Loss Function a roulette wheel has 38 slots: 18 red, 18 black, and

A loss (or cost) function is a measure of the degree 2 green (which cannot be selected as one of the
of error in a machine learning algorithm’s prediction, betting options). The odds of winning a red bet are
compared with actual values. Simple loss functions 18 to 20 (or 18:20), meaning that if you were to play
are sum of squared errors (for numeric data) and indefinitely you would lose 20 plays for every 18 that
percent misclassified (for categorical data). you win. This translates to a probability of 18/38 or
0.474. The odds can also be expressed as a ratio of
Multicollinearity probabilities, or p/(1 − p), or, for the roulette example,
Multicollinearity is when two or more predictors 0.474/0.526 = 0.901. An odds ratio is simply a ratio of
(features, attributes) are correlated with one another. two odds—for example, the odds of getting cancer
Multicollinearity in linear models can cause the given exposure to chemical X, divided by the odds of
model-fitting process to fail; one or more of the highly getting cancer without that exposure.
correlated predictors should be dropped. Because of
the high correlation, not much information will be lost. Optimization
Optimization is the process of “making the best
Mutually Exclusive Event possible.” The “possible” part of the definition is
Mutually exclusive events are events that cannot important, as optimization almost always involves
coexist or happen together, by definition. “Quarterly working within some limits. In business, it is typically
profit rises” and “quarterly profit falls” are mutually maximizing something (profit) or minimizing
exclusive events. something (cost), subject to constraints. In machine
learning, one typically optimizes the performance
Nonlinear Relationship metric (loss or cost function).
A nonlinear relationship is one that cannot be
described by a linear function. Some nonlinear Ordinal Scale
relationships become linear after a transformation of Ordinal data can be ordered or ranked by magnitude
the data (e.g., by taking the log). but are not necessarily arithmetically operable.
In ordering maximum attained education level as 1,
Nonparametric Hypothesis Test 2, 3, and 4, corresponding to elementary, secondary,
A nonparametric hypothesis test does not rely on college, and graduate, the most you can say is that
the data being normally-distributed. Most resampling each level is more than the preceding. You cannot say,
procedures are nonparametric. The assumptions for example, that level 4 is twice level 2.
about normally distributed data that accompany
standard formula-based hypothesis tests help Outliers
approximate the distribution of a test statistic under Outliers are values that are distant from the bulk
resampling. of the data in a dataset. The definition of “distant”
is arbitrary, although statistical rules of thumb
Normal Distribution are used. The handling of outliers depends on the
The normal distribution is a symmetric bell-shaped circumstances. Sometimes they signal a need for
curve. Its formula is not intuitive, but you can think of review and possibly signal removal as a distraction or
the normal distribution as the approximate result of a confirmation as valid. Sometimes they are just what
huge number of marbles being dropped down a pinball you are looking for (e.g., in anomaly detection).
board where, at each stage, the marble must bounce
left or right, before finally coming to rest in a set of Overfitting
columns. The columns at the extremes have only one Overfitting in machine learning results when a
path to them (repeated bounces left or right), and complex model is trained on data in such a way that
hence are short. The columns at the center have many it performs “too well” on the training data by virtue
arrival paths, and hence are taller. of learning not just the signal but the noise as well.
That same random noise will be different in new data,
Normalization so the model’s learnings will not only be useless but
See Scaling. counterproductive when deployed: interpreting random
noise as real patterns will introduce error. Data-centric
Odds Ratio models such as decision trees and neural networks
Odds are familiar to gamblers, being the comparison are especially susceptible to overfitting unless those
of one probabilistic event to another. For example, models are simplified or curtailed.

10
Data Science for Investment Professionals Glossary

Pearson Correlation Coefficient Serial Correlation

The Pearson correlation coefficient measures the Serial correlation is the presence of correlation
degree of linear association between two numeric between successive values in a time series.
variables, and ranges between −1 (perfect negative
correlation) and +1 (perfect positive correlation). It is Simple Linear Regression
calculated by averaging the product of the deviations Simple linear regression is a model that expresses one
from the mean for each variable, and then dividing by variable (the dependent variable) as a linear function
the product of their standard deviations: of another variable (the independent variable).

corr = [mean of (xi − mux)(yi − muy)]/(stdev x)(stdev y). Skewness

Skewness is the extension of one tail of a probability
Poisson Distribution distribution farther out than the other. If the right tail
The Poisson distribution is the frequency distribution is longer, it is referred to as “right skew,” and if the left
of the number of events per interval of time or area tail is longer, it is referred to as “left skew.”
of space, when there is a constant rate of random
events per time or space units. For example, the Spearman Rank Correlation Coefficient
number of people arriving at a queue in a specified The Spearman rank correlation coefficient for two
time span when the flow of customers is an average variables is found by converting the values in each
of 20 per hour, or the number of insects per square variable to ranks, and then calculating the Pearson
meter of beach when there are, on average, three per correlation coefficient.
square meter.
Standard Normal Distribution
R2 The standard normal distribution is a normal distribution
R2, or the coefficient of determination, is a measure of with a mean of 0 and a standard deviation of 1.
the proportion of variation in data that is explained by
a linear model. Standardization
See Scaling.
Random Walks
A random walk is a time series in which movements Statistical Regularity
from one period to the next are random. If a time Statistical regularity describes randomness in a
series is not distinguishable from a random walk, it is process that becomes predictable if the process
not likely to be predictable. continues long enough. The proportion of heads in
5 tosses of a coin is not predictable, but the proportion
Sample of heads in 5,000 tosses is quite predictable.
A sample is a set of observations or records from a
larger dataset. Often the sample is drawn randomly. T-Distribution
In the machine learning community, it sometimes has The t-distribution is a probability distribution that
a different meaning, especially when it is associated is symmetric, bell-shaped, and like the normal
more with computer science than statistics. In this distribution but with fatter tails. It is used in formulas
context, the term “sample” is often used to refer to to perform the t-test for A/B and other hypothesis
one record or observation. tests. It is really a family of distributions whose shape
and parameters depend on the sizes of the samples
Scaling in the tests.
Scaling is a data transformation that facilitates
placing multiple variables on the same scale, so that Test Statistic
their values fall roughly within the same range. Some A test statistic is a statistic (metric) applied to
machine learning algorithms fail if, for example, some samples in a hypothesis test. For example, in an A/B
variables are in the millions and others range between test of two web page designs, a relevant test statistic
0 and 1. There are several scaling options, including might be “proportion of clicks.”
standardization (subtracting the mean and dividing by
the standard deviation) and 0–1 scaling (i.e., “shrinking” Time Series
the data so that the maximum value is 1 and the A time series is a plot or table of successive values of
minimum is 0). a variable over time.

11
Data Science for Investment Professionals Glossary

Time-Series Model Variance Inflation Factor

A time-series model seeks to explain the pattern in a The variance inflation factor (VIF) is used to detect
time series and/or predict future values. multicollinearity among predictors in a (usually linear)
model. It is calculated for an individual predictor by
Training Set regressing that predictor against all others and finding
Training data are data used to fit (train) a machine the R2 for that model. If that R2 is high, then that
learning model. They are separate from holdout or predictor is largely explained by the other predictors,
validation data, where the model is used to make and there is likely to be multicollinearity. The VIF formula
predictions and the accuracy of those predictions can for a given predictor i uses the R2 from this model:
be measured.
VIFi = 1/(1 − R2i).
Transformation
Transformation of data is consistent modification of all Venn Diagram
the values of a variable to render them more tractable A Venn diagram is a two-dimensional visual depiction
for analysis. For example, data that seem to show an of possible outcomes of an event or categories of a
exponential trend can be transformed by taking the feature, in which a rectangle or circle represents a
log of all values, after which a linear analysis can be category or outcome, and the interior of the shape
performed. represents the observations that belong to that
outcome or category. Typically, the shapes intersect
Unconditional Probability to some degree and the space in the intersection
Unconditional probability is a probability (usually an area represents observations that belong to all the
estimate) that an event will occur, without considering categories or outcomes that the shapes enclosing
any other factors or conditions. For example, the that intersection space represent.
probability that any public firm will declare bankruptcy,
without taking financial history into account, is an
unconditional probability. The probability that a firm
will declare bankruptcy, given that they have run
losses for three years is a conditional probability.

Underfitting
Underfitting in a machine learning model is when
the model underperforms because there are some
patterns or relationships that it is failing to capture
effectively.

Unexplained Variation
Unexplained variation in a target or outcome variable
refers to that variation that is not explained by a
model. For example, in a revenue prediction model, the
differences between predicted revenue and actual
revenue constitute unexplained variation.

Validation Data
In a machine learning process, a model typically
learns from training data, and is then applied to
validation or holdout data, to see how well the model
predicts.

12
Data Science for Investment Professionals Glossary

Course 3 – Machine Learning CART

Classification and regression trees (CART) are
Accuracy
also called decision trees. The CART algorithm
In a machine learning model to predict categorical
repeatedly divides the dataset into partitions that are
data, accuracy is the proportion predicted correctly.
increasingly homogenous in the outcome variable,
resulting in a set of decision rules that predict a class
Agglomerative Clustering
or numerical value. For example, “If a web shopper has
Agglomerative clustering is a clustering algorithm that
made a prior purchase AND the shopper spends more
proceeds sequentially by grouping together nearby
than two minutes on the site AND the shopper clicks
clusters. At the initial stage, each record is its own
to view a product, THEN predict ‘purchase’.”
singleton cluster, and then the two closest records
are joined into a cluster. At each successive stage, the
Categorical Variable
two closest clusters are joined together.
A categorical variable can assume one of several
values that denote a category (e.g., a loan could be
Bagging
current, delinquent, or in default).
Bagging, or bootstrap aggregating, is the process of
repeatedly taking bootstrap samples of records and
Centroid
repeatedly applying a machine learning algorithm
The centroid of a dataset is the vector of the mean
to each sample. The results are then averaged, or
values of all the variables.
a majority vote is taken (for categorical outcomes),
yielding predictions that are more stable than those of
Classification Tree
individual models.
See CART.
Bayesian Classifier (or naive Bayesian classifier)
Clustering
In principle, an exact Bayesian classifier would
Clustering is the process of joining together similar
classify a record as follows: (1) find the records whose
records into a smaller set of clusters. The optimal
predictors have the same values as the record to be
number of clusters is determined by the analyst.
classified, and (2) find out what class is dominant
In technical terms, you seek clusters that are distinct
(or, if the outcome is numerical, find the average).
(distant) from other clusters, and whose members
Of course, with most data, it is unlikely that you will be
are close to one another. The organizational use
able to find exact matches. The “naive Bayes” shortcut
requirements are also considered: for example, for
drops the requirement for an exact match and works
customer clusters (segments), the sales process
with the independent probabilities (by class) of the
might deal well with 3 or 4 clusters, but not 20 or 30.
individual predictor categories.

Covariance Matrix
Bias Error
For multivariate data, a covariance matrix lists all the
Bias error is consistent predictive inaccuracy in one
variables both as row and column headers. Each cell
direction or the other and is not the result of random
is the covariance of the row and column variables.
variation.

Cross-Validation
Binary Classifier
Cross-validation is the repeated splitting of data into
A binary classifier is a machine learning algorithm that
model-fitting and model-assessment subsets. For
predicts the class of a record, where the record can
example, a model might be fit to 80% of the data, and
belong to one of two classes (e.g., a web visitor could
then assessed on the remaining 20%. This process
click or not, an account could be current or not).
is iterated, with a different 20% being held out for
assessment each time (or “fold”). The 80/20 split
Bootstrapping
would constitute five-fold cross-validation.
Bootstrapping in statistics is the process of repeatedly
taking samples with replacement from a dataset and
Cutoff Value
making an estimate from, or fitting a model to, each
A classification model generates an estimated
bootstrap sample. Bootstrapping is used to estimate
probability (propensity) that a record belongs to a
variability, and it also facilitate methods to avoid
particular class. The model algorithm, or the analyst,
overfitting in machine learning models.
can set a cutoff value (threshold value) for that
probability to distinguish records that are classified

13
Data Science for Investment Professionals Glossary

as a 1 (i.e., those records with a probability above the Euclidean Distance

cutoff) from those classified as a 0. The Euclidean distance between two records is the
square root of the sum of the squared differences
Decision Node between them.
In a decision tree, a decision node is a split in a
branch. The rule governing the split allocates records Forecasting
to one subbranch or another for further splitting. Forecasting is the prediction of future values in a time
series.
Deep Learning
Deep learning is machine learning using deep, or Hidden Layer
complex, neural networks. Deep neural nets (with In a neural network, a hidden layer contains nodes
many nodes and multiple layers), coupled with lengthy whose output is then fed into other nodes in a
training, are especially good at making sense of subsequent layer.
unstructured data (images and text) that lack the
high-level features found in tabular data. Hierarchical Clustering
Hierarchical clustering is the process of sequentially
Dendrogram creating clusters, either by repeatedly joining records
A dendrogram is a visual depiction of a hierarchical and then clusters together (agglomerative methods),
clustering process. Distance between clusters or by repeatedly dividing the data (divisive methods).
is represented on the y-axis, and clusters along The end result is a set of nested clusters, from a low
the x-axis. When two clusters are joined, they are level at which there are many small clusters (each
shown on the plot with two vertical lines linked by a record its own cluster in the extreme case) to the top
horizontal line at their between-cluster distance. level at which there is just one cluster (i.e., the entire
dataset).
Dimension Reduction
Too many features (dimensions) impair the Information Gain
performance of a machine learning algorithm by Many machine learning performance metrics
adding random noise associated with features that are based on prediction errors (losses or costs).
lack predictive value. Dimension reduction is the Another way of measuring performance is to look
process of reducing the number of features in a at information gain, which is the decrease in some
model, eliminating those that are not useful. metric of loss or error (e.g., entropy).

Divisive Clustering In-Sample Error

Divisive clustering is the process of creating In-sample error is a machine learning algorithm’s error
clusters (groups of records similar to one another) (i.e., the difference between predicted and actual
by repeatedly dividing the data into subgroups, values) in predicting training data.
increasing within-cluster homogeneity as you go.
Iterative Procedure
Ensemble An iterative procedure is one that is repeated over and
An ensemble in machine learning is a collection of over or “iterated,” often incorporating the results of
machine learning algorithms. Predictions from a set prior rounds.
of multiple machine learning algorithms, using the
average prediction (for numerical outcomes) or the K-Nearest Neighbor
majority-vote prediction (for classification), are usually k-Nearest neighbor is a prediction algorithm that
more accurate than those based on the individual assigns to a record the same class (or average value)
algorithms themselves. This is sometimes referred as a certain number (k) of records that are the closest
to as “the wisdom of the crowd,” and it is particularly to it (neighbors). K is set by the user, and Euclidean
useful when you are aggregating “weak learners,” or distance is a popular metric for measuring closeness.
algorithms with only modest predictive power for a
given task. K-Means Clustering
In k-means clustering, the analyst specifies in
Error Rate advance the number, k, of desired clusters. After
The error rate is the proportion of records that are an initial (usually random) division into k clusters,
misclassified in a machine learning model. records are iteratively reassigned to the closest

14
Data Science for Investment Professionals Glossary

cluster. The process stops when the only available output that is interpreted as predictions. Initially,
reassignments increase within-cluster dispersion. the predictions are essentially random, but they are
compared to actual values, and the weights that
Labeled Dataset govern the mathematical operations are modified in
A labeled dataset is one that has records whose additional passes through the data to improve the
outcomes are known, usually as a result of a human predictions.
review that assigns a label (e.g., an insurance claim
could be fraudulent or normal). Node
In a neural network, a node is a location where
LASSO weights are applied to inputs to generate an output:
LASSO (least absolute shrinkage and selection either an intermediate output that becomes an input
operator) is a regression method that penalizes to another node, or a final output that becomes a
regression coefficients in a way that causes prediction, which, in turn, is either final, or provisional
noninfluential predictors to drop from the model. This (if the latter, it is compared to the actual known value).
decreases variance in the coefficient estimates and
yields a model that is more parsimonious. Noise
Noise in data and models is variation that is not
Market Impact Analysis explainable by a statistical or machine learning model.
In finance, a market impact analysis is the
assessment of how much a purchase or sale of Nonparametric
securities affects the security price. Nonparametric statistical methods or estimates do
not incorporate assumptions about a normal (or other)
Mean Squared Error data distribution. Such assumptions reduce input
In machine learning, mean squared error (MSE) is the data to parameters (e.g., mean, variance) that are
average of the sum of squared errors (where errors often required for it to be used in many mathematical
are the differences between actual and predicted formulas.
values).
Out-of-Sample Error
Multilayer Convolutional Neural Network In implementing machine learning models, a subset
A convolutional neural network (CNN) is a deep neural of the data is typically held out (holdout or validation
net that adds convolutions to its learning process. data) to allow the model to be applied to data that
A convolution is an operation applied to multiple were not used to train the model. The model’s error on
observations at the same time, and it helps uncover these out-of-sample data is a more unbiased estimate
higher level features that are not discoverable at the of how the model will perform than the error on the
granular level. In processing images, for example, training data.
operations can be applied to small matrices of
pixels, moving sequentially across the image. These Output Layer
operations help reveal basic higher level features In a neural network, the output layer contains a node
(e.g., edges or lines) that are not discoverable at the whose output is the prediction (either final or, if there
individual pixel level but appear when you look at are to be further passes through the data, provisional).
multiple pixels.
Parsimonious Models
Natural Language Processing Parsimonious machine learning models are those that
Natural language processing (NLP) is the application use only essential and useful predictors and records.
of machine learning methods to natural languages, Including variables and records that do not provide
like English. The goal may be classification, extraction information that is useful for making predictions adds
of meaning, or generative (producing content). noise and degrades performance.
The older term “text mining” is sometimes used
synonymously. Partition
In machine learning, data are typically split into two or
Neural Network three subsets. The “training” partition is used to train
A neural network is a machine learning algorithm that or fit the model. A second partition is used to assess
performs numerous mathematical operations (both the model and tune (adjust) its parameters. A third
parallel and sequential) on input data, generating partition may be used at the end to estimate how well

15
Data Science for Investment Professionals Glossary

the model will do in the real world (no further model Semi supervised Technique
tweaking is allowed after this). The second partition In some machine learning situations, labeled data are
is sometimes called the validation partition and may scarce or expensive (e.g., insurance claims, which
also be called a “holdout” partition. The third partition require a human review to determine validity), but there
is also sometimes called the “holdout” partition and is is a lot of unlabeled data. In semi supervised learning,
also called the “test” partition. an initial model is trained on a limited set of trained
data, and then is applied to the unlabeled data to
Penalized Regression generate predictions. The predictions that have the most
Standard linear regression may include predictors that confidence (i.e., have the highest probability attached to
introduce instability, reflected in high variance for the their classification by the model) are then added to the
coefficients. This can result from multicollinearity, original labeled data as “pseudo-labeled data,” and a new
from the presence of noise, or when there are simply model is trained. This process is then repeated.
too many variables. Regression models can be
improved by penalizing coefficients if their variables Sentiment Analysis
do not contribute stable and useful information. Sentiment analysis is the analysis of human-generated
text to classify it to a sentiment (e.g., positive/negative,
Principal Component Analysis enthusiastic/indifferent).
Principal component analysis (PCA) transforms
multivariate data into a limited set of variables, each Supervised Learning
of which is a linear combination of others. The first Many machine learning models have as their goal
such variable or component seeks to capture as much the prediction of something, such as whether a
of the variability in the data as possible. This makes it web visitor will click on a link, how long a patient will
possible to create a parsimonious machine learning need to be hospitalized, or whether a firm presents a
model that uses just a handful of these newly created financial risk to stakeholders. Models are “trained” on
variables. data for which the outcome is known, and then are
applied to data for which the outcome is not known.
Pruning This process is termed “supervision.”
If allowed to grow indefinitely, decision trees overfit
the training data and do not perform well with new Tensor
data. An optimal tree is one that is not fully grown. A tensor is a multidimensional matrix map. A scalar
One way to achieve smaller trees is to allow them to is a tensor of rank 0, a vector is a tensor of rank 1,
grow fully, and then to prune them back to a point at a two-dimensional matrix is a tensor of rank 2, and
which they have minimal error on validation data. so on. Tensors are central to the ability of deep neural
nets to cope with complex data and features that are
Random Forest Classifier at a higher level than the input values (e.g., shapes
A form of bagging, a random forest is a collection of instead of pixel values); a popular deep learning
decision trees that are grown using a set of bootstrap environment is termed “TensorFlow.”
samples. The bootstrapping, or bagging, is done by
taking random samples, with replacement from the Terminal Node
records; random selection (without replacement) In a decision tree, a terminal node is a final split in a
is also applied to the predictors at each stage. The branch. The records resulting from the terminal node
results are then averaged, or a majority vote is taken split are assigned classes or values, and they are not
(for categorical outcomes), yielding predictions that split further.
are more stable than those of individual models.
Threshold Value
Regression Tree See Cutoff Value.
A regression tree is a decision tree where the
outcome is numerical. See CART. Unsupervised Learning
Unsupervised learning methods are machine learning
Reinforcement Learning methods that do not involve the prediction of an
Reinforcement learning is a machine learning method outcome variable. Clustering and recommendation
in which the algorithm interacts with people (e.g., engines are examples of unsupervised learning.
shoppers at a website) and, from their reactions,
learns the optimal assignment of options.

16
Data Science for Investment Professionals Glossary

Course 4 – Natural Language Processing metrics can be calculated from the confusion matrix,
including accuracy rate, error rate, sensitivity (recall),
Alternative Hypothesis
specificity, false discovery rate, and false omission
In a statistical hypothesis test, the alternative
rate.
hypothesis is the proposition you hope to
demonstrate, for example, that a treatment (e.g.,
Constraint
a new drug, or a different web page) had an effect
In optimization, constraints (e.g., limited supplies of
beyond what chance might produce.
labor and capital) are almost always present and are
an integral part of the optimization algorithm.
Area Under the Curve (AUC)
The area under the curve (AUC) is a metric that
Cross-Sectional Data
measures how effectively a predictive algorithm
Cross-sectional data are measurements of many
separates two classes. The curve in question is
different entities (e.g., people in a medical study, web
the ROC curve, and an area of 1 indicates perfect
visitors, emails) at the same time, or in contexts in
separation, while an area of 0.5 indicates that the
which time is not a factor.
model does no better than chance. See Receiver
Operator Characteristics Curve.
Curating Text Data
Text data encompasses a huge variety of topics and
Bag of Words
types, such as scholarly articles, tweets, news items,
In natural language processing, the bag of words
maintenance reports, medical notes, and dictionaries.
technique treats a document as simply a collection of
A curated corpus (body) of text is text from one
terms, without regard to order.
topic or type that has been selected, reviewed,
and processed to a sufficient degree that it can be
Binary Classification
subjected to analysis.
Binary classification algorithms predict classifications
for records that have two (binary) outcomes. Typically,
Data Frame
the class of interest is in the minority (e.g., a sale at a
A data frame is a 2 x 2 tabular set of data, like a
website, the presence of a disease, a fraudulent tax
spreadsheet range, with records as rows and columns
return), and is labeled 1, while the other records are
as variables.
labeled 0.

Data Mining
Chi-Square Test for Independence
Data mining is the science of gaining insight from data
The chi-square test of independence tests whether
and finding patterns. It encompasses techniques like
two or more samples differ in the proportion of 1’s
predictive modeling, clustering, principal component
to a greater extent than chance would allow, if the
analysis, and more. The term has largely been
samples were drawn from the same population with
superseded by the term “machine learning.”
a common proportion of 1’s. The test derives from a
chance model in which all samples are pooled, are
Data Wrangling
dealt out again in random order, and the 1’s in each
Data wrangling is the process of obtaining raw data
sample are counted.
and preparing it for analysis and modeling. Obtaining
the data, dealing with missing values, understanding
Complexity in a Model
variables, reconciling different sources, and achieving
In machine learning, a complex model is contrasted
consistent formatting are all part of data wrangling.
with a parsimonious one. A complex model will have
many variables and hyperparameters. It may fit the
Document Frequency
data better, but it is vulnerable to overfitting and
In natural language processing (NLP), document
misinterpreting noise as signal.
frequency is the number of documents in which a
term appears.
Confusion Matrix
A confusion matrix for a classification model is a
Document Term Matrix
2 x 2 table of predicted versus actual outcomes.
In natural language processing (NLP), the document
The columns typically represent predictions (1 or 0),
term matrix is a matrix in which rows are documents,
and the rows actual classes (1 or 0). The upper left
columns are terms, and the entries are the counts of a
cell, for example, represents the number or records
given term in the document for that row.
predicted as 1’s that are, in fact, 1’s. Numerous

17
Data Science for Investment Professionals Glossary

Entity Recognition Grid Search

In natural language processing (NLP), entity Most machine learning models have hyperparameters
recognition is a technique for identifying named (settings) controlled by the user, such as the number
entities (e.g., the names of individuals or of hidden layers in a neural network. One way of
organizations) in a set of documents. finding out which setting is best is to try multiple
combinations and assess model performance, and
Entity Resolution then use the hyperparameter combination that works
Entity recognition (or entity resolution) is a method for best. This process is a grid search.
identifying matches in a set of records of entities. For
example, if a person purchases something at a web Model Tuning
retailer, the company would like to know whether this Model tuning is the process of setting model
is an existing customer or contact. Sometimes the hyperparameters—for example, the number of layers in
customer will provide exactly the same information a neural network or how far to grow the decision trees.
as the company has on file and the match is easy, Tuning can be done manually by the user or can be
but more often, there may be some differences (e.g., done automatically.
missing information, spelling variations, the person
moved to a new address). Entity resolution can N-Gram
provide an estimate of the probability of a match. An N-gram is a sequence of N terms (or other units of
text) in a document. For example, “and seven years
Exploratory Data Analysis ago” is a four-gram (one of many four-grams) from
Exploratory data analysis consists of various President Lincoln’s Gettysburg Address. N-grams are a
techniques for human review and understanding of useful unit of analysis in natural language processing
a dataset. Descriptive statistics (e.g., mean, median, (NLP). You can see them in action when you type and
variance, outliers) are included, as are visualizations, your software suggests corrections: for example,
such as scatterplots, histograms, and boxplots. if you type “the output of a dep neural net” in a paper
Exploratory analysis typically precedes automated on machine learning, the software will reference
modeling. five-grams or six-grams containing similar correctly
spelled phrases and suggest “deep” as the correction.
False Negative
A false negative is a record that is classified as a 0 Parsimony in a Model
but is actually a 1. See also Binary Classification and In machine learning, a complex model is contrasted
Confusion Matrix. with a parsimonious one. A parsimonious model will
have relatively few variables and hyperparameters.
False Positive Other things being equal, a parsimonious model is
A false positive is a record that is classified as a 1 preferred to a complex one.
but is actually a 0. See also Binary Classification and
Confusion Matrix. Penalized Regression
In penalized regression, penalty terms are attached
Feature Engineering to regression coefficients, causing the least squares
Preparatory to fitting a statistical or machine learning algorithm to “shrink” (regularize) them. While this
model, some work is typically done on the features introduces bias into the coefficients, this bias is
(predictor variables). This includes understanding their offset by a reduction in variance of estimators and an
meaning, standardization (normalization), binning of increase in model simplicity and interpretability.
numerical variables if appropriate, consolidation of
categories if there are too many, deriving or finding Porter Stemmer Algorithm
new features, imputing missing values, and more. The Porter stemmer algorithm in natural language
processing (NLP) is a set of rules for removing suffixes
Feature Selection and reducing words to a common stem. See also
Feature selection is the process of selecting the Stemming.
features (variables) to be used in a machine learning
model. Often this involves distinguishing between the Precision
variables to be included in a model and those to be Precision in a machine learning classifier is the
left out. proportion of predicted 1’s that turn out to actually be
1’s. See also Confusion Matrix.

18
Data Science for Investment Professionals Glossary

Random Sampling classification and whose removal simplifies analysis.

Simple random sampling is the process of selecting Usually, modifications to the default generic list,
an observation from a dataset in such a way that each or substitution of a custom list, are permitted.
element in the dataset has an equal probability of
being selected. Other (non simple) forms of sampling Term-Frequency Inverse Document Frequency
incorporate randomness but lack this strict condition (TF-IDF)
(e.g., stratified sampling). TF-IDF helps identify document-term pairs that are
unusual, specifically where a term that is relatively
Recall (Sensitivity) rare overall appears in a document. TF-IDF yields
Recall in a machine learning classifier is the proportion relatively high values in such a case, and low values
of actual 1’s that are correctly classified as 1’s. See when the term is either absent from the document
also Confusion Matrix. or relatively common overall. Definitions of TF-IDF can
vary slightly, but according to a common definition,
Receiver Operator Characteristics (ROC) Curve it is the product of two things:
The receiver operator characteristics curve is the plot of • term frequency (TF) = # times term t appears in
a machine learning algorithm’s predictions, specifically document d, and
recall (the y-axis) and specificity (the x-axis, with 1 on • inverse document frequency (IDF) = log(total #
the left and 0 on the right) as you change the cutoff docs)/(# docs with term t) + 1.
for classifying a record as a 1. The name arose in World
War II to describe the performance of radar receiving Term-Frequency Matrix
stations, whose job was to determine whether radar The term-frequency matrix for a set of documents
blips were incoming aircraft. summarizes how often different terms appear.
Columns are terms, rows are documents, and the
Regularization entry for a given row and column indicates how often
See Penalized Regression. the term for the column appears in the document for
the row.
Scraping
Scraping is the automated collection of data from Tokenization
websites. Tokenization is the process of taking a text and, in
automated fashion, dividing it into separate terms or
Sensitivity “tokens,” which become the basic unit of analysis.
See Recall. A word separated by spaces is a token, “2 + 3” would
divide into three tokens, and “23” would be one token.
Shrinkage Models NLP software comes with default tokenizers, whose
See Penalized Regression. rules can be altered.

Specificity True Negative

Specificity in a machine learning algorithm is the In binary classification, a true negative is a 0 that
proportion of actual 0’s that are correctly classified was correctly classified as a 0. See also Binary
as 0. See also Confusion Matrix. Classification and Confusion Matrix.

Stemming True Positive

The huge scale of natural language vocabularies In binary classification, a true positive is a 1 that
poses a challenge for machine learning methods of was correctly classified as a 1. See also Binary
natural language processing. Stemming (also called Classification and Confusion Matrix.
lemmatization) reduces multiple variants of a word to
just one word for analysis purposes (e.g., “departure, Type I Error
departed, departing” all become “depart”). In statistical hypothesis testing, a type I error is
mistakenly accepting the alternative hypothesis
Stop Words that an effect or phenomenon (e.g., a new therapy
In natural language processing (NLP), stop words are improves health, or a stock price is not a random walk)
terms in a document that are dropped before analysis. is real. In other words, a type I error is being fooled
NLP programs typically offer a generic list of stop by chance patterns into thinking that something
words like “the,” “an,” and “to” that do little to help interesting is going on.

19
Data Science for Investment Professionals Glossary

Type II Error
In statistical hypothesis testing, a type II error is
mistakenly accepting the null hypothesis that an
effect or phenomenon (e.g., a new therapy improves
health, or a stock price is not a random walk) is not
real and is just the product of chance. A type II error
(which can occur only when the effect is real) usually
results from an inadequate sample size.

Unstructured Data
Unstructured data are data that do not occur naturally
in the form of a table with rows and columns. The text
in documents, images, and “messy” data (e.g., medical
notes that have a mixture of numeric and text data)
are all unstructured data.

20
Data Science for Investment Professionals Glossary

Course 5 – Mitigating Biases in the Data Science Binning

Pipeline A grouping of continuous observations into ranges.
Accountability
Black-Box Model
The principle suggesting that a set of rules and
Adjective to describe the opacity of a model
responsibilities must be established to identify who
framework.
is accountable in case the data science tool goes
wrong, who is able to explain the model assumptions
Boosting
and results, and who controls the decisions based on
The sequential learning of models, where each new
the quantitative models’ output.
model learns from the errors of the previous model.
Accuracy
Bootstrap Aggregating
The ratio of all correctly predicted classes to the total
A technique whereby the original training dataset is
number of predictions.
used to generate n new training datasets or bags of
data (also called bagging).
Acquiescence Bias
A bias that arises when participants respond to a
Bootstrapping
survey question in a manner that is not consistent
The repeated resampling of data with replacement
with their position.
from a dataset.
Adaptive Windowing (ADWIN)
Cause–Effect Bias
An algorithm that uses moving time-period windows,
Also known as correlation bias, occurs when the
comparing the means in the moving windows, and
correlation is mistaken for causation.
dropping older windows when their means are
significantly different compared with those in the
Characteristic Stability Index (CSI)
more recent windows.
A metric for comparison of the distributions of
the features in the training data with that of the
Alpha
deployment data.
Manager skill or the premium return over market
averages that cannot be explained by common factor
Citation Bias
betas, especially that of market beta.
A potential bias when there is a reliance on specific
research rather than the entirety of the extant
Back-Propagation
published research.
The process of tuning the weights in a neural network
process to reduce error.
Classification and Regression Tree (CART) Algorithm
A supervised machine learning algorithm based on
Baseline Model
decision trees.
The model produced using the training dataset.

Cohen-D
Bias
A statistic that is used to measure the effect of size in
A measurable, systematic difference between actual
power analysis.
results and the correct results in statistics, or a belief
or assumption that is implicit or explicit.
Coefficient of Determination
The percentage of the variation of the dependent
Bias Error
variable in a regression explained by the independent
The degree to which the model fits the training data,
or explanatory variables. Also known as R2.
where the lower the bias, the better the model’s
performance in the training dataset.
Complexity Bias
Refers to a behavioral bias that the complex can be
Behavioral Bias
more appealing than the simple. In reality, a simple
Typically, cognitive, or emotional bias in decision
model that is not underfitted is usually better than a
making that leads to errors in judgment. Personality
complex model.
difference can exert a relatively huge influence on its
overall impact.

21
Data Science for Investment Professionals Glossary

Composite Variable Data Leakage Bias

A variable that combines two or more variables that A situation in which the training dataset in model
are statistically strongly related. development uses data that are outside of the
training data to be used in model building.
Concept Drift
A change in the target or feature variable, which is Data Lifecycle
often abrupt but also may be incremental or recurring. People select, clean, analyze, and interpret data to
make it useful for various purposes. This is an iterative,
Conformation Bias ongoing process, not a simple or linear one. How
Also known as observer bias. Occurs during model people work with data affects what they can learn
interpretation, where the data scientist interprets or from those data. Also called data journey or data
looks for information consistent with their beliefs. biography.

Confounding Variables Data Provenance

Variables that are correlated with each other because The detailed record of the source of the data, the data
they share a common (possibly hidden) relationship. collection process, the data cleaning process, and the
data exploration process.
Confusion Matrix
A 2 x 2 matrix that is used to classify a model’s results Data Reduction Bias
into four cells (true positives, false positives, false The situation in which a smaller dataset is used in
negatives, and true negatives) and is used to measure the analysis, such that the smaller dataset is not
accuracy, prediction, and recall. representative of the population.

Conjectural Bias Dimension Reduction

A bias that may arise when we form a conclusion The process of eliminating variables that are not
or opinion based on preliminary analytics results or intended to be used in the data analysis and modeling.
based on incomplete or suboptimal data.
Data Snooping Bias
Correlation A bias that occurs when a model is developed
The degree of co-movement in an asset’s or not from an economic or financial rationale and
investment strategy’s return relative to the return of domain-specific knowledge, but rather by searching
the relevant market or benchmark. through data to find a statistically significant model.

Covariate Drift Deployment Data

A change in the relationship between feature and Data used in the implementation of the model.
target variables.
Effect Size
Cross-Validation A value measuring the strength of the relationship
A technique for estimating out-of-sample (OOS) error between two variables in a population or the
directly by determining the error in validation samples. difference between groups.

Data Availability Bias Efficient Frontier

The bias that arises when using readily available Output of modern portfolio theory that generates a set
data, which results from limited data sources or the of optimal portfolios with the highest return for a given
researcher using other researchers’ shared data. level of risk (standard deviation of return).

Data Drift Eigenvalue

A situation in which the data used in model The proportion of the total variance in a dataset that
deployment are different from that used in model is explained by a composite variable (the eigenvector)
building and training, sufficient to affect the model’s that results from principal components analysis.
performance.

22
Data Science for Investment Professionals Glossary

Eigenvector Funding Bias

A new, mutually uncorrelated composite variable that This bias occurs when the results of a model are
is a linear combination of original features in principal biased to support the financial sponsor of a project.
component analysis.
Generalization Bias
Endogeneity Bias Unintentional tendency to generalize model results to
A bias that occurs when an outcome of a model a larger population because of the cognitive bias of a
influences the effect of the model’s inputs. researcher.

Ensemble Bagging Gini Index

The use of a group of models that outperforms any A measure of the likelihood of misclassifying an
one individual model. observation if the observation is labeled randomly.

Ensemble Learning Grid Search

A technique that involves combining the predictions A method of systematically training a machine
from a collection of models. learning model by using various combinations
of hyperparameter values, cross validating each
Ensemble Method model, and determining which combination of
A combination of multiple learning algorithms. hyperparameter values ensures the best model
performance.
Entropy
The randomness of a variable. High Leverage Point
An extreme data point in an explanatory variable.
Environmental Social Governance (ESG)
A broad framework to score an investment based Hyperparameter
on its expected contribution to climate risk, societal A parameter whose value must be set by the
improvement, and prudent governance. Also called researcher before machine learning begins.
sustainability.
Influence Analysis
Exclusion Bias The process that identifies both outliers and high
The situation in which the data are not representative leverage points.
of the population because of the elimination of
subgroups in the population. Influential Data Bias
The situation in which an outlier or a high-leverage
Explainable Artificial Intelligence (XAI) data point that is not representative of the population
Refers to various approaches to explain complex affects the statistical results of a data analysis.
machine learning algorithms that cannot be directly
interpreted. Informative Censoring
This bias occurs when the flow of data abruptly drops
Fairness off, for a variety of reasons, during the extraction
The principle suggesting that results of data science process.
tools should give fair results, without any bias and
discrimination toward certain ethnic, gender, or minority Instrumental Variable Bias
groups of people. Also called objective accuracy. A bias that occurs when a variable unintentionally
proxies for another feature.
F1 Score
A measure of classification model fit that is the Interpretability
harmonic mean of the precision and recall metrics. A model is interpretable when it is fully transparent;
the user can understand why and how the model
Feature Shuffling makes its predictions.
A method of testing a model in which a feature’s
values are randomly reassigned among observations Interview Bias
and the resultant model is compared with the A bias in the data collection that may occur if
baseline model, that is, the model without shuffling. questions are confusing or misleading.

23
Data Science for Investment Professionals Glossary

Irrelevant Variable Market Beta

A variable that contributes nothing to the solution The measure of the magnitude of price sensitivity of
and interferes by adding random noise to the model a stock or investment strategy relative to the relevant
that can fool predictive algorithms or by making market or benchmark over time.
their searches less effective. Also called superfluous
variables. Measurement Bias
A bias that may occur in data collection when
K-Fold Cross-Validation collecting or calculating features resulting in
A method in which the data (excluding the test data) inaccurate or erroneous data.
are shuffled randomly and then divided into k equal
subsamples, with k – 1 samples used as training Model Monitoring
samples and one sample, the kth, used as a validation The use of techniques to determine whether there are
sample. issues that may arise, such as data drift, that need to
be identified and addressed.
K-Nearest Neighbor (k-NN)
A supervised machine learning method that works by Model Tuning
finding the k known training cases closest to a new The process of adjusting model hyperparameters to
case and then combining (e.g., by averaging) those find the point of minimum total error.
answers to estimate its value or class.
Modern Portfolio Theory (Mean-Variance
Kolmogorov–Smirnov (K-S) Test Optimization)
A nonparametric test statistic that can be used to A portfolio optimization framework that produces
compare the distributions of two datasets. a diversified portfolio with the highest expected or
mean return given the level of expected variance in
Label Bias return.
A data collection bias where the variable is labeled or
identified inconsistently. Nonrepresentative Bias
The situation in which the sample is not
Least Absolute Shrinkage and Selection Operator representative of the population.
(LASSO) Regression
A type of regression that uses regularization to Nonresponse Bias
penalize regression coefficients, resulting in a A bias from surveys that occurs when participants fail
parsimonious model. to answer some of the questions.

Local Interpretable Model-Agnostic Explanation Neural Networks (NNs)

(LIME) A flexible, nonlinear machine learning algorithm that
A second or surrogate prediction model designed learns from the data using three types of layers: input,
for supervised learning models. Provides local linear hidden, and output.
interpretability to complex models, such as deep
learning, by testing the variability of the prediction by Omitted Variable Bias
iterating small changes to individual input data. A bias that occurs when one or more relevant
variables are not included in the dataset and in model
Look-Ahead Bias development, resulting in a model that is not reliable.
A bias that results from using variables that contain
information that could be known only at a future date. Open-Source Risk
Relates to, among others, risks of intellectual property
Majority-Vote Classifier infringement claims, risks stemming from biases
A method of using the results of ensemble methods and errors in data obtained from public sources, and
that assigns the predicted label with the most votes security risks from vulnerabilities embedded in code
to a new data point. or software obtained from public sources.

24
Data Science for Investment Professionals Glossary

Out-of-Sample Testing Prediction Bias

Testing using data that was not used in the training of This situation implies that the prediction for the total
the model. Also called hold-out sample testing. dataset is inferior to the predictions for either of the
subgroups.
Outlier
An extreme data point for the explained or dependent Principal Components Analysis (PCA)
variable. A method used to transform highly correlated features
into a few main, uncorrelated composite variables
Overfitting referred to as eigenvectors.
The situation in which a model fits the training data
well but does not fit the test data well. Protected Classes, Protected Characteristics
The Equal Credit Opportunity Act of 1974 aimed at
P-Hacking ensuring access to credit on a nondiscriminatory
The use of multiple testing to search for a statistically basis by introducing protected classes of individuals,
significant results, resulting in false discoveries (i.e., such as age, race, gender, national origin, and other
false positives). parameters that could not be used in a credit decision.

P-Value Publication Bias

The smallest level of significance at which the null A bias that may occur when relying on published
hypothesis can be rejected. research that likely has a bias from the higher
likelihood that studies with significant results are
Page-Hinkley Method published compared with studies with insignificant
Identifies concept drift by comparing deployment data results.
means with the training data means and indicates
abrupt changes based on a user-defined tolerance. Random Forest
An ensemble of decision trees.
Penalized Regression
A computationally efficient technique useful for Recall
reducing a large number of features to a manageable The ratio of correctly predicted positive classes to all
set and for making good predictions, thus avoiding actual positive classes. Also called sensitivity or true
overfitting. positive rate.

Permutation Importance Recall Bias

A measure of variable importance that compares A bias that arises when survey participants are asked
the prediction accuracy of a variable with that of the for information about events or experiences in the
same variable that has had its observations shuffled. past, with the possibility that some participants have
better recall than others.
Population Stability Index (PSI)
A method of detecting data drift in the target variable Receiver Operating Characteristic (ROC) Curve
that compares the distributions of the target variable An approach for comparing models that involves
in two different datasets. a graphical plot of a curve showing the trade-off
between the false positive rate (on the x-axis) and the
Power Analysis true positive rate or recall (on the y-axis).
The analysis of statistical procedures to determine
whether they generate sufficient explanatory power Recursive Feature Elimination (RFE)
at a specific significance level. Power analysis also A process related to backward selection is
estimates the minimum sample size required given sequentially removing features based on a ranking of
the required significance level and effect size. explanatory power.

Precision Redundant Variables

The ratio of correctly predicted positive classes to all Highly correlated variables that do not contribute to
predicted positive classes. the solution.

25
Data Science for Investment Professionals Glossary

Regularization Stepwise Regression

A method that simplifies the model by penalizing A method in both linear and nonlinear regression to
high-valued regression coefficients with the goal of simplify the set of features by either adding features
reducing the risk of overfitting. sequentially (forward stepping) or deleting features
sequentially (backward selection or backward
Regulatory Risk stepping).
The risk of regulatory noncompliance that can result in
steep fines and/or loss of reputation. Survivorship Bias
The use of data in modeling that includes only
Relational Database surviving entities or securities.
A database that consists of rows and columns.
Target Shuffling
Robust Statistic Random shuffling of the target variable, so the
A statistic that performs well for a wide-range of connection between it and the model’s features
probability distributions. is broken. The model’s fit with the target shuffled
dataset is compared with its fit using the unshuffled
Root Mean Squared Error (RMSE) target dataset. Performance with the unshuffled
An error metric that is the square root of the mean dataset is expected to dominate, assuming the model
of the squared errors; the differences between the is uncovering a significant result and not a random
predicted and actual values of the target variable. result.

Sample Selection Bias Time-Lag Bias

The non representativeness of a sample due to The possible bias when relying on published research
improper criteria or processes in choosing the sample. for which the study was completed well before
publication.
Sampling Bias
A bias that may arise when the sample is not Time Period Bias
representative of the population. A bias that may arise when selecting time-series data
when the data distribution changes over time.
Shapley Additive Explanations (SHAP)
A second or surrogate model used to create an Transparency
interpretability framework for complex models. The principal suggesting that data analysts and
It does not assume that the local model is linear. decision makers have to understand and be able to
Based on cooperative game theory, the model explain the model assumptions and data used for the
determines the marginal contribution of a given algorithmic learning and also how the algorithm has
feature to the prediction. The sum of individual reached the final results.
Shapley variables is equal to the difference between
the average prediction and the total prediction. Trimming
Method of dealing with extreme observations by
Simpson’s Paradox deleting extreme observations in the database, often
The situation in which the relationship between two based on observations outside a percentile range
variables that is observed for subgroups sampled (e.g., deleting the lowest and highest 1%).
from the population is different than that of the entire
sample. Type I Error
Rejection of a true null hypothesis. Also called a false
Social Desirability Bias positive.
A bias that arises when participants answer a
question in a way to conform with acceptable norms Type II Error
or to appear more socially attractive. Nonrejection of a false null hypothesis. Also called a
false negative.
Statistical Power
The likelihood that a model can detect an effect when
there is one.

26
Data Science for Investment Professionals Glossary

Underfitting Variable Importance Analysis (VIA)

The situation in which the model does not fit the Methods that identify and rank the importance of
training data well because the model has not learned features in a model.
the patterns in the training data and, therefore, the
model does not perform well using the test data. Winsorizing
Method of dealing with extreme data that involves
Validation Sample substituting values for extreme observations,
Subsample of the training data that is used to validate generally based on converting observations below
or test the model. a specified percentile to the value at that percentile
and repeating this for those above another specified
Variance Error percentile to the value at that percentile.
A measure of the degree to which the model results
change from the training dataset to the validation and
testing datasets.

Variance Inflation Factor (VIF)

A metric used to identify and quantify multicollinearity
among features in a dataset.

27