QUANTITATIVE RESEARCH:

MEASUREMENT
&
DATA COLLECTION
 LEARNING OBJECTIVES
■ Identify types of data that researchers collect ;

■ The different characteristics of measures — nominal,

ordinal, interval, ratio

■ Understand what constitutes an effective and high - quality

measure.

■ Identify the various strategies for collecting data, and be

able to match the collection strategy to a particular
research question.
MEASUREMENT IN RESEARCH

• The first step in data collection is to

figure out how to measure what you are
interested in studying.

• A variable is a construct that can take on

two or more distinct values. A researcher
needs good measures for both
independent and dependent variables.

• A collection of measured variables from a sample make up what

researchers call data (singular-datum)
 Measurement of Variables : Three-step Process
To conduct good research, it is vital to construct meaningful
variables & to measure the variables properly

1. A researcher must turn abstract concepts into measurable

events (operational definition)

2. A researcher must determine that a measure is of high quality

3. A researcher must gather the information from the potential

participants in the study
MEASUREMENT IN RESEARCH

• An operational definition is simply

the definition of a variable in terms
of the actual procedures used to
measure or manipulate it (Graziano
& Raulin, 2004).
• Measurement can be defined as a
process through which researchers
describe, explain, and predict the
phenomena and constructs of our
daily existence ……..(Kaplan, 1964;
Pedhazur & Schmelkin, 1991).
 Non-Metric and Metric Data
• There are two basic categories of data

• Nonmetric data (qualitative data) are typically attributes,

characteristics, or categories that describe an individual and cannot
be quantified

• Metric data (quantitative data) allow researchers to examine

amounts and magnitudes
There are four main scales of measurement subsumed under the broader
categories of nonmetric and metric measurement:
 Scales of Measurements…. Nominal scales
Nominal scales are the least sophisticated type of measurement
and are used only to qualitatively classify or categorize.
 Scales of Measurements…..Ordinal Scales
• Ordinal scale measurement is characterized by the ability to
measure a variable in terms of both identity and magnitude.
• This makes it a higher level of measurement
Scales of Measurements……The Interval Scale
Scales of Measurements
…The Ratio Scale
Are the following nominal, ordinal, interval or ratio data?

1. Temperatures measured on the Kelvin scale.

2. Military ranks.
3. NIC numbers.
4. Number of passengers on buses from Lahore to Pindi .
5. Code numbers given to the religion of persons attempting
suicide.
 RE-CAP

• One value is different from another, you have a nominal scale;

• One value is bigger,MEASUREMENT
better or more of anything
SCALES than another, you have
an ordinal scale;
 Nominal : gender, religion etc
• One value is
 Ordinal so many
: lower, units or
middle, (degrees, inches)
upper class etc.more or less than
another, you
 Interval have an interval
: temperature scales;scale;
attitudes(arbitrary zero)etc.
• One valueage,
 Ratio: is soheight,
many distance,
times as big
andor bright Etc.
income. or tall or heavy as
another, you have a ratio scale.
Aptitude/Intelligence Tests: Measure capability and
intelligence

Skills Tests: Measure a job relevant skill and your proficiency

in that skill. For example, an accounting position would most
likely require a skills test on Quickb ooks.

Personality Tests: Measure personality traits, values and fit in

a position or company culture

Integrity Tests: Measure honesty, integrity and other

psychological traits
 Sources of Error in Measurement

• Respondent Researcher must know that correct

measurement depends on
• Situation successfully meeting all of the
• Measurer problems listed here. He must, to
the extent possible, try to eliminate,
• Instrument
neutralize or otherwise deal
with all the possible sources of
error so that the final results may
not be contaminated.
"Any research can be affected by
different kinds of factors which,
while extraneous to the concerns of
the research, can invalidate the
findings"
(Seliger & Shohamy 1989, 95).
It is not enough simply to
create a numeric
measure. They must also create a
The measure must be measure that is consistent
truthful — it must — it must yield the same
accurately reflect the results across time,
construct. If researchers circumstances, and groups
create a measure of of people.
intelligence, they
must be sure that this
measure actually reflects
intelligence.
Validity is about truthfulness.
A measure shows validity if it actually
measures what it claims (or is intended) to
measure.
 Types of Validity

1. Face Validity
2. Criterion Related Validity
3. Construct-related Validity
4. Content-related Validity
 Types of Validity-Face Validity
• Face validity is concerned with how a measure or procedure appears. Does it
seem like a reasonable way to gain the information the researchers are
attempting to obtain? Does it seem well designed? Does it seem as though it
will work reliably?

• Face validity, as the name suggests, is a measure of how representative a

research project is ‘at face value,’ and whether it appears to be a good
project. It requires personal judgment.

• An example could be, after a group of students sat a test, you asked for
feedback, specifically if they thought that the test was a good one. This
enables refinements for the next research project and adds another
dimension to establishing validity.
 Types of Validity-Content Validity

• Content Validity sometimes called logical or

rational validity, refers to the extent to which
the items or behaviors fully represent the
concept being measured

• Employing a science teacher…

• How can you establish content validity?

Types of validity-Criterion Related Validity

• Criterion related validity, also referred to as instrumental

validity, is used to demonstrate the accuracy of a measure or
procedure by comparing it with another measure or procedure
which has been demonstrated to be valid.

• To measure the criterion validity of a test, researchers must

calibrate it against a known standard or against itself.

• One of the simplest ways to assess criterion related validity is

to compare it to a known standard.
• COCA-COLA – the cost of neglecting criterion validity
Types of validity- Construct Validity

• Construct validity defines how well a test or experiment measures up

to its claims. It refers to whether the operational definition of a
variable actually reflect the true theoretical meaning of a concept.

• For example, you might design whether an educational program

increases artistic ability amongst pre-school children. Construct
validity is a measure of whether your research actually measures
artistic ability, a slightly abstract label.
It is not enough simply to
create a numeric
measure. They must also create a
The measure must be measure that is consistent
truthful — it must — it must yield the same
accurately reflect the results across time,
construct. If researchers circumstances, and groups
create a measure of of people.
intelligence, they
must be sure that this
measure actually reflects
intelligence.
 Reliability
• Reliability has to do with the quality of measurement. In its everyday sense,
reliability is the "consistency" ; “stability” or "repeatability" of your
measures.
• A measure is considered reliable if it would give us the same result over and
over again (assuming that what we are measuring isn't changing!).
• Reliability is the extent to which an experiment, test, or any measuring
procedure yields the same result on repeated trials.

• Reliability is usually determined using a correlation coefficient (it is called a

reliability coefficient in this context).
• Reliability coefficients of .70 or higher are generally considered to be
acceptable for research purposes. Reliability coefficients of .90 or higher
are needed to make decisions that have impacts on people's lives (e.g., the
clinical uses of tests).
 Reliability Estimators
• There are four good methods of measuring reliability:

1. test-retest
2. multiple forms
3. inter-rater
4. split-half
• The test-retest technique is to
administer your test, instrument,
survey, or measure to the same group
of people at different points in time

• E.g. An educational test retaken after a

month should yield the same results as
the original. (caution )

An example would be the method of maintaining weights used by the U.S. Bureau
of Standards. Platinum objects of fixed weight (one kilogram, one pound, etc...) are
kept locked away. Once a year they are taken out and weighed, allowing scales to
be reset so they are "weighing" accurately. Keeping track of how much the scales
are off from year to year establishes a stability reliability for these instruments. In
this instance, the platinum weights themselves are assumed to have a perfectly
fixed stability reliability
• Parallel-Forms or multi-forms technique
• …create a large set of questions that address the same construct
and then randomly divide the questions into two sets. You
administer both instruments to the same sample of people
• The success of this method hinges on the equivalence of the two
forms of the test.
• If, for example, rater A •Inter-Rater Reliability.
observed a child act out • Inter-Scorer Reliability refers to the consistency
aggressively eight times, we or degree of agreement between two or more
would want rater B to observe
scorers, judges, or raters.
the same amount of aggressive
• You could have two judges rate one set of
acts. If rater B witnessed 16
aggressive acts, then we know papers. Then you would just correlate their two
at least one of these two raters sets of ratings to obtain the inter-scorer
is incorrect. reliability coefficient, showing the consistency
• If there ratings are positively of the two judges’ ratings.
correlated, however, we can be
reasonably sure that they are
measuring the same construct
of aggression. It does not,
however, assure that they are
measuring it correctly, only
that they are both measuring it
the same.
• Split-half reliability
• A measure of consistency where a test is split in two and the
scores for each half of the test is compared with one another. If the
test is consistent it leads the experimenter to believe that it is most
likely measuring the same thing.
 Reliability and Validity

Reliability: Consistency of the results of the test

Validity: Does the test measure what it is actually supposed to

measure?
• A researcher devises a new test that measures IQ more quickly than
the standard IQ test:
(e.g. a candidate’s real IQ is 120)
• If the new test delivers scores for a candidate of 87, 65, 143 and
102, then the test is
not reliable or valid, and it is fatally flawed.

• If the test consistently delivers a score of 100 when checked, then

the
test is reliable, but not valid..
• If the researcher’s test delivers a consistent score of 118, then,.

that is pretty close and the test can be considered both valid and reliable.