CHAPTER 2: SCIENTIFIC METHODS

In this chapter, we provide a foundation for the science behind the research reported in this
textbook and in the area of I/O psychology. It is important for students to have a solid
understanding of this chapter to help them appreciate material from all of the other chapters.

Learning Outcomes
After reading this chapter you should be able to
 Appreciate the difference between information discovered through scientific and non-
scientific ways of knowing
 Understand the scientific method and several different types of research strategies
 Understand the important role that measurement plays in the scientific process and in
describing differences between individuals
 Know what a correlation coefficient is, along with a few other basic statistical concepts
used in personnel selection
 Recognize the importance and necessity of establishing the reliability and validity of
measures used in organizational research
 Appreciate that professionals must adhere to a code of ethics in all of their work

Chapter Summary
There are several ways in which we can acquire information about people, places, things, or
events. These different methods fall into four broad categories. Although the three non-scientific
methods of tenacity, authority, and rationalization provide us with information, the scientific
method produces the most accurate information. The difference between scientific and non-
scientific methods is that science is self-correcting.
Information that is acquired through a scientific process can be “falsified”; that is, the
information, if incorrect, can be disproved through objective, empirical means. This means
stating the relationship in such a way that it can be tested and disproved if that relationship does
not exist.
Science produces information that is based on accepting as true only that objective
information that can withstand continued attempts to cast doubt on its accuracy. The accuracy of
scientific statements is examined empirically through methods that can be observed, critiqued,
and used by others. Scientific procedures allow for the measurement of important human
characteristics that are related to industrial and organizational psychology. The most critical step
is to gather empirical evidence that is relevant to the hypothetical relationship. The scientific
method includes four main steps: (1) Statement of the problem, (2) Stating the hypotheses, (3)
Reasoning and deducing, and (4) Observational test or experiment via a research plan. A
research plan, or research design, lays out the framework for making measurements or
observations on the variables involved. Research designs can include observational/correlational,
quasi-experimental, and experimental designs.
For industrial/organizational psychology, the purpose of measurement is to describe
differences among individuals, teams, groups, or organizations with respect to those constructs
that are important to the task at hand. Several methods can be used to assess differences in work
settings. The most prominent include self-reports, behavioural observations, reports about others,
and unobtrusive measures.
The reliability and validity of the information used as part of I/O psychology procedures

Copyright © 2011 Nelson Education Ltd. 8

must be established empirically. The measures used by I/O psychologists to make inferences
about people and organizations must be able to withstand attempts to cast doubt on their
correctness.
In all of their work, I/O psychologists are obligated to follow the codes of ethics endorsed
by their profession.

Discussion Topics & Learning Activities

1. Go through a recent newspaper or online news service and find examples of research in
organizations. What question are the researchers trying to answer? What are the predictors
and criteria?

2. Conduct a random search using a literature search engine (e.g., Google Scholar, PsycINFO,
PsycARTICLES, EBSCO, etc.) to look up I/O psychology topics that interest you. Choose
one or two of these articles, and look at the methods and results section. What type of study
design do the researchers use? What are the variables they examine? How do they
operationally define the constructs?

9 Copyright © 2011 Nelson Education Ltd.

Chapter 2—Scientific Methods

MULTIPLE CHOICE
1. Which of the following is a strength of quasi-experimental research designs?
a. They allow researchers to make cause and effect conclusions.
b. They have the advantage of being conducted in a lab setting.
c. They involve studying naturally-occurring work groups in applied work settings.

d. Researchers can randomly assign the participants to different conditions or groups.

ANS: C PTS: 1 BLM: Higher Order REF: 31

2. You have conducted a field study with 300 employees from a variety of organizations. You are
examining the relationship between employee personality and job performance. You have found
that performance is negatively correlated with one personality component (i.e., neuroticism; r =
-.68, p<.001). Which of the following are you MOST likely to conclude?
a. Highly neurotic employees are successful.
b. Low neuroticism scores cause higher job performance.
c. Employees who score low on neuroticism tend to have higher levels of
performance.
d. The relationship between neuroticism and performance is determined by a third
variable (such as supervisor support).
ANS: C PTS: 1 BLM: Higher Order REF: 36

3. As the district manager of McBurger King Restaurants across Canada, you believe that your
employees’ performance is based upon whether or not they have had any previous work
experience, but only when personality is taken into account. In this example, what is
performance?
a. a criterion
b. a mediator
c. a moderator
d. a predictor
ANS: A PTS: 1 BLM: Higher Order REF: 38

Copyright © 2011 Nelson Education Ltd. 10

4. As the district manager of McBurger King Restaurants across Canada, you believe that your
employees’ performance is based upon whether or not they have had any previous work
experience, but only when personality is taken into account. In this example, what is work
experience?
a. a criterion
b. a mediator
c. a moderator
d. a predictor
ANS: D PTS: 1 BLM: Higher Order REF: 38

5. What is c in the equation Y = bX + c ?

a. predictor
b. slope
c. intercept
d. criterion
ANS: C PTS: 1 BLM: Remember REF: 38

6. What is b in the equation Y = bX + c ?

a. constant
b. slope
c. intercept
d. baseline
ANS: B PTS: 1 BLM: Remember REF: 38

11 Copyright © 2011 Nelson Education Ltd.

7. Future Inc., a high tech firm, wants to implement a new computerized selection program for its
assembly personnel. First, however, it wants to ensure that the new program is at least as
effective as the existing paper and pencil test.
Last year, all entry-level employees were administered both the paper and pencil and
computerized tests, but were hired based only on paper and pencil results. This year,
performance data for these employees were compared with the computerized test results. What
can this comparison best be described as?
a. an indicator of predictive validity
b. an indicator of equivalent forms reliability
c. an indicator of concurrent validity
d. an indicator of test-retest reliability

ANS: A PTS: 1 BLM: Higher Order REF: 46

8. Future Inc., a high tech firm, wants to implement a new computerized selection program for its
assembly personnel. First, however, it wanted to ensure that the new program is at least as
effective as the existing paper and pencil test.
Last year all entry-level employees were administered both the paper and pencil and
computerized tests and performance data for these employees were collected. Their performance
was then compared with the computerized test results. What can this comparison best be
described as?
a. an indicator of predictive validity evidence
b. an indicator of concurrent evidence
c. an indicator of equivalent forms reliability
d. an indicator of test-retest reliability
ANS: B PTS: 1 BLM: Higher Order REF: 46
9. Which one of the following does the reliability of a test involve?
a. the ability to make inferences about constructs from test scores
b. the extent to which scores are free from measurement errors
c. a set of systematically interrelated concepts
d. the extent to which your inferences about a test are legitimate.
ANS: B PTS: 1 BLM: Remember REF: 39

Copyright © 2011 Nelson Education Ltd. 12

10. Ideally, when determining the criterion-related validity of a test, we DO NOT use data from that
test to hire the employees because of which problem?
a. multiple regression lines
b. differential validity
c. restriction of range
d. validity generalization
ANS: C PTS: 1 BLM: Higher Order REF: 48

11. The extent of linear relationship between any two variables can be described numerically using
which of the following?
a. an F-test
b. a scatter plot
c. a meta-analysis
d. a correlation coefficient
ANS: D PTS: 1 BLM: Remember REF: 37

12. What does strict adherence to ethical principles by I/O psychologists ensure?
a. I/O psychologists are aware of changes in their environment
b. information about individuals or clients is shared with others as required
c. data is used only for the purpose(s) for which they were intended
d. I/O psychologists may engage in research
ANS: C PTS: 1 BLM: Higher Order REF: 51

13. Which of the following is a principle of fairness as it relates to assessment and selection?
a. Fairness is clearly defined in terms of psychometric properties.
b. Fairness refers to the content of a test.
c. Fairness is a statistical concept that can be defined quantitatively.
d. Fairness refers to value judgments about decisions made based on assessments.
ANS: D PTS: 1 BLM: Higher Order REF: 50–51

13 Copyright © 2011 Nelson Education Ltd.

14. You want to conduct a study to examine whether an existing measure, the Job Knowledge Scale
(JKS), is valid, such that it predicts job performance. However, you are unable to collect enough
data to make any meaningful conclusions. Therefore, you decide to get evidence through a meta-
analysis of other studies that have examined its validity. Which of the following will you
MOST LIKELY conduct?

a. fairness analysis
b. validity generalization study
c. qualitative analysis
d. psychometric assessment
ANS: B PTS: 1 BLM: Higher Order REF: 47

15. Which of the following is a characteristic of bias?

a. Bias refers to systematic errors in measurement
b. Bias involves value judgments about the decisions pertaining to a test
c. Bias primarily pertains to qualitative assessments
d. Bias refers to a lack of relationship to group membership
ANS: A PTS: 1 BLM: Higher Order REF: 49

16. What is a hypothesis?

a. a prediction about the relationship between two or more events, objects, people, or
phenomena
b. a value judgment about the decisions pertaining to a test or analysis in a study
c. a causal relationship among two or more variables or events
d. a qualitative assessment about an observation pertaining to behaviours or
phenomena
ANS: A PTS: 1 BLM: Remember REF: 49

Copyright © 2011 Nelson Education Ltd. 14

17. According to Alfred Whitehead (1967), how can we characterize “science”?
a. science is atheoretical
b. science does not accept prediction and causality as legitimate
c. science uses specific rules and procedures in the search for new knowledge
d. science is primarily concerned with observers’ perception
ANS: C PTS: 1 BLM: Higher Order REF: 49

TRUE/FALSE

1. A hypothesis is a tentative prediction about the relationship between two or more variables.
ANS: T PTS: 1 REF: 27

2. Correlation coefficients range from 0.00 to 1.00.

ANS: F PTS: 1 REF: 37

3. The degree of linear relationship between two variables is known as a criterion variable.
ANS: F PTS: 1 REF: 37

4. True score is the degree to which the content (or items) of a test are representative of the domain
in question.
ANS: F PTS: 1 REF: 44

5. For the equation: Y = bX + a, b = slope.

ANS: T PTS: 1 REF: 38

6. For the equation: Y = bX + a, b = constant.

ANS: F PTS: 1 REF: 38

7. Quasi-experimental designs allow researchers to manipulate and control variables in an applied

research setting.
ANS: T PTS: 1 REF: 31

15 Copyright © 2011 Nelson Education Ltd.

 8. Experimental designs allow researchers to manipulate and control variables in an applied work
setting, such that there is no random assignment.
ANS: F PTS: 1 REF: 31

SHORT ANSWER
1. Compare and contrast bias and fairness.

REF: 49-50

2. What is restriction of range? Why is it important to I/O Psychology?

REF: 48
3. Explain Alfred Whitehead’s (1967) five characteristics of the essential features of “science.” How
do these characteristics affect the work done by organizational researchers?

REF: 25

Copyright © 2011 Nelson Education Ltd. 16