Session 7

Factor Analysis

Chapter Outline
1) Overview
2) Basic Concept
3) Factor Analysis Model
4) Statistics Associated with Factor
Analysis

Factor Analysis
Factor analysis is a general name denoting a class of
procedures primarily used for data reduction and
summarization.
Factor analysis is an interdependence technique in
that an entire set of interdependent relationships is
examined without making the distinction between
dependent and independent variables.
Factor analysis is used in the following circumstances:
To identify underlying dimensions, or factors, that
explain the correlations among a set of variables.
To identify a new, smaller, set of uncorrelated
variables to replace the original set of correlated
variables in subsequent multivariate analysis
(regression or discriminant analysis).

Factor Analysis Model

The unique factors are uncorrelated with each
other and with the common factors. The common
factors themselves can be expressed as linear
combinations of the observed variables.
Fi = Wi1X1 + Wi2X2 + Wi3X3 + . . . + WikXk

Where:

Fi

estimate of i th factor

Wi

weight or factor score coefficient

number of variables

Conducting Factor Analysis:

Formulate the Problem
The objectives of factor analysis should be identified.
The variables to be included in the factor analysis
should be specified based on past research, theory,
and judgment of the researcher. It is important that
the variables be appropriately measured on an
interval or ratio scale.
An appropriate sample size should be used. As a
rough guideline, there should be at least four or five
times as many observations (sample size) as there
are variables.

Correlation
Matrix

Analysis:
Construct the Correlation
Matrix
The analytical process is based on a matrix of correlations
between the variables.

Bartlett's test of sphericity can be used to test the null

hypothesis that the variables are uncorrelated in the
population: in other words, the population correlation
matrix is an identity matrix. If this hypothesis cannot be
rejected, then the appropriateness of factor analysis
should be questioned.
Another useful statistic is the Kaiser-Meyer-Olkin
(KMO) measure of sampling adequacy. Small values of
the KMO statistic indicate that the correlations between
pairs of variables cannot be explained by other variables
and that factor analysis may not be appropriate.

Conducting Factor Analysis:

Determine the Method of Factor
Analysis

In principal components analysis, the total variance in the

data is considered. The diagonal of the correlation matrix
consists of unities, and full variance is brought into the factor
matrix. Principal components analysis is recommended when
the primary concern is to determine the minimum number of
factors that will account for maximum variance in the data for
use in subsequent multivariate analysis. The factors are called
principal components.

In common factor analysis, the factors are estimated based

only on the common variance. Communalities are inserted in
the diagonal of the correlation matrix. This method is
appropriate when the primary concern is to identify the
underlying dimensions and the common variance is of interest.
This method is also known as principal axis factoring.

Results of Principal Components Analysis

Results of Principal Components Analysis

Results of Principal Components Analysis

Results of Principal Components Analysis

The lower-left triangle contains the reproduced

correlation matrix; the diagonal, the communalities;
the upper-right triangle, the residuals between the
observed correlations and the reproduced
correlations.

Conducting Factor Analysis:

Determine the Number of
Factors

A Priori Determination. Sometimes, because of prior

knowledge, the researcher knows how many factors to
expect and thus can specify the number of factors to be
extracted beforehand.

Determination Based on Eigenvalues. In this approach,

only factors with Eigenvalues greater than 1.0 are retained.
An Eigenvalue represents the amount of variance associated
with the factor. Hence, only factors with a variance greater
than 1.0 are included. Factors with variance less than 1.0 are
no better than a single variable, since, due to
standardization, each variable has a variance of 1.0. If the
number of variables is less than 20, this approach will result
in a conservative number of factors.

Conducting Factor Analysis:

Determine the Number of
Factors

Determination Based on Scree Plot. A scree plot is a

plot of the Eigenvalues against the number of factors in
order of extraction. Experimental evidence indicates that
the point at which the scree begins denotes the true number
of factors. Generally, the number of factors determined by a
scree plot will be one or a few more than that determined by
the Eigenvalue criterion.

Determination Based on Percentage of Variance. In

this approach the number of factors extracted is determined
so that the cumulative percentage of variance extracted by
the factors reaches a satisfactory level. It is recommended
that the factors extracted should account for at least 60% of
the variance.

Scree Plot
3.0

Eigenvalue

2.5
2.0
1.5
1.0
0.5
0.0
1

4
5
6
3Component Number

Conducting Factor Analysis:

Rotate Factors

Although the initial or unrotated factor matrix

indicates the relationship between the factors and
individual variables, it seldom results in factors
that can be interpreted, because the factors are
correlated with many variables. Therefore,
through rotation, the factor matrix is transformed
into a simpler one that is easier to interpret.
In rotating the factors, we would like each factor
to have nonzero, or significant, loadings or
coefficients for only some of the variables.
Likewise, we would like each variable to have
nonzero or significant loadings with only a few
factors, if possible with only one.
The rotation is called orthogonal rotation if the
axes are maintained at right angles.

Conducting Factor Analysis:

Rotate Factors

The most commonly used method for rotation is the

varimax procedure. This is an orthogonal method
of rotation that minimizes the number of variables
with high loadings on a factor, thereby enhancing
the interpretability of the factors. Orthogonal
rotation results in factors that are uncorrelated.
The rotation is called oblique rotation when the
axes are not maintained at right angles, and the
factors are correlated. Sometimes, allowing for
correlations among factors can simplify the factor
pattern matrix. Oblique rotation should be used
when factors in the population are likely to be
strongly correlated.

Factor Matrix Before and After Rotation

Factors
Variables
1
2
3
4
5
6

1
X
X
X
X
X

(a)
High Loadings
Before Rotation

2
X
X
X
X

Factors
Variables
1
2
3
4
5
6

1
X

2
X

X
X
X

(b)
High Loadings
After Rotation

Conducting Factor Analysis:

Interpret Factors
A factor can then be interpreted in terms of
the variables that load high on it.
Another useful aid in interpretation is to plot
the variables, using the factor loadings as
coordinates. Variables at the end of an axis
are those that have high loadings on only
that factor, and hence describe the factor.

Factor Loading
Plot
Rotated Component Matrix
Component 2
Component
Variable
1
2

Component Plot
in
Rotated
Space
Component
1
V4
V6
1.0
V2

V1
-2.66E-02

0.5
0.0

V3
V1

V5

V2
0.848

0.962
-5.72E-02

-0.5

V3
-0.146

-9.83E-02

-1.0

V4
0.854
V5
-8.40E-02

-0.933

1.0

0.5

0.0

-0.5

-1.0

0.934

Analysis:
Select Surrogate
Variables
By examining
the factor matrix, one could

select for each factor the variable with the

highest loading on that factor. That
variable could then be used as a surrogate
variable for the associated factor.

However, the choice is not as easy if two or

more variables have similarly high loadings.
In such a case, the choice between these
variables should be based on theoretical
and measurement considerations.

Statistics Associated with Factor Analysis

Bartlett's test of sphericity. Bartlett's test of sphericity
is a test statistic used to examine the hypothesis that the
variables are uncorrelated in the population. In other
words, the population correlation matrix is an identity
matrix; each variable correlates perfectly with itself (r = 1)
but has no correlation with the other variables (r = 0).
Correlation matrix. A correlation matrix is a lower
triangle matrix showing the simple correlations, r,
between all possible pairs of variables included in the
analysis. The diagonal elements, which are all 1, are
usually omitted.

Statistics Associated with Factor Analysis

Communality. Communality is the amount of
variance a variable shares with all the other
variables being considered. This is also the
proportion of variance explained by the common
factors.
Eigenvalue. The eigenvalue represents the total
variance explained by each factor.
Factor loadings. Factor loadings are simple
correlations between the variables and the factors.
Factor or Component matrix. A factor matrix
contains the factor loadings of all the variables on
all the factors extracted.

Statistics Associated with Factor Analysis

Factor scores. Factor scores are composite scores
estimated for each respondent on the derived
factors.
Kaiser-Meyer-Olkin (KMO) measure of
sampling adequacy. The Kaiser-Meyer-Olkin (KMO)
measure of sampling adequacy is an index used to
examine the appropriateness of factor analysis.
High values (between 0.5 and 1.0) indicate factor
analysis is appropriate. Values below 0.5 imply that
factor analysis may not be appropriate.
Percentage of variance. The percentage of the
total variance attributed to each factor.
Scree plot. A scree plot is a plot of the Eigenvalues
against the number of factors in order of extraction.