Assume you have scores on a number of variables (most often 10-100 variables). The purpose of factor analysis is to discover simple patterns in the pattern of relationships among the variables. In particular, it seeks to discover if the observed variables can be explained largely or entirely in terms of a much smaller number of variables called factors.
Factor Analysis or Principal Components Analysis
If your purpose is to reduce the information in many variables into a set of weighted linear combinations of those variables, use Principal Components Analysis (PCA), which does not differentiate between common and unique variance.
If your purpose is to identify the latent variables which are contributing to the common variance in a set of measured variables, use Factor Analysis (FA), which will attempt to exclude unique variance from the analysis.
Exploratory or Confirmatory Factor Analysis
If you wish to restrict the number of factors extracted to a particular number and specify particular patterns of relationship between measured variables and common factors, and this is done a priori (before seeing the data), then the confirmatory procedure is for you. If you have no such well specified a priori restrictions, then use the exploratory procedure.
Exploratory factor analysis (EFA) is often recommended when researchers have no hypotheses about the nature of the underlying factor structure of their measure. Exploratory factor analysis has three basic decision points:
(1) Decide the number of factors,
(2) Choosing an extraction method,
(3) Choosing a rotation method.
Decide the Number of factors
If more factors extracted, it is likely to lead to a solution where the major factors are well estimated by the obtained loadings but where there are also additional poorly defined factors.
If too few factors extracted, it is likely to lead to factors that are poorly estimated (poor correspondence between the structure of the true factors and that of the estimated factors).
One can use scree test, then select the factors based on Kaiser’s run (eigenvalue >1) or parallel analysis, or else.
Choosing Factor Extraction Method
Maximum Likelihood (ML) extraction allows computation of assorted indices of goodness-of-fit (of data to the model) and the testing of the significance of loadings and correlations between factors, but requires the assumption of multivariate normality.
We can first examine the distributions of the measured variables for normality. Unless there are severe problems (|skew| > 2, kurtosis > 7), we can use Maximum Likelihood extraction. If there are severe problems, we can also try to correct the problems by transforming variables.
Principal Factors (PF) methods have no distributional assumptions, but it is not preferred.
The extraction method will produce factor loadings for every item on every extracted factor. Researchers hope their results will show what is called simple structure, with most items having a large loading on one factor but small loadings on other factors.
Choosing a Rotation Method
Once an initial solution is obtained, the loadings are rotated. Rotation is a way of maximizing high loadings and minimizing low loadings so that the simplest possible structure is achieved. There are two basic types of rotation: orthogonal and oblique.
· Orthogonal means the factors are assumed to be uncorrelated with one another.
Common algoriths for orthogonal rotation: varimax, quartamax, equamax.
· Oblique rotation derives factor loadings based on the assumption that the factors are correlated, and this is probably most likely the case for most measures. So, oblique rotation gives the correlation between the factors in addition to the loadings.
Common algorithms for oblique rotation: oblimin, promax, direct quartimin.
Empower gives “varimax” and “promax” for you to choose.
To produce scores for each observation, you can choose "regression” or “Bartlett” method. “Regression” gives Thompson’s scores, “Bartlett” gives Bartlett’s weighted least-squares scores.
Below is the sample input window
Below is the sample output of the above model:
Maximum likelihood factor analysis
factanal(x = tmp.xx, factors = 3, scores = "regression", rotation = "promax")
RATING COMPLAINTS PRIVILEGES LEARNING RAISES CRITICAL ADVANCE
0.227 0.080 0.639 0.005 0.239 0.771 0.299
Factor1 Factor2 Factor3
RATING 0.872 -0.140 0.122
PRIVILEGES 0.408 0.185 0.117
LEARNING 0.148 0.132 0.839
RAISES 0.402 0.570
CRITICAL 0.499 -0.231
ADVANCE -0.304 0.829 0.265
Factor1 Factor2 Factor3
SS loadings 2.206 1.333 0.863
Proportion Var 0.315 0.190 0.123
Cumulative Var 0.315 0.505 0.629
Test of the hypothesis that 3 factors are sufficient.
The chi square statistic is 2.06 on 3 degrees of freedom.
The p-value is 0.56
Empower also output following loading plot.