fa {psych}  R Documentation 
Among the many ways to do latent variable exploratory factor analysis (EFA), one of the better is to use Ordinary Least Squares to find the minimum residual (minres) solution. This produces solutions very similar to maximum likelihood even for badly behaved matrices. A variation on minres is to do weighted least squares. Perhaps the most conventional technique is principal axes. An eigen value decomposition of a correlation matrix is done and then the communalities for each variable are estimated by the first n factors. These communalities are entered onto the diagonal and the procedure is repeated until the sum(diag(r)) does not vary. Yet another estimate procedure is maximum likelihood. For well behaved matrices, maximum likelihood factor analysis (either in the fa or in the factanal fuction) is probably preferred. Bootstrapped confidence intervals the loadings and interfactor correlations are found by fa in n.iter > 1 and for dichotomous or polytomous items, by fa.poly.
fa(r,nfactors=1,n.obs = NA,n.iter=1, rotate="oblimin", scores="regression", residuals=FALSE, SMC=TRUE, covar=FALSE,missing=FALSE,impute="median", min.err = 0.001, max.iter = 50,symmetric=TRUE, warnings=TRUE, fm="minres", alpha=.1,p=.05,oblique.scores=FALSE,np.obs,...) fac(r,nfactors=1,n.obs = NA, rotate="oblimin", scores="tenBerge", residuals=FALSE, SMC=TRUE, covar=FALSE,missing=FALSE,impute="median",min.err = 0.001, max.iter=50,symmetric=TRUE,warnings=TRUE,fm="minres",alpha=.1, oblique.scores=FALSE,np.obs,...) fa.poly(x,nfactors=1,n.obs = NA, n.iter=1, rotate="oblimin", SMC=TRUE, missing=FALSE, impute="median", min.err = .001, max.iter=50, symmetric=TRUE,warnings=TRUE,fm="minres",alpha=.1, p =.05, oblique.scores=TRUE,...) factor.minres(r, nfactors=1, residuals = FALSE, rotate = "varimax",n.obs = NA, scores = FALSE,SMC=TRUE, missing=FALSE,impute="median",min.err = 0.001, digits = 2, max.iter = 50,symmetric=TRUE,warnings=TRUE,fm="minres") #deprecated factor.wls(r,nfactors=1,residuals=FALSE,rotate="varimax",n.obs = NA, scores=FALSE,SMC=TRUE,missing=FALSE,impute="median", min.err = .001, digits=2,max.iter=50,symmetric=TRUE,warnings=TRUE,fm="wls") #deprecated
r 
A correlation matrix or a raw data matrix. If raw data, the correlation matrix will be found using pairwise deletion. 
x 
For fa.poly.ci, only raw data may be used 
nfactors 
Number of factors to extract, default is 1 
n.obs 
Number of observations used to find the correlation matrix if using a correlation matrix. Used for finding the goodness of fit statistics. Must be specified if using a correlaton matrix and finding confidence intervals. 
np.obs 
The pairwise number of observations. Used if using a correlation matrix and asking for a minchi solution. 
rotate 
"none", "varimax", "quartimax", "bentlerT", "geominT" and "bifactor" are orthogonal rotations. "promax", "oblimin", "simplimax", "bentlerQ, "geominQ" and "biquartimin" and "cluster" are possible rotations or transformations of the solution. The default is to do a oblimin transformation, although versions prior to 2009 defaulted to varimax. 
n.iter 
Number of bootstrap interations to do in fa or fa.poly 
residuals 
Should the residual matrix be shown 
scores 
the default="regression" finds factor scores using regression. Alternatives for estimating factor scores include simple regression ("Thurstone"), correlaton preserving ("tenBerge") as well as "Anderson" and "Bartlett" using the appropriate algorithms (see factor.scores). Although scores="tenBerge" is probably preferred for most solutions, it will lead to problems with some improper correlation matrices. 
SMC 
Use squared multiple correlations (SMC=TRUE) or use 1 as initial communality estimate. Try using 1 if imaginary eigen values are reported. If SMC is a vector of length the number of variables, then these values are used as starting values in the case of fm='pa'. 
covar 
if covar is TRUE, factor the covariance matrix, otherwise factor the correlation matrix 
missing 
if scores are TRUE, and missing=TRUE, then impute missing values using either the median or the mean 
impute 
"median" or "mean" values are used to replace missing values 
min.err 
Iterate until the change in communalities is less than min.err 
digits 
How many digits of output should be returned– deprecated – now specified in the print function 
max.iter 
Maximum number of iterations for convergence 
symmetric 
symmetric=TRUE forces symmetry by just looking at the lower off diagonal values 
warnings 
warnings=TRUE => warn if number of factors is too many 
fm 
factoring method fm="minres" will do a minimum residual (OLS), fm="wls" will do a weighted least squares (WLS) solution, fm="gls" does a generalized weighted least squares (GLS), fm="pa" will do the principal factor solution, fm="ml" will do a maximum likelihood factor analysis. fm="minchi" will minimize the sample size weighted chi square when treating pairwise correlations with different number of subjects per pair. 
alpha 
alpha level for the confidence intervals for RMSEA 
p 
if doing iterations to find confidence intervals, what probability values should be found for the confidence intervals 
oblique.scores 
When factor scores are found, should they be based on the pattern matrix (default) or the structure matrix (oblique.scores=TRUE). 
... 
additional parameters, specifically, keys may be passed if using the target rotation, or delta if using geominQ, or whether to normalize if using Varimax 
Factor analysis is an attempt to approximate a correlation or covariance matrix with one of lesser rank. The basic model is that nRn = nFk kFn' + U2 where k is much less than n. There are many ways to do factor analysis, and maximum likelihood procedures are probably the most preferred (see factanal
). The existence of uniquenesses is what distinguishes factor analysis from principal components analysis (e.g., principal
). If variables are thought to represent a “true" or latent part then factor analysis provides an estimate of the correlations with the latent factor(s) representing the data. If variables are thought to be measured without error, then principal components provides the most parsimonious description of the data.
The fa function will do factor analyses using one of four different algorithms: minimum residual (minres), principal axes, weighted least squares, or maximum likelihood.
Principal axes factor analysis has a long history in exploratory analysis and is a straightforward procedure. Successive eigen value decompositions are done on a correlation matrix with the diagonal replaced with diag (FF') until ∑(diag(FF')) does not change (very much). The current limit of max.iter =50 seems to work for most problems, but the HolzingerHarmon 24 variable problem needs about 203 iterations to converge for a 5 factor solution.
Not all factor programs that do principal axes do iterative solutions. The example from the SAS manual (Chapter 26) is such a case. To achieve that solution, it is necessary to specify that the max.iterations = 1. Comparing that solution to an iterated one (the default) shows that iterations improve the solution. In addition, fm="minres" or fm="mle" produces even better solutions for this example.
Principal axes may be used in cases when maximum likelihood solutions fail to converge, although fm="minres" will also do that and tends to produce better (smaller residuals) solutions.
The fm="minchi" option is a variation on the "minres" (ols) solution and minimizes the sample size weighted residuals rather than just the residuals.
A problem in factor analysis is to find the best estimate of the original communalities. Using the Squared Multiple Correlation (SMC) for each variable will underestimate the communalities, using 1s will over estimate. By default, the SMC estimate is used. In either case, iterative techniques will tend to converge on a stable solution. If, however, a solution fails to be achieved, it is useful to try again using ones (SMC =FALSE). Alternatively, a vector of starting values for the communalities may be specified by the SMC option.
The iterated principal axes algorithm does not attempt to find the best (as defined by a maximum likelihood criterion) solution, but rather one that converges rapidly using successive eigen value decompositions. The maximum likelihood criterion of fit and the associated chi square value are reported, and will be worse than that found using maximum likelihood procedures.
The minimum residual (minres) solution is an unweighted least squares solution that takes a slightly different approach. It uses the optim
function and adjusts the diagonal elements of the correlation matrix to mimimize the squared residual when the factor model is the eigen value decomposition of the reduced matrix. MINRES and PA will both work when ML will not, for they can be used when the matrix is singular. At least on a number of test cases, the MINRES solution is slightly more similar to the ML solution than is the PA solution. To a great extent, the minres and wls solutions follow ideas in the factanal
function.
The weighted least squares (wls) solution weights the residual matrix by 1/ diagonal of the inverse of the correlation matrix. This has the effect of weighting items with low communalities more than those with high communalities.
The generalized least squares (gls) solution weights the residual matrix by the inverse of the correlation matrix. This has the effect of weighting those variables with low communalities even more than those with high communalities.
The maximum likelihood solution takes yet another approach and finds those communality values that minimize the chi square goodness of fit test. The fm="ml" option provides a maximum likelihood solution following the procedures used in factanal
but does not provide all the extra features of that function.
Test cases comparing the output to SPSS suggest that the PA algorithm matches what SPSS calls uls, and that the wls solutions are equivalent in their fits. The wls and gls solutions have slightly larger eigen values, but slightly worse fits of the off diagonal residuals than do the minres or maximum likelihood solutions. Comparing the results to the examples in Harman (76), the PA solution with no iterations matches what Harman calls Principal Axes (as does SAS), while the iterated PA solution matches his minres solution. The minres solution found in psych tends to have slightly smaller off diagonal residuals (as it should) than does the iterated PA solution.
Although for items, it is typical to find factor scores by scoring the salient items (using, e.g., score.items
) factor scores can be estimated by regression as well as several other means. There are multiple approaches that are possible (see Grice, 2001) and the one taken here was developed by tenBerge et al. (see factor.scores
. The alternative, which will match factanal is to find the scores using regression – Thurstone's least squares regression where the weights are found by W = inverse(R)S where R is the correlation matrix of the variables ans S is the structure matrix. Then, factor scores are just Fs = X W.
In the oblique case, the factor loadings are referred to as Pattern coefficients and are related to the Structure coefficients by S = P Phi and thus P = S Phi^1. When estimating factor scores, fa
and factanal
differ in that fa
finds the factors from the Structure matrix and factanal
seems to find them from the Pattern matrix. Thus, although in the orthogonal case, fa and factanal agree perfectly in their factor score estimates, they do not agree in the case of oblique factors. Setting oblique.scores = FALSE will produce factor score estimate that match those of factanal
.
It is sometimes useful to extend the factor solution to variables that were not factored. This may be done using fa.extension
. Factor extension is typically done in the case where some variables were not appropriate to factor, but factor loadings on the original factors are still desired.
For dichotomous items or polytomous items, it is recommended to analyze the tetrachoric
or polychoric
correlations rather than the Pearson correlations. This is done automatically when using irt.fa
or fa.poly
functions. In the first case, the factor analysis results are reported in Item Response Theory (IRT) terms, although the original factor solution is returned in the results. In the later case, a typical factor loadings matrix is returned, but the tetrachoric/polychoric correlation matrix and item statistics are saved for reanalysis by irt.fa
Of the various rotation/transformation options, varimax, Varimax, quartimax, bentlerT, geominT, and bifactor do orthogonal rotations. Promax transforms obliquely with a target matix equal to the varimax solution. oblimin, quartimin, simplimax, bentlerQ, geominQ and biquartimin are oblique transformations. Most of these are just calls to the GPArotation package. The “cluster” option does a targeted rotation to a structure defined by the cluster representation of a varimax solution. With the optional "keys" parameter, the "target" option will rotate to a target supplied as a keys matrix. (See target.rot
.)
The "bifactor" rotation implements the Jennrich and Bentler (2011) bifactor rotation by calling the GPForth function in the GPArotation package and using two functions adapted from the MatLab code of Jennrich and Bentler.
There are two varimax rotation functions. One, Varimax, in the GPArotation package does not by default apply Kaiser normalization. The other, varimax, in the stats package, does. It appears that the two rotation functions produce slightly different results even when normalization is set. For consistency with the other rotation functions, Varimax is probably preferred.
When factor analyzing items with dichotomous or polytomous responses, the irt.fa
function provides an Item Response Theory representation of the factor output. The factor analysis results are available, however, as an object in the irt.fa output.
The function fa
will repeat the analysis n.iter times on a bootstrapped sample of the data (if they exist) or of a simulated data set based upon the observed correlation matrix. The mean estimate and standard deviation of the estimate are returned and will print the original factor analysis as well as the alpha level confidence intervals for the estimated coefficients. The bootstrapped solutions are rotated towards the original solution using target.rot. The factor loadings are ztransformed, averaged and then back transformed. The default is to have n.iter =1 and thus not do bootstrapping.
fa.poly
will find confidence intervals for a factor solution for dichotomous or polytomous items (set n.iter > 1 to do so). Perhaps more useful is to find the Item Response Theory parameters equivalent to the factor loadings reported in fa.poly by using the irt.fa
function.
Some correlation matrices that arise from using pairwise deletion or from tetrachoric or polychoric matrices will not be proper. That is, they will not be positive semidefinite (all eigen values >= 0). The cor.smooth
function will adjust correlation matrices (smooth them) by making all negative eigen values slightly greater than 0, rescaling the other eigen values to sum to the number of variables, and then recreating the correlation matrix. See cor.smooth
for an example of this problem using the burt
data set.
For those who like SPSS type output, the measure of factoring adequacy known as the KaiserMeyerOlkin KMO
test may be found from the correlation matrix or data matrix using the KMO
function. Similarly, the Bartlett's test of Sphericity may be found using the cortest.bartlett
function.
values 
Eigen values of the common factor solution 
e.values 
Eigen values of the original matrix 
communality 
Communality estimates for each item. These are merely the sum of squared factor loadings for that item. 
rotation 
which rotation was requested? 
n.obs 
number of observations specified or found 
loadings 
An item by factor (pattern) loading matrix of class “loadings" Suitable for use in other programs (e.g., GPA rotation or factor2cluster. To show these by sorted order, use 
Structure 
An item by factor structure matrix of class “loadings". This is just the loadings (pattern) matrix times the factor intercorrelation matrix. 
fit 
How well does the factor model reproduce the correlation matrix. This is just (sum(r^2ij  sum(r*^2ij))/sum(r^2ij (See 
fit.off 
how well are the off diagonal elements reproduced? 
dof 
Degrees of Freedom for this model. This is the number of observed correlations minus the number of independent parameters. Let n=Number of items, nf = number of factors then

objective 
value of the function that is minimized by maximum likelihood procedures. This is reported for comparison purposes and as a way to estimate chi square goodness of fit. The objective function is

STATISTIC 
If the number of observations is specified or found, this is a chi square based upon the objective function, f. Using the formula from 
PVAL 
If n.obs > 0, then what is the probability of observing a chisquare this large or larger? 
Phi 
If oblique rotations (using oblimin from the GPArotation package or promax) are requested, what is the interfactor correlation. 
communality.iterations 
The history of the communality estimates (For principal axis only.) Probably only useful for teaching what happens in the process of iterative fitting. 
residual 
If residuals are requested, this is the matrix of residual correlations after the factor model is applied. 
rms 
This is the sum of the squared (off diagonal residuals) divided by the degrees of freedom. Comparable to an RMSEA which, because it is based upon chi^2, requires the number of observations to be specified. The rms is an empirical value while the RMSEA is based upon normal theory and the noncentral chi^2 distribution. 
TLI 
The Tucker Lewis Index of factoring reliability which is also known as the nonnormed fit index. 
BIC 
Based upon chi^2 
R2 
The multiple R square between the factors and factor score estimates, if they were to be found. (From Grice, 2001). Derived from R2 is is the minimum correlation between any two factor estimates = 2R21. 
r.scores 
The correlations of the factor score estimates using the regression model, if they were to be found. Comparing these correlations with that of the scores themselves will show, if an alternative estimate of factor scores is used (e.g., the tenBerge method), the problem of factor indeterminacy. For these correlations will not necessarily be the same. 
weights 
The beta weights to find the factor score estimates. These are also used by the 
scores 
The factor scores as requested. Note that these scores reflect the choice of the way scores should be estimated (see scores in the input). That is, simple regression ("Thurstone"), correlaton preserving ("tenBerge") as well as "Anderson" and "Bartlett" using the appropriate algorithms (see factor.scores). The correlation between factor score estimates (r.scores) is based upon using the regression/Thurstone approach. The actual correlation between scores will reflect the rotation algorithm chosen and may be found by correlating those scores. 
valid 
The validity coffiecient of course coded (unit weighted) factor score estimates (From Grice, 2001) 
score.cor 
The correlation matrix of course coded (unit weighted) factor score estimates, if they were to be found, based upon the loadings matrix rather than the weights matrix. 
SPSS will sometimes use a Kaiser normalization before rotating. This will lead to different solutions than reported here. To get the Kaiser normalized loadings, use kaiser
.
The communality for a variable is the amount of variance accounted for by all of the factors. That is to say, for orthogonal factors, it is the sum of the squared factor loadings (rowwise). The communality is insensitive to rotation. However, if an oblique solution is found, then the communality is not the sum of squared pattern coefficients. In both cases (oblique or orthogonal) the communality is the diagonal of the reproduced correlation matrix where nRn = nPk kΦ k kPn' where P is the pattern matrix and Φ is the factor intercorrelation matrix. Similarly, the eigen values are the diagonal of the product Φ_{k k}P'_nnP_k.
Thanks to Erich Studerus for some very helpful suggestions about various rotation and factor scoring algorithms, and to Gumundur Arnkelsson for suggestions about factor scores for singular matrices.
The fac function is the original fa function which is now called by fa repeatedly to get confidence intervals.
William Revelle
Gorsuch, Richard, (1983) Factor Analysis. Lawrence Erlebaum Associates.
Grice, James W. (2001), Computing and evaluating factor scores. Psychological Methods, 6, 430450
Harman, Harry and Jones, Wayne (1966) Factor analysis by minimizing residuals (minres), Psychometrika, 31, 3, 351368.
Revelle, William. (in prep) An introduction to psychometric theory with applications in R. Springer. Working draft available at http://personalityproject.org/r/book/
principal
for principal components analysis (PCA), irt.fa
for Item Response Theory analyses using factor analysis, VSS
for the Very Simple Structure (VSS) and MAP criteria for the number of factors, ICLUST
for hierarchical cluster analysis alternatives to factor analysis or principal components analysis, predict.psych
to find predicted scores based upon new data, fa.extension
to extend the factor solution to new variables, omega
for hierarchical factor analysis.
KMO
and cortest.bartlett
for various tests that some people like.
#using the Harman 24 mental tests, compare a principal factor with a principal components solution pc < principal(Harman74.cor$cov,4,rotate="varimax") pa < fa(Harman74.cor$cov,4,fm="pa" ,rotate="varimax") #principal axis uls < fa(Harman74.cor$cov,4,rotate="varimax") #unweighted least squares is minres wls < fa(Harman74.cor$cov,4,fm="wls") #weighted least squares #to show the loadings sorted by absolute value print(uls,sort=TRUE) #then compare with a maximum likelihood solution using factanal mle < factanal(covmat=Harman74.cor$cov,factors=4) factor.congruence(list(mle,pa,pc,uls,wls)) #note that the order of factors and the sign of some of factors may differ #finally, compare the unrotated factor, ml, uls, and wls solutions wls < fa(Harman74.cor$cov,4,rotate="none",fm="wls") pa < fa(Harman74.cor$cov,4,rotate="none",fm="pa") minres < factanal(factors=4,covmat=Harman74.cor$cov,rotation="none") mle < fa(Harman74.cor$cov,4,rotate="none",fm="mle") uls < fa(Harman74.cor$cov,4,rotate="none",fm="uls") factor.congruence(list(minres,mle,pa,wls,uls)) #in particular, note the similarity of the mle and min res solutions #note that the order of factors and the sign of some of factors may differ #an example of where the ML and PA and MR models differ is found in Thurstone.33. #compare the first two factors with the 3 factor solution Thurstone.33 < as.matrix(Thurstone.33) mle2 < fa(Thurstone.33,2,rotate="none",fm="mle") mle3 < fa(Thurstone.33,3 ,rotate="none",fm="mle") pa2 < fa(Thurstone.33,2,rotate="none",fm="pa") pa3 < fa(Thurstone.33,3,rotate="none",fm="pa") mr2 < fa(Thurstone.33,2,rotate="none") mr3 < fa(Thurstone.33,3,rotate="none") factor.congruence(list(mle2,mr2,pa2,mle3,pa3,mr3))