Non-positive definite matrix in SPSS FACTOR

I found an interesting comment on non-positive definite matrix. Here :

Date:         Fri, 29 Sep 1995 18:38:27 GMT
Reply-To:     Stat-l Discussion List <[log in to unmask]>
Sender:       Stat-l Discussion List <[log in to unmask]>
Comments:     Warning -- original Sender: tag was [log in to unmask]
From:         David Nichols <[log in to unmask]>
Organization: SPSS, Inc.
Subject:      Re: Non-positive definite matrix in SPSS FACTOR

Stephen Eyres wrote:

> I’m working with SPSS for windows running a principal components
>analysis. A message appears at the beginning of the output to the
>effect of:
> “Warning: Matrix is not positive definite.”
>
> The output continues as I would have expected. The only
>unexpected thing is that it does not produce the Bartlett’s test of
>sphericity that I ordered.
>
> Has anyone encountered this message? What is a positive definite
>matrix and what are the consequences? Does any following output mean
>anything or is it gibberish as a result?

Sorry this took me longer than normal to address. One reason for this is that I don’t believe the precise wording given here appeared in an SPSS error or warning message. I can diagnose and respond more quickly to this kind of issue if an error or warning number is given, or at least the precise text. If you run the SPSS FACTOR procedure in SPSS for Windows asking for a principal components analysis and check the box for KMO and Bartlett’s Test of Sphericity (keyword KMO on the PRINT subcommand), there are three possible sets of responses from FACTOR relevant to this question (at least in a broad sense):

1) No warning or error messages appear, and you get the requested KMO measures, Bartlett test and PC analysis. This implies that the correlation matrix is positive definite (all eigenvalues positive).

2) You get the following warning message:

>Warning # 11301
>The correlation matrix is not positive definite.

You do not get the KMO measures or Bartlett test, but you do get the PC analysis. This implies that the correlation matrix is positive semidefinite but not positive definite (no negative eigenvalues, but at least one 0 eigenvalue). In this case, the determinant of the matrix is 0, which means that the matrix is singular (cannot be inverted using a standard inverse). The KMO measures require inversion of the matrix, so they cannot be printed. The Bartlett test statistic is a function of the log of the determinant, so it also cannot be computed, since the log of 0 is undefined. The PC analysis results are nevertheless valid, though you may want to identify the sources of the dependencies and reduce the variables accordingly (as suggested by Jolliffe, for example, in _Principal Component Analysis_, Springer-Verlag, 1986).

3) You get two messages:

>Warning # 11301
>The correlation matrix is not positive definite.

and

>Warning # 11283
>Non-positive eigenvalues have been found and the matrix is not positive
>definite. This may be due to pairwise deletion of missing values.

You get no KMO measures, no Bartlett test and no PC analysis results. This means that there are negative eigenvalues. If your correlation matrix is computed in FACTOR from raw data without pairwise deletion, this would imply imprecision in our calculation of eigenvalues, since correlation matrices involving Pearson correlations and full data are by definition non-negative definite (have no negative eigenvalues). I don’t recall ever seeing this. As the warning message says, if you use pairwise deletion of missing values, matrices of Pearson correlations can have negative eigenvalues. Negative eigenvalues can also result from the input of correlation matrices calculated outside of FACTOR, either because of loss of precision in translation (common when people attempt to replicate results of published analyses where the correlations are rounded to 2-3 digits), or because of the use of other types of correlation coefficients that do not guarantee positive definite correlation matrices even in complete data (tetrachoric correlations being perhaps the most common example). When there are negative eigenvalues, current versions of SPSS will not provide any PC results, since we don’t know how one would interpret an analysis where p-k components can account for more than 100% of the variance for the p variables, which is what is implied by the values of the eigenvalues when some number (k) of them are negative. Older releases of SPSS blithely went on their way and printed the results of the math, but since neither I nor any of the Development statisticians here could interpret it, they stopped doing that.