Statistical Inference of Minimum Rank Factor Analy

StatisticalInferenceofMinimumRankFactorAnalysisAlexanderShapiro¤SchoolofIndustrialandSystemsEngineeringGeorgiaInstituteofTechnologyAtlanta,Georgia30332-0205,USAe-mail:[e-mail]ofPsychologicalResearchUniversityofGroningenGroteKruisstraat2/19712TSGroningen,TheNetherlandsemail:[e-mail]¤Thisworkwassupported,inpart,bygrantDMI-9713878fromtheNationalScienceFoundation.AbstractForanygivennumberoffactors,MinimumRankFactorAnalysisyieldsoptimalcommunalitiesforanobservedcovariancematrixinthesensethattheunexplainedcommonvariancewiththatnumberoffactorsisminimized,subjecttotheconstraintthatboththediagonalmatrixofuniquevariancesandtheobservedcovariancematrixminusthatdiagonalmatrixarepositivesemide¯nite.Asaresult,itbecomespossibletodistinguishtheexplainedcommonvariancefromthetotalcommonvariance.ThepercentageofexplainedcommonvarianceissimilarinmeaningtothepercentageofexplainedobservedvarianceinPrincipalComponentAnalyis,buttypicallytheformerismuchcloserto100thanthelatter.Sofar,nostatisticaltheoryofMRFAhasbeendeveloped.Thepresentpaperisa¯rststart.Ityieldsclosed-formexpressionsfortheasymptoticbiasoftheexplainedcommonvariance,or,moreprecisely,oftheunex-plainedcommonvariance,undertheassumptionofmultivariatenormality.Also,theasymptoticvarianceofthisbiasisderived,andalsotheasymptoticcovariancematrixoftheuniquevariancesthatde¯neaMRFAsolution.Thepresentedasymptoticsta-tisticalinferenceisbasedonarecentlydevelopedperturbationtheoryofsemide¯niteprogramming.Anumericalexampleisalsoo®eredtodemonstratetheaccuracyoftheexpressions.Keywords:Factoranalysis,communalities,propersolutions,explainedcommonvariance,semide¯niteprogramming,largesamplesasymptotics,asymptoticnormal-ity,asymptoticbias.1IntroductionFactoranalysisisbasedonthenotionthat,givenasetofstandardizedvariablesz1;:::;:zp,eachvariablezjcanbedecomposedintoacommonpartcj,givingrisetocorrelationbetweenzjandzk,j6=k,andauniquepartuj,assumedtobeuncorrelatedwithanyvariableexceptzj,j=1;:::;p.Uponwritingzj=cj+uj;(1.1)j=1;:::;p,andusingtheassumptiononuj,wehave§=§c+ª;(1.2)where§isthecorrelationmatrixofthevariables,ªisthediagonalmatrixofuniquevariances,and§cisthevariance-covariancematrixofthecommonpartscj,j=11;:::;m,ofthevariables.Thevariancesofthesecommonpartsareinthediagonalof§c.Theyaretheso-calledcommunalitiesofthevariables.Theidealoffactoranalysisisto¯ndadecomposition(1.2)with§coflowrankr,whichcanbefactoredas§c=FF0,withFap£rmatrix.Toaccomplishthis,communalitiesarerequiredthatreducetherankof§¡ªtosomesmallvalue.Althoughtheearlydaysoffactoranalysiswerecharacterizedbygreatoptimisminthisrespect(Ledermann,1937),theidealoflowreducedrankwillneverbeattainedinpractice,seeGuttman(1958)andShapiro(1982).AhistoricaloverviewofhowtheidealoflowreducedrankwasshatteredcanbefoundinTenBerge(1998).Forpracticalpurposes,thereisnochoiceotherthantryingtoapproximatetheideallowranksituation,anddecompose§,forsomesmallvalueofr,as§=FF0+(§c¡FF0)+ª:(1.3)Thismeansthatthevariancesofthevariables(diagonalelementsof§)aredecom-posedintoexplainedcommonvariances(diagonalelementsofFF0),unexplainedcommonvariances(diagonalelementsof§c¡FF0),anduniquevariances(diagonalelementsofª).Itisessentialtonotethatapropersolutionfor(1.3)requiresthatbothmatrices§candªshouldbepositivesemide¯nite(denoted§cº0andªº0,respectively).Negativeelementsinª,knownasHeywoodcases,havedrawnalotofattention,andareususallynottolerated.However,when§c,thecovariancematrixforthecommonpartsofthevariables,wouldappeartobeinde¯nite,thatwouldbenolessembarrassingthanhavinganegativeuniquevarianceinª.Nevertheless,popularmethodsofcommonfactoranalysisgenerallyignoretheconstraintthat§c¡FF0mustbepositivesemide¯nite.TheonlyexceptionseemstobeMinimumRankFactorAnalysis(MRFA).ThismethodwasoriginallyproposedbyTenBergeandKiers(1991)asAMRFA,butthe\Aofapproximatehasworno®inthemeantime.MRFAo®ersadecompositionof§thatsatis¯es(1.3),withbothªand§c¡FF0positivesemide¯nite.Subjecttothesetwoconstraints,MRFAconstructsthesolutionthatminimizestheunexplainedcommonvarianceforany¯xednumberoffactorsr.Inotherwords,MRFAapproximatestheidealoflowreducedrankbyminimizingtheamountofcommonvariancethatisleftunexplainedwhenasfewasrfactorsaremaintained.Forap£psymmetricmatrixSwedenoteby¸1(S)¸:::¸¸p(S)itseigenvaluesarrangedindecreasingorder.Formally,MRFAminimizes,for¯xedr,thefunctionfmrfa(ª):=pXi=r+1¸i(§¡ª)(1.4)subjectto§¡ªº0andªº0.ThisisverysimilartoMINRES/IPFA/ULS,2wherethefunctionfminres(ª):=pXi=r+1¸2i(§¡ª)(1.5)isminimized,withoutanyconstraintonthesignoftheseeigenvalues(HarmanandJones(1966),JÄoreskog(1967)).AsimilareigenvalueinterpretationofMaximumLike-lihoodFactorAnalysishasbeengivenbyJÄoreskog(1967,p.449),alsoseeTenBerge(1998)foradiscussion.Akeyfeatureof(1.3)isthedistinctionbetweencommunalitiesasvariances\tobeexplainedontheonehand,andtheexplainedvariancesofthevariables,thedi-agonalelementsofFF0ontheother.Thedi®erencerestsintheunexplainedpartsofthecommunalities,inthediagonalof§c¡FF0.Whenthisdistinctionispre-served,itispossibletoevaluatetowhatextentthecomm