Iterative estimating equations Linear convergence

arXiv:0712.0901v1[math.ST]6Dec2007TheAnnalsofStatistics2007,Vol.35,No.5,2233–2260DOI:10.1214/009053607000000208cInstituteofMathematicalStatistics,2007ITERATIVEESTIMATINGEQUATIONS:LINEARCONVERGENCEANDASYMPTOTICPROPERTIESByJimingJiang,1YihuiLuan2andYou-GanWangUniversityofCalifornia,Davis,ShandongUniversityandCSIROMathematicalandInformationSciencesWeproposeaniterativeestimatingequationsprocedureforanal-ysisoflongitudinaldata.Weshowthat,underverymildconditions,theprobabilitythattheprocedureconvergesatanexponentialratetendstooneasthesamplesizeincreasestoinfinity.Furthermore,weshowthatthelimitingestimatorisconsistentandasymptoticallyefficient,asexpected.Themethodappliestosemiparametricregres-sionmodelswithunspecifiedcovariancesamongtheobservations.Inthespecialcaseoflinearmodels,theprocedurereducestoiterativereweightedleastsquares.Finitesampleperformanceoftheprocedureisstudiedbysimulations,andcomparedwithothermethods.Anu-mericalexamplefromamedicalstudyisconsideredtoillustratetheapplicationofthemethod.1.Introduction.Longitudinaldataisoftenencounteredinmedicalre-searchandeconomicsstudies.Intheanalysisoflongitudinaldata(e.g.,Dig-gle,LiangandZeger[3]),theproblemofmaininterestisoftenrelatedtotheestimationofthemeanresponseswhich,underasuitableparametricorsemiparametricmodel,dependonavectorβofunknownparameters.How-ever,themissioniscomplicatedbythefactthattheresponsesarecorrelatedandthecorrelationsareunknown.SupposethatYisavectorofresponsesthatisassociatedwithama-trixXofcovariates,whichmayalsoberandom.Supposethatthe(condi-tional)meanofYisassociatedwithavectorofparameters,θ.FornotationalReceivedNovember2005;revisedJanuary2007.1SupportedinpartbyNSFGrantsSES-99-78101andDMS-04-02824.PartofthisworkwasdonewhilethefirstauthorwasvisitingtheInstituteforMathematicalSciences,NationalUniversityofSingaporein2005.ThevisitwassupportedbytheInstitute.2SupportedinpartbyNSFofChinaGrant10441004.AMS2000subjectclassifications.62J02,65B99,62F12.Keywordsandphrases.Asymptoticefficiency,consistency,iterativealgorithm,linearconvergence,longitudinaldata,semiparametricregression.ThisisanelectronicreprintoftheoriginalarticlepublishedbytheInstituteofMathematicalStatisticsinTheAnnalsofStatistics,2007,Vol.35,No.5,2233–2260.Thisreprintdiffersfromtheoriginalinpaginationandtypographicdetail.12J.JIANG,Y.LUANANDY.-G.WANGsimplicity,writeμ=μ(X,θ)=Eθ(Y|X)andV=Var(Y|X).HereafterVarorEwithoutthesubscriptθismeanttobetakenatthetrueθ.Con-siderthefollowingclassofestimatingfunctionsG={G=A(Y−μ)},whereA=A(X,θ).SupposethatVisknown.Then,byTheorem2.1ofHeyde[7],itiseasytoshowthattheoptimalestimatingfunctionwithinGisgivenbyG∗=˙μ′V−1(Y−μ),thatis,withA∗=˙μ′V−1.Therefore,theoptimalestimatingequationisgivenby˙μ′V−1(Y−μ)=0.Inthecaseoflongitu-dinaldata,theresponsesareclusteredaccordingtothesubjects.LetYibethevectorofresponsescollectedfromtheithsubject,andXithematrixofcovariatesassociatedwithYi.Letμi=E(Yi|Xi)=μi(Xi,β),whereβisavectorofunknownparameters.Then,undertheassumptionthat(Xi,Yi),i=1,...,n,areuncorrelatedwithknownVi=Var(Yi|Xi),theoptimalesti-matingequation(forβ)isgivenbyPni=1˙μ′iV−1i(Yi−μi)=0,whichisknownasthegeneralizedestimatingequations(GEE)(e.g.,Diggle,LiangandZeger[3]).However,theoptimalGEEdependsonVi,1≤i≤n,whichareusuallyun-knowninpractice.Therefore,theoptimalGEEestimatorisnotcomputable.Theproblemofobtainingan(asymptotically)optimal,orefficient,estima-torofβwithoutknowingtheVi’sisthemainconcernofthecurrentpaper.Tomotivateourapproach,letusfirstconsiderasimpleexample.SupposethattheYi’ssatisfya(classical)linearmodel,thatis,E(Yi)=Xiβ,whereXiisamatrixoffixedcovariates,Var(Yi)=V1,whereV1isanunknownfixedcovariancematrix,andY1,...,Ynareindependent.IfV1isknown,βcanbeestimatedbythefollowingbestlinearunbiasedestimator(BLUE),whichistheoptimalGEEestimatorinthisspecialcase:ˆβBLUE=nXi=1X′iV−11Xi!−1nXi=1X′iV−11Yi,(1.1)providedthatPni=1X′iV−11Xiisnonsingular.Ontheotherhand,ifβisknown,V1canbeestimatedconsistentlyasˆV1=1nnXi=1(Yi−Xiβ)(Yi−Xiβ)′.(1.2)Itisclearthatthereisacycle,whichmotivatesthefollowingalgorithm,whenneitherβnorV1isknown.StartingwiththeidentitymatrixIforV1,use(1.1)withV1replacedbyItoobtaintheinitialestimatorforβ,whichisknownastheordinaryleastsquares(OLS)estimator;thenuse(1.2)withβreplacedbytheOLSestimatortoupdateV1;thenuse(1.1)withthenewV1toupdateβ,andsoon.Theprocedureisexpectedtoresultinanestimatorthatis,atleast,moreefficientthantheOLSestimator;butcanweaskafurtherquestion,thatis,istheproceduregoingtoproduceanestimatorthatisasymptoticallyasefficientastheBLUE?BeforethisquestioncanITERATIVEESTIMATINGEQUATIONS3beanswered,however,anotherissueneedstoberesolvedfirst,thatis,doestheiterativeprocedureconverge?Thesequestionswillbeansweredinthesequel.Theexampleconsideredaboveiscalledbalanceddata,inwhichtheob-servationsarecollectedatacommonsetoftimesforallthesubjects.Inthiscase,theprocedure(withoutiterations)isknownasrobustestimationforanalysisoflongitudinaldata.However,aspointedoutbyDiggle,LiangandZeger[3],page77,sofarthemethodhasbeenrestrictedtothecaseofbalanceddata.Inmanypracticalsituations,however,the