Asymptotics for L 1 regression estimators under ge

AsymptoticsforL1regressionestimatorsundergeneralconditionsKeithKnightDepartmentofStatisticsUniversityofTorontoAbstract:Itiswell-knownthatL1-estimatorsofregressionparametersareasymptoti-callyNormalifthedistributionfunctionhasapositivederivativeat0.Inthispaper,wederivetheasymptoticdistributionsundermoregeneralconditionsonthebehaviourofthedistributionfunctionnear0.SecondorderorweakBahadur-Kieferrepresentationsarealsoderived.1IntroductionConsiderthelinearregressionmodelYi=0+1x1i++pxpi+i(1)where0;1;;pareunknownparametersandfigareunobservableindependent,identicallydistributed(i.i.d.)randomvariableseachwithmedian0.Forsimplicity,wewillassumethatthexki’sarenon-randomalthoughtheresultswilltypicallyholdforrandomxki’s.WewillconsidertheasymptoticbehaviourofL1-estimatorsof=(0;;p);thatis,b0;b1;bpminimizetheobjectivefunctiongn()=nXi=1jYi01x1ipxpijoverall=(0;;p).TheasymptoticbehaviourofL1-estimatorsinregressioniswell-known,atleastinthecasewheretheerrorshaveadistributionfunctionF(t)whichisdierentiableat0withthederivativepositive.Inparticular,ifwedenotethisderivativeby=F0(0),wehave(XTnXn)1=2(b)convergesindistributiontoa(p+1)-variateNormaldistributionwithmeanvector0andcovariancematrix(42)1Iprovidedthatmax1inxTi(XTnXn)1xi!0asn!1wherexTi=(1;x1i;;xpi)andXnisthen(p+1)matrixwhosei-throwisxTi.(NotethatxTi(XTnXn)1xi(i=1;;n)aresimplythediagonalelementsoftheso-calledhatmatrix1Xn(XTnXn)1XTn.)Ifn1(XTnXn)!CforsomepositivedenitematrixCthenitwillfollowthatpn(b)convergesindistributiontothe(p+1)-variateNormaldistributionwhosecovariancema-trixis(42)1C1.(See,forexample,BassettandKoenker(1978),BloomeldandSteiger(1983),Baietal(1990)andPollard(1991)forvariousapproachestoprovingtheasymptoticNormality.)SecondorderresultsaregivenbyArcones(1996a,1996b),Babu(1989)andHeandShao(1996).Anaturalquestiontoaskiswhathappenswhenthedistributionfunctiondoesnothaveapositivederivativeat0.Whilethesecasesmayseempathological,theyare,infact,farfromit.Indeed,whileassumingtheexistenceofadensityseemsreasonable,itisanassumptionwhichisdiculttoverify.Infact,previousworksuggeststhattheasymptoticbehaviourofL1-estimatorsisverysensitivetothisassumption.Fori.i.d.observations,Smirnov(1952)identiesfourpossibletypesoflimitingdistributionsforsamplequantilesandcharacterizestheirdomainsofattraction.Jureckova(1983)considerstheasymptoticbehaviourofM-estimatorsoflocationundernon-regularconditions;herresultsincludetheL1estimatoroflocation(namelythesamplemedian)asaspecialcase.Onanotherfront,Arcones(1994)considerstheasymptoticbehaviourofso-calledLp-median(thatis,minimizersofPni=1jYijp)for0p1=2andshowsthattheconvergencerateisslowerthanOp(n1=2).ToconsidertheasymptoticbehaviourofL1-estimators,wewillstartbydening(forsomesequenceofconstantsan),n(t)=Zt0pn(F(s=an)F(0))ds(2)whichforeachnisaconvexfunction.Ifthelimitoffn(t)gexistsforeacht,dene(t)=limn!1n(t):(3)(t)(ifitexists)isaconvexfunctiontakingvaluesin[0;1];notethat(t)mayequal1althoughtypically(t)willbenite.(SeeExamples3and4insection3forcaseswhere(t)=1forcertaint.)Theexactformofin(3)canbemoreeasilyobtainedbyconsideringthelimitofpn(F(t=an)F(0));iflimn!1pn(F(t=an)F(0))=(t)(4)thentypically(t)=Zt0(s)ds:InthecasewhereF(x)isdierentiableatx=0(withF0(0)0)thenan=pnand(t)=twhere=F0(0)andso(t)=t2=2.Moregenerally,condition(4)includescaseswhereFhasone-sidedderivativesat0((t)=+tfort0and(t)=tfort0wherean=pn)orisregularlyvaryinginaneighbourhoodof0.TheseconditionsareverysimilartothosegivenbySmirnov(1952).TheseassumptionsaresomewhatweakerthanthoseusedinJureckova(1983)2forthelocationcase.Inparticular,noticethatitisnotnecessarytoassumethatFisabsolutelycontinuous(withrespecttoLebesguemeasure);infact,Fcancontaindiscretecomponents.Wewillshowthat(undersuitableregularityconditionsonthedesign)an(bn)convergesindistribution.Todothis,wewillrstmodifytheobjectivefunctiongnasfollows:Zn(u)=anpnnXi=1hjixTiu=anjjiji:(5)ItiseasytoseethatthevectorbunwhichminimizesZnissimplyan(bn).IfonenowregardsfZngasasequenceofrandomconvexfunctionsonRp+1andifthenitedimensionaldistributionsofZn(u)convergeindistributiontothoseofsomefunctionZ(u)whichhasauniqueminimumUthenitwillfollowthatUn=an(bn)!dU=argmin(Z)asn!1(seeHjrtandPollard,1993;Geyer,1996).2LimitingdistributionsWewillnowformallystatetheregularityconditionsneededtondthelimitingdistributionoftheL1-estimator.(A1)figarei.i.d.randomvariableswithmedian0withdistributionfunctionFcontinuousat0.(A2)ForsomepositivedenitematrixC,limn!11nXTnXn=C:(A3)Foreachu,limn!11nnXi=1n(uTxi)=(u)forsomeconvexfunction(u)takingvaluesin[0;1]wherefn(t)gisdenedasin(2)(forsomesequencefang).Atthispoint,itisworthmakingafewcommentsontheregularityconditions.Condition(A2)isstandardandimplies,forexample,that1nmax1inxTixi!0:(A3)issimilarinspiritto(A2);itisessentiallyanothermomentconditionforthexi’s.If(t)(denedin(3))isniteforalltthen(u)in(A3)cansometimesbeevaluatedas(u)=limn!11nnXi=1(uTxi)3(assumingtheconvergenceofntoissucientlyuniform).Ifthisisth