An Introduction to Bootstrap Methods using Arc

AnIntroductiontoBootstrapMethodsusingArcIainPardoeDepartmentofAppliedStatisticsSchoolofStatisticsUniversityofMinnesota,St.Paul,MN55108TechnicalReportNumber631WorkSupportedbytheNationalScienceFoundation,GrantDUE96-52887February24,2000AbstractThisreportpresents(1)thebasicideasofbootstrappingwhenappliedtomul-tiplelinearregression,asdescribedin[2,3],and(2)howtoimplementtheseideasusingArc,thecomputerpackagethataccompanies[1].1IntroductionThisreportprovides(1)anintroductiontobootstrapmethodsinlinearregressionanal-ysesasdiscussedin[2,3],and(2)computercodeforusewiththeprogramArc,de-scribedin[1],thatimplementstheseanalyses.Theremainderofthissectionoutlinesthegeneralideasbehindlinearregression.InSection2,Isummarizethebasicboot-strapapproachtostatisticalinference,andpresenttwowaysofapplyingittolinearregression.Next,IdescribesomeoftheissuesinvolvedwithestimatingandtestingmeanfunctioncoefficientsinSections3and4respectively.Finally,Ipresentsomeex-amplesinSection5,andoutlinesomeoftheotherareasinregressionwherebootstrapmethodscouldbeusedtogoodeffectinSection6.ThemostrecentversionofArccanbeobtainedontheInternetfromthelinkfileficproblemsdiscussed.Asthesemayhaveinterestinvarioussituations,manyreaderswillfindtheadditionsuseful.Theadditionscanalsoprovideastarting1pointforimplementationofthebootstrapinothersituations,butforthisthereadermustbeabletoreadandwritecomputerprogramsinthelanguagelisp.Thereareseveralwaysofgettingstarted.Tierneyin[4]providesaveryreadableintroduction.Severalon-linereferencescanbeobtainedfrom://“SimulationsusingArc”maybehelpful,andisavailablefrom[1],regressionconcernsaresponseandpredictors,.Thegeneralgoalinregressionistostudyhowtheconditionaldistributionofchangesasthevalueofchanges,oftenconcentratingonthemeanfunction,.Inmanyregressionproblems,theresponseiswritten!#whereiscalledthestatisticalerrorandtheweights#%$’&areknown,positivenum-bers.Anotherfeatureoftheconditionaldistributionofthatisoftenstudiedinregressionisthevariancefunction(*)+,*’(*)+-./0#.Let1bea24365vectoroftermsderivedfrom.Typically,1willconsistofaconstant1foranintercept,and72985,additionalfunctionsof,likepolynomialsorothertransformations.Thelinearregressionmodelhasmeanfunction:*;1*=?@=ACBDEEE=GF,HIBF0HJ’K1(1)whereKLMNO=I=F0Hisa2*3P5vectorofmeanfunctioncoefficients,andvariancefunction(*)+QRS.0#(2)TheseassumedformsofthemeanandvariancefunctionsimplythatO0TU&and(*)+,O.VWRS.Thisreflectsanalternativewayofspecifyingthegeneralformofthelinearregressionmodel—thelinearmeanfunction(1)togetherwiththeassumptionthatthedistributionoftheerrorsisindependentof.Forafullparametricanalysis,thedistributionof,oralternativelyof,mustbespecified.Fornormallydistributederrors,theleastsquarestheoryofregressionestimationandinferenceprovidesstraightforward,exactmethodsforanalysis.Butfornon-normalerrors,thesemethodshavethepotentialtobeinaccurateormisleading.Resamplingmethodssuchasthebootstrapprovideanalternativemethodology,withthepotentialtobothXreinforceconclusionsarrivedatusingnormaltheory,and2Ytoprovideestimationandinferencetechniquesinsituationswherenormaltheorydoesnotseemtobejustified.Theexamplesinthisreportfocusmainlyonthefirstofthesegoals,althoughSec-tion6mentionssomeareasthatcouldinvolvemoreinthewayofthesecondofthesegoals.2Twoalternativeparadigmsforusingbootstrapmeth-odsinlinearregressionThebootstrapisadata-basedsimulationmethodforstatisticalinference.Thebasicideaisasfollows.Iwishtomakeaninferenceabouta(population)quantity,sayZ,forwhichIhaveadata-basedestimate,[Z.Ithenwanttogetsomeideaofthedistributionofmyestimate,withouthavingtomakeassumptionsaboutmydata(forexample,thatitcomesfromamultivariatenormaldistribution).Onewaytodothisistoresamplewithreplacementfrommydatatogetabootstrapsample(ofthesamesizeasmyoriginalsample,andmadeupofcasesfrommyoriginalsample,someappearingonce,sometwice,andsoon,andsomenotappearingatall).Ithencreatealargenumber,\,ofsuchbootstrapsamples,andcalculate[Zforeachsample.(Fornotation,Idenotebootstrapestimateswithastar,andhence[Zforabootstrapsampleisdenoted[ZG].)These\[ZG]’scontaininformationthatcanbeusedtomakeinferencesfromthedata;essentially,[ZG]isto[Zas[ZistoZ.Someofthetypesofinferencepos