On confidence intervals for gams based on penalize

OnconfidenceintervalsforGAMsbasedonpenalizedregressionsplinesSimonN.WoodDepartmentofStatistics,UniversityofGlasgow,Glasgow,G128QQUKMarch17,2004AbstractGeneralizedadditivemodelsrepresentedusingpenalizedregressionsplines,estimatedbypenalizedlikelihoodmaximisationandwithsmoothnessselectedbygeneralizedcrossvalidationorsimilarcriteria,provideacomputationallyefficientgeneralframeworkforpracticalsmoothmodelling.VariousauthorshaveproposedapproximateBayesianintervalestimatesforsuchmodels,basedonextensionsofSilverman’s(1985)re-derivationoftheintervalsproposedbyWahba(1983)forsmoothingsplinemodelsofGaussiandata,buttestingofsuchintervalshasbeenratherlimitedandthereislittlesupportingtheoryfortheapproximationsusedinthegeneralizedcase.Thispaperaimstoimprovethissituationbyprovidingsimulationtestsandobtainingasymptoticresultssupportingtheapproximationsemployedforthegeneralizedcase.Thepaperalsosuggestsasimpleandefficientsimu-lationschemefordealingwiththepoorintervalcoveragethatcansometimesresultfromconditioningonsmoothingparameterestimates.Keywords:Penalizedregressionspline;Generalizedadditivemodel;GCV;multiplesmooth-ingparameters;Bayesianconfidenceinterval.11IntroductionPenalizedregressionmethodsprovideausefulmeansformodellinginsituationsinwhichtheclassofpossibleregressionmodelsforasetofdataisratherlarge,butthecomplexityor“wiggli-ness”ofthesemodelscanbemeasured,withwigglymodelsconsideredtobeapriorilesslikelythansmoothmodels.TypicalexamplesarethesmoothingsplineANOVAmodelsofGu(2002),Wahba(1990)andco-workers,ortherathersimilar,butcomputationallylessdemanding,gen-eralizedadditivemodelsofHastieandTibshirani(1990)(seealsoMarxandEilers,1998,orWoodandAugustin,2002).Asimpleexampleofpenalizedregressionmodellingarisesinthecontextofestimatingthesmoothfunctionffromx;ydatamodelledas:yi=f(xi)+²i;²i»N(0;¾2)wherethe²iaremutuallyindependent.Thestandardapproachestothisproblememployeithersmoothingsplines(seee.g.Reinch1971,Duchon1977,Wahba1990,GreenandSilverman,1994,Gu2002)orpenalizedregressionsplines(seeWahba,1980,1990,ParkerandRice,1985,fortheoriginalidea;EilersandMarx,1996,foranelegantdiscretepenaltymethod;Wood,2003,foranoptimalapproach).Theformerrequireasmanyparametersasdata,whilethelatterusefewerparameters,butbothareestimatedbysolutionofaproblemwiththegeneralstructure:minimiseky¡X¯k2+¸¯0S¯w:r:t:¯whereXisadesignmatrix,¯avectoroffreeparameters,Samatrixoffixedcoefficientssuchthat¯0S¯measureswigglinessoffand¸isasmoothingparametercontrollingthetradeoffbetweenmodelfitandmodelsmoothness.Usuallythequadraticpenaltytermisequivalenttoanintegralofsquaredderivativesofthefunction,forexampleR[f00(x)]2dx,butthereareotherpossibilitiessuchasthediscretepenaltiesadvocatedbyEilersandMarx(1986)orWhittaker(1923).Inferencewiththesemodelsiscomplicatedbythefactthat,whilethequadraticpenaltytermactstolimitestimatorvariance,italsobiasestheparameterestimators,ˆ¯=(X0X+¸S)¡1X0y:Asaresultconfidenceintervalsbasedonnaiveuseofˆ¯andthecorrespondingcovariancematrix,Vˆ¯=(X0X+¸S)¡1X0X(X0X+¸S)¡1¾2;generallygivepoorresultsintermsofrealizedcoverageprobabilities.InthecontextofsmoothingsplinemodelsWahba(1983)overcamethepoorperformanceofnaiveintervalsusingaBayesianapproach,inwhichanimproperpriorforfwasconstructedusinganintegratedWienerprocess.Usingthispriorsheshowedthattheusualsplineestimateˆfisthemodeoftheposteriordistributionforfandthattheposteriordistributionoff=(f(x1);f(x2);:::)0isN(ˆf;A¾2)whereAistheinfluencematrixsuchthatˆf=Ay(i.eA=X(X0X+¸S)¡1X0).Silverman(1985)arrivedatthesameresultusingasimpler,moreintuitiveprior.InsimulationstudiesBayesianconfidenceintervalsbasedonthisapproachprovedtohavegoodcoverageproperties,providedcoverageismeasured“acrossthefunction”,ratherthanpointwise,andmostlaterworkinthesmoothingsplinemodellingliteraturehasusedtheseintervalsandtheircomponentwiseextensions(GuandWahba1993),directlyintheGaussiancontext,andusingaGaussianapproximationinthegeneralexponentialfamilycase(e.g.Gu1992).Inotherpenalizedregressionsettingsthetendancyhasbeeneithertousethenaiveintervals(HastieandTibshirani,1990;EilersandMarx,1996)ortoemployapproximateversionsoftheWahba-Silvermanresults(e.g.WoodandAugustin,2002,Wood2000,LinandZhang,1999).Thispaper(i)reviewshow,intheGaussiancase,Silverman’s(1985)approachgeneralizesinastraightforwardmannertoBayesianconfidenceintervalcalculationfortheparametersofgeneralpenalizedregressionmodels;(ii)showsviaasimplere-parameterizationthenaturalnessofthepriorsusedinthesecalculations;(iii)derivesasymptoticresultswhichcanbeusedtocalculateconfidenceintervalsinnon-Gaussiansettings,inparticularforgeneralizedadditivemodelsconstructedfrompenalizedregressionsplines(e.g.MarxandEilers1998;WoodandAugustin,2002;Wood,2003);(iv)providesquiteextensivesimulationtestsoftheintervalsand(v)suggestsasimplemethodfordealingwiththeproblemsthatmayarisebyconditioningonestimatedsmoothingparameters.(iii)isparticularlyusefulinthatitaddssometheoreticalsupporttointervalscalculatedforthisclassofmodelswhichwasnotpreviouslyavailable.Theresultsalsoprovideacomputationallyefficientandselfconsistentmeansforo