Smoothing hazard functions and time-varying effect

SmoothingHazardFunctionsandTime-VaryingEffectsinDiscreteDurationandCompetingRisksModelsLudwigFAHRMEIRandStefanWAGENPFEILAddressforcorrespondence:Prof.FahrmeirInstituteofStatistics,UniversityofMunichLudwigstr.33/II80539Munich,GermanyTel.:+89/2180-2220Fax:[phone]Email:[e-mail]:LudwigFahrmeirisProfessorofStatistics,andStefanWagenpfeilisResearchAssistantintheSonderforschungsbereich386,StatistischeAnalysediskreterStrukturen,InstituteofStatistics,UniversityofMunich,80539Munich,Germany.ThisresearchwassupportedbytheGermanScienceFoundation(DFG).TheauthorsthankLeonhardKnorr-Heldandthetworefereesforvaluablecomments.1_____________________________________________________________________________Statespaceordynamicapproachestodiscreteorgroupeddurationdatawithcompetingrisksormultipleterminatingeventsallowsimultaneousmodellingandsmoothestimationofhazardfunctionsandtime-varyingeffectsinaflexibleway.FullBayesianorposteriormeanestimation,usingnumericalintegrationtechniquesorMonteCarlomethods,canbecomecomputationallyratherdemandingoreveninfeasibleforhigherdimensionsandlargerdatasets.Therefore,basedonpreviousworkonfilteringandsmoothingformulticategoricaltimeseriesandlongitudinaldata,ourapproachusesposteriormodeestimation.Thuswehavetomaximizeposteriordensitiesor,equivalently,apenalizedlikelihood,whichenforcessmoothnessofhazardfunctionsandtime-varyingeffectsbyaroughnesspenalty.DroppingtheBayesiansmoothnesspriorandadoptinganonparametricviewpoint,onemightalsostartdirectlyfrommaximizingthispenalizedlikelihood.WeshowhowFisherscoringsmoothingiterationscanbecarriedoutefficientlybyiterativelyapplyinglinearKalmanfilteringandsmoothingtoaworkingmodel.ThisalgorithmcanbecombinedwithanEM-typeproceduretoestimateunknownsmoothing-orhyperparameters.Themethodsareappliedtoalargersetofunemploymentdurationdatawithoneand,inafurtheranalysis,multipleterminatingeventsfromtheGermansocio-economicpanelGSOEP.KEYWORDS:Fisherscoring;IterativelyweightedKalmansmoothing;Multiplemodesoftheterminatingevent;Penalizedlikelihood;Posteriormodesmoothing;Survivalanalysis._____________________________________________________________________________1.INTRODUCTIONInmanyapplicationsdurationorsurvivaltimesarenotobservedcontinuously,butareonlyknowntoliebetweenapairofconsecutivefollowups.AtypicalexamplearedataondurationofunemploymentintheGermansocio-economicpanelGSOEP,wheretimeismeasuredinmonths.Oftenthereare,saym,differenttypesofterminatingcausesorcompetingrisksasinourapplicationtounemploymentdatafromtheGSOEPinSection4,wherewe2distinguishbetweenfull-timejobs,part-timejobsandothercausesforendofunemployment.Ofcourse,inthisexamplecompetingchanceswouldbemoreappropriatethancompetingrisks.Inthefollowinglettimebedividedintokintervalsbbbbbbbqqq01121,,,,...,,,,ggii−∞ofequallengthwhereq=k-1.Forthefirstintervalb00=maybeassumedandbqdenotesthefinalfollowup.Insteadofcontinuoustime,discretetimeΤ∈1,...,kkpisobservedwhereΤ=tdenotesendofdurationwithintheinterval[)bbtt−1,.Distinctterminatingcauses,competingrisksortypesoffailurearedenotedbyR∈{1,...,m}.ClassicalformulationsofcompetingrisksmodelsintroducelatentdurationsTTm1,...,,oneforeachterminatingcause.TheobserveddurationTandterminatingcauseRarethenviewedasT=min(TTm1,...,)andR=rifT=Tr.Thisclassicalconceptisquitenaturalforexampleinanindustrialstudy,whereTTm1,...,arefailuretimesofdifferentcomponentsofasystemordevice,anditmayormaynotbemeaningfulinamedicalstudy.However,insocialsciencesitgenerellydoesnotappeartobemeaningful.Forexample,inourapplicationtodurationofunemployment,thenotionofatimeitwouldtakeanunemployedpersontogetapart-timejobgivenheisprecludedtogetafull-timejobisnotmeaningfulorofprimeinterest.Therefore,ase.g.inPrenticeetal.(1978)andLancaster(1990,pp.99),weformulatedurationandcompetingrisksmodelsintermsofcause-specifichazardfunctionsasthebasiccharacteristicsfortheobservablesTandR.Thecause-specificdiscretehazardfunctionresultingfromcauseorriskrisgivenbytheconditionalprobabilityλrtttxprtRrtxchch===≥ΤΤ,,,(1.1)r=1,...,m,t=1,...,q,wherextisavectorof,possiblytime-dependent,covariates.Theoverallhazardfunction,regardlessofcause,isgivenby3λλtxtxPttxtrrmttchchch=∑==≥=1ΤΤ,.Modellingofcause-specifichazardfunctionscanbebasedonmulticategoricalresponsemodels.Afirstcandidateforunorderedeventsisthemultinomiallogitmodelλγβγβrttrtrtitiimtxxxchbgbg=+′++′∑=expexp11.(1.2)Theparametersγγ1rqr,...,representthecause-specificbaselinehazardfunctionandβristhecause-specificeffect.Iftheeventsareordered,ordinalmodelslikecumulativeorsequentialmodelsareappropriate.Ifthereisonlyonetypeofterminatingevent,wehavethesituationofdiscreteorgroupedsurvivaldata.Then(1.1)reducestothediscretehazardfunction()()λtxprttxtt==≥ΤΤ,,t=1,...,q,and(1.2)tothebinarylogitmodelλγβγβtxxxtttttchbgbg=+′++′expexp1,(1.3)seee.g.Thompson(1977),ArjasandHaara(1987).Alternatively,onemayconsiderthegroupedproportionalhazardsorCoxmodelλγβtxxtttchbgmr=−−+′1expexp,(1.4)seee.g.KalbfleischandPrentice(1980).Iftheintervalsareshort,themodelsbecomeverysimilarashasbeenshownbyThompson