CONDITIONAL INDEPENDENCE MODELS FOR SEEMINGLY UNRE

CONDITIONALINDEPENDENCEMODELSFORSEEMINGLYUNRELATEDREGRESSIONSWITHINCOMPLETEDATAMATHIASDRTON,STEENA.ANDERSSON,ANDMICHAELD.PERLMANAbstract.WeconsidernormalGaussianseeminglyunrelatedregressions(SUR)modelswithincompletedata(ID).Imposinganaturalminimalsetofconditionalindependencecon-straints,wendrestrictedSUR/IDmodelsforwhichthelikelihoodfunctionandtheparameterspacefactorsintotheproductofthelikelihoodfunctionsandtheparameterspacesofstandardcompletedatamultivariateanalysisofvariancemodels.Hence,therestrictedmodelhasauni-modallikelihoodandpermitsexplicitlikelihoodinference.Therestrictedmodelmaybeusedtodirectlymodelthedataactuallyobserved.Alternatively,themaximumlikelihoodestimatesintherestrictedmodelcanyieldimprovedstartingvaluesforiterativemethodstomaximizethelikelihoodoftheunrestrictedSUR/IDmodel.Inthedevelopmentofourmethodology,wereviewandextendexistingresultsforcompletedataSURmodelsandthemultivariateIDproblem.TheresultsarepresentedintheframeworkofbothlatticeconditionalindependencemodelsandgraphicalMarkovmodelsbasedonacyclicdirectedgraphs.Date:October1,2003.Keywordsandphrases.Acyclicdirectedgraph,graphicalmodel,incompletedata,latticeconditionalindepen-dencemodel,MANOVA,maximumlikelihoodestimator,multivariateanalysis,multivariatelinearmodel,missingdata,seeminglyunrelatedregressions.12MATHIASDRTON,STEENA.ANDERSSON,ANDMICHAELD.PERLMANContents1.Introduction32.Latticeconditionalindependencetheory52.1.LatticesandLCImodels52.2.Thealgebraofgeneralizedblock-triangularmatriceswithlatticestructure62.3.TheMANOVAmodel72.4.ThelinearLCImodelanditslikelihoodfactorization83.Latticeinclusion93.1.Aninclusioncriterionbasedonjoin-irreducibleelements93.2.Aninclusioncriterionbasedonthealgebraofgeneralizedblock-triangularmatrices104.Seeminglyunrelatedregressions114.1.TheSURmodel114.2.LCIrestrictionsforaSURmodel124.3.MinimalityoftheLCIrestrictionsforaSURmodel135.Multivariateincompletedata145.1.IDpatterns145.2.TheIDlattice156.Linearincompletedatamodels176.1.LinearIDsubspaces176.2.LCIrestrictionsforalinearIDmodel206.3.MinimalityoftheLCIrestrictionsforalinearIDmodel227.Seeminglyunrelatedregressionswithincompletedata237.1.TheSUR/IDmodel237.2.LCIrestrictionsforaSUR/IDmodel257.3.MinimalityoftheLCIrestrictionsforaSUR/IDmodel278.Acyclicdirectedgraphtheory288.1.Directedgraphs288.2.NormalgraphicalMarkovmodelsbasedonacyclicdirectedgraphs298.3.EquivalenceoftransitiveADGandLCImodels298.4.ConstructionoftheTADGsequivalenttotheparsimoniousLCImodels309.Examples3010.SummaryandConclusion43References44CIMODELSFORSURWITHINCOMPLETEDATA31.IntroductionTheseeminglyunrelatedregressions(SUR)modelisanextensionofthemultivariateanalysisofvariance(MANOVA)model.InMANOVA,eachoftheobservedvariablesisregressedonthesamemeanspace.InSUR,thisisrelaxedbyallowingdierentvariablestoberegressedondierentmeanspaces.TheSURmodelwasmadeprominentinthe1960sbyZellner[40,41]whoestablishedtheasymptoticeciencyofhistwo-stageestimator.AccordingtoGoldberger[13,p.323],it\playsacentralroleincontemporaryeconometrics.AnintroductiontoSURcanbefound,forexample,intheeconometricstextbooksbyGoldberger[13]andbyGreene[15]butalsointhemultivariatestatisticsmonographbyMardia,Kent,andBibby[25].IntheSURliterature,severaldistributionalassumptionshavebeenconsidered.Recentex-amplesincludeKowalskietal.[17]whoassumet-distributederrors,andLefkovitch[20]whoconsidersSURmodelsbasedongeneralizedlinearmodels.Inthispaper,however,wedealexclusivelywiththeclassicalcaseofthenormal(Gaussian)model.Ingeneral,likelihoodinferenceinanormalSURmodelrequiresiterativemethodstomaximizethelikelihoodfunction(LF).ThemostcommonmethodistheiteratedversionofZellner'stwo-stageestimator;analternativeisTelser'smethod[36].NormalSURmodelsarecurvedexponentialfamilies(vanGarderen[37])andtheLFmaybemultimodal.AbivariateexamplewithmultimodalLFisstudiedinDrtonandRichardson[11].However,inthemonotone(triangularnested)caseinwhichtheregressionspacesforthedierentvariablesaretotallyorderedbyinclusiontheLFisunimodalandexplicitlikelihoodinferenceispossiblebyfactoringtheSURmodelintoaproductofMANOVAmodels,cf.AnderssonandPerlman[8].Simplespecialcasescanbefoundearlier,seee.g.Oksanen[30]forabivariateexample.AnderssonandPerlman'smethodology[8]alsocoversnonmonotoneSURmodels.Usinglatticeconditionalindependence(LCI)theorycf.[6],theyshowthatanonmonotoneSURmodeldeterminesauniqueminimalsetofconditionalindependence(CI)restrictionss.t.theLCI-restrictednonmonotoneSURmodelallowsforexplicitlikelihoodinferenceandhasaunimodalLF.Asinthemonotonecase,thekeyideaisthefactorizationoftheSURmodelintoaproductofMANOVAmodels.InaMonteCarlostudy,WuandPerlman[39]comparethenitesampleperformanceoftheestimatorsobtainedintheLCI-restrictedSURmodeltotraditionalmethodssuchasordinaryleastsquaresorZellner'stwo-stageestimator.LCItheoryalsomaybeappliedtoincompletedata(ID)problemswhere,asassumedthrough-outthisarticle,dataismissingatrandomandthemissingdatacanbeignoredintheformulationofthelikelihoodfortheincompletedata(compareLittleandRubin[21,Ch.6]).Forthecaseofi.i.d.multivariatenormalobservations