Bayesian Analysis of Comparative Survey Data Bruce

BayesianAnalysisofComparativeSurveyDataBruceWestern1FilizGaripPrincetonUniversityApril20051DepartmentofSociology,PrincetonUniversity,PrincetonNJ08544.WethankSaraCurranformakingtheNangRongmigrationdataavailableforthispaper.ThisresearchwassupportedbythePrincetonUniversityScienceFund.WinBUGScodefortheanalysisreportedinthispaperisavailableat:∼western.AbstractBayesianhierarchicalmodelsprovideausefulwayofanalyzingmultilevelsurveydata.TheBayesianestimateshavegoodstatisticalproperties,makegoodpredictions,andrealisticallyaccountforclusteringinthedata.StilltheBayesianestimatescanbebiasedinthepresenceofomittedvariablesandfixedeffectmodelsmightsometimesbepreferable.Bayesianstatisticsformodelcomparisonandevaluation—posteriorpredictivechecksandtheDe-vianceInformationCriterion—assistanempiricalapproachtodistinguishingbetweenhierarchicalmodelsandtheiralternatives.Theseideasareillus-tratedwithananalysisofmigrationdatafrom22villagesintheNangRongdistrictofThailand.Bayesianstatisticscanmakeaspecialcontributiontocomparativeandhistoricalsocialscience.Comparativedataareoftennotgeneratedbyawell-definedprobabilitymechanism,soaresearcher’suncertaintymaybebetterdescribedbyadegree-of-beliefthanthefrequencybehaviorofsamplestatis-tics(Berketal.1994).Comparativeandhistoricalresearchersalsounearthrichqualitativeinformationaboutparticularcountries,regions,andhistor-icalperiods.Inaclassicialanalysis,thisnonsampleinformationgenerallyprovidesinformalguidestomodelchoiceortheposthocinterpretationofresults.Bayesianpriordistributionsexplicitlyincorporatenon-sampleinfor-mationthatofteninfluencesdataanalysisinamoreinformalway(WesternandJackman1994).Priorinformationcanhavealargeeffectincomparativeanalysisbecausedatasetscanbesmallandcollinear.Undertheseconditions,thefinalresultsmayalsodependcloselyonthechoiceofmodels.Bayesianstatisticscanincorporateuncertaintyaboutthemodelspecification,pushinginferenceinamoreconservativedirection(Western1995).Finally,akeymessageofcomparativesocialscienceisthatsocialandpoliticalprocessesvaryacrosscountries,regions,andtimeperiods.Bayesianhierarchicalmod-elshelpusanalyzethesekindsofheterogeneity(Western1998;WesternandKleykamp2004).Thissymposiumontheanalysisofmultilevelsurveydataprovidesan-otheropportunitytoapplyBayesianmethodstothespecialmethodologicalproblemsofcomparativeresearch.Multilevelsurveydataarecollectedfrom,say,adozenormorecountries,perhapsatseveralpointsintime.Thisdatastructuresharessomefeatureswiththepooledtimeseriesfamiliartocompar-ativeresearchers—observationsareclusteredbycountryandthereislikelycausalheterogeneityacrosscountries.Butunlikeothercomparativedata,multilevelsurveydataprovideenoughinformationabouteachcountryto1conductacountry-levelanalysis.Inthiscase,thedata(withincountries,atleast)areusuallygeneratedbyprobabilitysamplingandpriorinformationwillbelessinfluentialbecausesamplesizeswithincountriesarelarge.Bayesianhierarchicalmodelsprovideausefulwaytostudytheseclus-tered,causallyheterogeneous,surveydata.ButBayesianmodelscanyieldbiasedestimateswhennon-Bayesianalternativesdobetter.TheBayesianmodelscanalsobedifficulttocomparetonon-Bayesianalternativesbecausemodelsarenon-nestedandnullhypotheseslieontheboundaryofparameterspaces.InthispaperwereviewtheBayesianhierarchicalmodelformultilevelsurveydata.WearguethatthemeritsofBayesianandnon-BayesianmodelsshouldbeassessedempiricallysowedescribesomeBayesianstatisticsformodelcomparisonandevaluation.Weillustratethesemethodsusingsurveydataonmigrationfrom22villagesinThailand.BayesianInferenceBayesianstatisticalinferencepoolssampledatawithnonsampleinformationtomakeposteriorprobabilitystatementsaboutstatisticalparameters.Givenasamplingdistributionforthedata,p(y|θ),andapriordistributionfortheparameters,p(θ),theposteriorinferencesaboutθgiventhedataaremadebyapplyingBayesrule:p(θ|y)∝p(y|θ)×p(θ)Ifyisnormallydistributedwithmeanθandvariance,σ2,andθhasanormalpriordistributionwithmeanθ0andvarianceτ2,thentheposteriordistribu-tionforθisalsonormal,wheretheposteriormeanistheweightedaverageofthesamplemean,¯y,andthepriormean,θ0,θ1=wθ0+(1−w)¯y.2Theweight,w,dependsontherelativesizeofthepriorvarianceandthevarianceofthesamplingdistributionfory,w=σ2/(τ2+σ2).Theposteriorvarianceofθisgivenby,V(θ|y)=1/(τ−2+σ−2).Ifthepriorvarianceislarge,indicatinggreatuncertaintyaboutthelocationofθbeforethedataareobserved,thenwwillbesmallandtheposteriormeanθ1willbenearthesamplemean¯y.Inpractice,thevarianceofthesamplingdistribution,σ2mustalsobeestimatedandgivenapriordistributioninaBayesiananalysis.Themainintuitionisunaffected:theposteriormeanisacompromisebetweenthepriorandsamplemeanthatdependsontherelativesizeofthepriorandsamplevariance.WhydoBayesianinference?Inthiscase,theposteriorvarianceissmallerthantheusualsamplingvariance.Thegaininprecisionisnotcostless,however.Theposteriormean,θ1,willgener-allybebiasedandthechoiceofpriordistributionissubjective,sodifferentresearchersmaychoosedifferentvaluesforθ0andτ2.BayesianInferenceforComparativeAnalysisHowcanthismachinerybeusedforcomparativeresearch?Ifwearein-ter