mcmc方法用于Bayesian推断-不错的ppt简明教程

AnIntroductiontoMCMCmethodsandBayesianStatisticsWhatwillwecoverinthisfirstsession?•WhatisBayesianStatistics?(asopposedtoclassicalorfrequentiststatistics)•WhatisMCMCestimation?•MCMCalgorithmsandGibbsSampling•MCMCdiagnostics•MCMCModelcomparisonsWHATISBAYESIANSTATISTICS?WhydoweneedtoknowaboutBayesianstatistics?•TherestofthisworkshopisprimarilyaboutMCMCmethodswhichareafamilyofestimationmethodsusedforfittingrealisticallycomplexmodels.•MCMCmethodsaregenerallyusedonBayesianmodelswhichhavesubtledifferencestomorestandardmodels.•AsmoststatisticalcoursesarestilltaughtusingclassicalorfrequentistmethodsweneedtodescribethedifferencesbeforegoingontoconsiderMCMCmethods.BayesTheoremBayesianstatisticsnamedafterRev.ThomasBayes(1702-1761)BayesTheoremforprobabilityeventsAandBOrforasetofmutuallyexclusiveandexhaustiveevents(i.e.):)()()|()|(BpApABpBAp=iiiiApAp∑==1)()(∑=jjjiiiAPABpApABpBAp)()|()()|()|(Example–cointossingLetAbetheeventof2Headsinthreetossesofafaircoin.Bbetheeventof1stcoinisaHead.Threecoinshave8equallyprobablepatterns{HHH,HHT,HTH,HTT,THH,THT,TTH,TTT}A={HHT,HTH,THH}→p(A)=3/8B={HHH,HTH,HTH,HTT}→p(B)=1/2A|B={HHT,HTH}|{HHH,HTH,HTH,HTT}→p(A|B)=1/2B|A={HHT,HTH}|{HHT,HTH,THH}→p(B|A)=2/3P(A|B)=P(B|A)P(A)/P(B)=(2/3*3/8)/(1/2)=1/2Example2–DiagnostictestingAnewHIVtestisclaimedtohave“95%sensitivityand98%specificity”InapopulationwithanHIVprevalenceof1/1000,whatisthechancethatapatienttestingpositiveactuallyhasHIV?LetAbetheeventpatientistrulypositive,A’betheeventthattheyaretrulynegativeLetBbetheeventthattheytestpositiveDiagnosticTestingcontinued:Wewantp(A|B)“95%sensitivity”meansthatp(B|A)=0.95“98%specificity”meansthatp(B|A’)=0.02SofromBayesTheorem:045.0999.002.0001.095.0001.095.0)'()'|()()|()()|()|(=×+××=+=ApABpApABpApABpBApThusover95%ofthosetestingpositivewill,infact,nothaveHIV.BeingBayesian!SothevitalissueinthisexampleishowshouldthistestresultchangeourpriorbeliefthatthepatientisHIVpositive?Thediseaseprevalence(p=0.001)canbethoughtofasa‘prior’probability.Observingapositiveresultcausesustomodifythisprobabilitytop=0.045whichisour‘posterior’probabilitythatthepatientisHIVpositive.ThisuseofBayestheoremappliedtoobservablesisuncontroversialhoweveritsuseingeneralstatisticalanalyseswhereparametersareunknownquantitiesismorecontroversial.BayesianInferenceInBayesianinferencethereisafundamentaldistinctionbetween•Observablequantitiesx,i.e.thedata•Unknownquantitiesθθcanbestatisticalparameters,missingdata,latentvariables…•ParametersaretreatedasrandomvariablesIntheBayesianframeworkwemakeprobabilitystatementsaboutmodelparametersInthefrequentistframework,parametersarefixednon-randomquantitiesandtheprobabilitystatementsconcernthedata.PriordistributionsAswithallstatisticalanalyseswestartbypositingamodelwhichspecifiesp(x|θ)Thisisthelikelihoodwhichrelatesallvariablesintoa‘fullprobabilitymodel’HoweverfromaBayesianpointofview:•θisunknownsoshouldhaveaprobabilitydistributionreflectingouruncertaintyaboutitbeforeseeingthedata•Thereforewespecifyapriordistributionp(θ)NotethisisliketheprevalenceintheexamplePosteriorDistributionsAlsoxisknownsoshouldbeconditionedonandhereweuseBayestheoremtoobtaintheconditionaldistributionforunobservedquantitiesgiventhedatawhichisknownastheposteriordistribution.Thepriordistributionexpressesouruncertaintyaboutθbeforeseeingthedata.Theposteriordistributionexpressesouruncertaintyaboutθafterseeingthedata.)|()()|()()|()()|(θθθθθθθθxppdxppxppxp∝=∫ExamplesofBayesianInferenceusingtheNormaldistributionKnownvariance,unknownmeanItiseasiertoconsiderfirstamodelwith1unknownparameter.SupposewehaveasampleofNormaldata:Letusassumeweknowthevariance,σ2andweassumeapriordistributionforthemean,µbasedonourpriorbeliefs:Nowwewishtoconstructtheposteriordistributionp(µ|x)..,...,1,),(~2niNxi=σµ),(~200σµµNPosteriorforNormaldistributionmeanSowehave))//()//1(exp()/)(exp()2()/)(exp()2()|()()|(henceand)/)(exp()2()|()/)(exp()2()(220022022121221220202120222122020212021212121consxnxxppxpxxppiiNiiii++++−∝−−×−−==−−=−−=∑∏=−−−−σσµµσσµσµπσσµµπσµµµσµπσµσµµπσµPosteriorforNormaldistributionmean(continued)ForaNormaldistributionwithresponseywithmeanθandvarianceφwehave}/exp{}/)(exp{)2()(122122121consyyyyf++−∝−−=−−φθφφθπφWecanequatethistoourposteriorasfollows:)//(and)//1())//()//1(exp(220012202200220221σσµφθσσφσσµµσσµ∑∑+=+=→++++−∝−iiiixnconsxnPrecisionsandmeansInBayesianstatisticstheprecision=1/varianceisoftenmoreimportantthanthevariance.FortheNormalmodelwehave))//(/(and)//1(/12200220nxnσσµφθσσφ+=+=Inotherwordstheposteriorprecision=sumofpriorprecisionanddataprecision,andtheposteriormeanisa(precisionweighted)averageofthepriormeananddatamean.LargesamplepropertiesAsn→∞PosteriorprecisionSoposteriorvariancePosteriormeanAndsoposteriordistributionComparedtointhefrequentistsetting2220/)//1(/1σσσφnn→+=n/2σ→xnx→+=))//(/(2200σσµφθ)/,()|(2nxNxpσµ→)/,()|(2nNxpσµµ=GirlsHeightsExample10girlsaged18hadboththeirheightsandweightsmeasured.Theirheights(incm)wereasfollows:169.6,166.8,157.1,181.1,158.4,165.6,166.7,156.5,168.1,165.3Wewillassumethepopulationvarianceisknowntobe50.Twoindividualsgavethefollo