Chapter9 the central limit theorem

Chapter9CentralLimitTheorem9.1CentralLimitTheoremforBernoulliTrialsThesecondfundamentaltheoremofprobabilityistheCentralLimitTheorem.ThistheoremsaysthatifSnisthesumofnmutuallyindependentrandomvariables,thenthedistributionfunctionofSniswell-approximatedbyacertaintypeofcontinuousfunctionknownasanormaldensityfunction,whichisgivenbytheformulaf¹;¾(x)=1p2¼¾e¡(x¡¹)2=(2¾2);aswehaveseeninChapter4.3.Inthissection,wewilldealonlywiththecasethat¹=0and¾=1.Wewillcallthisparticularnormaldensityfunctionthestandardnormaldensity,andwewilldenoteitbyÁ(x):Á(x)=1p2¼e¡x2=2:AgraphofthisfunctionisgiveninFigure9.1.Itcanbeshownthattheareaunderanynormaldensityequals1.TheCentralLimitTheoremtellsus,quitegenerally,whathappenswhenwehavethesumofalargenumberofindependentrandomvariableseachofwhichcon-tributesasmallamounttothetotal.InthissectionweshalldiscussthistheoremasitappliestotheBernoullitrialsandinSection9.2weshallconsidermoregeneralprocesses.Wewilldiscussthetheoreminthecasethattheindividualrandomvari-ablesareidenticallydistributed,butthetheoremistrue,undercertainconditions,eveniftheindividualrandomvariableshavedi®erentdistributions.BernoulliTrialsConsideraBernoullitrialsprocesswithprobabilitypforsuccessoneachtrial.LetXi=1or0accordingastheithoutcomeisasuccessorfailure,andletSn=X1+X2+¢¢¢+Xn.ThenSnisthenumberofsuccessesinntrials.WeknowthatSnhasasitsdistributionthebinomialprobabilitiesb(n;p;j).InSection3.2,325326CHAPTER9.CENTRALLIMITTHEOREM-4-202400.10.20.30.4Figure9.1:Standardnormaldensity.weplottedthesedistributionsforp=:3andp=:5forvariousvaluesofn(seeFigure3.5).Wenotethatthemaximumvaluesofthedistributionsappearedneartheex-pectedvaluenp,whichcausestheirspikegraphstodrifto®totherightasnin-creased.Moreover,thesemaximumvaluesapproach0asnincreased,whichcausesthespikegraphsto°attenout.StandardizedSumsWecanpreventthedriftingofthesespikegraphsbysubtractingtheexpectednum-berofsuccessesnpfromSn,obtainingthenewrandomvariableSn¡np.Nowthemaximumvaluesofthedistributionswillalwaysbenear0.Topreventthespreadingofthesespikegraphs,wecannormalizeSn¡nptohavevariance1bydividingbyitsstandarddeviationpnpq(seeExercise6.2.12andEx-ercise6.2.16).De¯nition9.1ThestandardizedsumofSnisgivenbyS¤n=Sn¡nppnpq:S¤nalwayshasexpectedvalue0andvariance1.2SupposeweplotaspikegraphwiththespikesplacedatthepossiblevaluesofS¤n:x0,x1,...,xn,wherexj=j¡nppnpq:(9.1)Wemaketheheightofthespikeatxjequaltothedistributionvalueb(n;p;j).Anexampleofthisstandardizedspikegraph,withn=270andp=:3,isshowninFigure9.2.Thisgraphisbeautifullybell-shaped.Wewouldliketo¯tanormaldensitytothisspikegraph.Theobviouschoicetotryisthestandardnormaldensity,sinceitiscenteredat0,justasthestandardizedspikegraphis.Inthis¯gure,we9.1.BERNOULLITRIALS327-4-202400.10.20.30.4Figure9.2:Normalizedbinomialdistributionandstandardnormaldensity.havedrawnthisstandardnormaldensity.Thereaderwillnotethatahorriblethinghasoccurred:Eventhoughtheshapesofthetwographsarethesame,theheightsarequitedi®erent.Ifwewantthetwographsto¯teachother,wemustmodifyoneofthem;wechoosetomodifythespikegraph.Sincetheshapesofthetwographslookfairlyclose,wewillattempttomodifythespikegraphwithoutchangingitsshape.Thereasonforthedi®eringheightsisthatthesumoftheheightsofthespikesequals1,whiletheareaunderthestandardnormaldensityequals1.Ifweweretodrawacontinuouscurvethroughthetopofthespikes,and¯ndtheareaunderthiscurve,weseethatwewouldobtain,approximately,thesumoftheheightsofthespikesmultipliedbythedistancebetweenconsecutivespikes,whichwewillcall².Sincethesumoftheheightsofthespikesequalsone,theareaunderthiscurvewouldbeapproximately².Thus,tochangethespikegraphsothattheareaunderthiscurvehasvalue1,weneedonlymultiplytheheightsofthespikesby1=².ItiseasytoseefromEquation9.1that²=1pnpq:InFigure9.3weshowthestandardizedsumS¤nforn=270andp=:3,aftercorrectingtheheights,togetherwiththestandardnormaldensity.(This¯gurewasproducedwiththeprogramCLTBernoulliPlot.)Thereaderwillnotethatthestandardnormal¯tstheheight-correctedspikegraphextremelywell.Infact,oneversionoftheCentralLimitTheorem(seeTheorem9.1)saysthatasnincreases,thestandardnormaldensitywilldoanincreasinglybetterjobofapproximatingtheheight-correctedspikegraphscorrespondingtoaBernoullitrialsprocesswithnsummands.Letus¯xavaluexonthex-axisandletnbea¯xedpositiveinteger.Then,usingEquation9.1,thepointxjthatisclosesttoxhasasubscriptjgivenbythe328CHAPTER9.CENTRALLIMITTHEOREM-4-202400.10.20.30.4Figure9.3:Correctedspikegraphwithstandardnormaldensity.formulaj=hnp+xpnpqi;wherehaimeanstheintegernearesttoa.Thustheheightofthespikeabovexjwillbepnpqb(n;p;j)=pnpqb(n;p;hnp+xjpnpqi):Forlargen,wehaveseenthattheheightofthespikeisveryclosetotheheightofthenormaldensityatx.Thissuggeststhefollowingtheorem.Theorem9.1(CentralLimitTheoremforBinomialDistributions)Forthebinomialdistributionb(n;p;j)wehavelimn!1pnpqb(n;p;hnp+xpnpqi)=Á(x);whereÁ(x)isthestandardnormaldensity.TheproofofthistheoremcanbecarriedoutusingStirling'sapproximationfromSection3.1.Weindicatethismethodofproofbyconsideringthecasex=0.Inthiscase,thetheoremstatesthatlimn!1pnpqb(n;p;hnpi)=1p2¼=:3989::::