多元线性回归模型：估计

1MultipleRegressionAnalysisy=b0+b1x1+b2x2+...bkxk+u1.Estimation2ParallelswithSimpleRegressionb0isstilltheinterceptb1tobkallcalledslopeparametersuisstilltheerrorterm(ordisturbance)Stillneedtomakeazeroconditionalmeanassumption,sonowassumethatE(u|x1,x2,…,xk)=0Stillminimizingthesumofsquaredresiduals,sohavek+1firstorderconditions3theOLSregressionlineorthesampleregressionfunction(SRF).istheOLSinterceptestimateandaretheOLSslopeestimates.Stilluseordinaryleastsquarestogettheestimates:nixxxyikkiii,,1ˆˆˆˆˆ22110bbbb0ˆbkbbbˆ,,ˆ,ˆ212122110ˆˆˆˆminniikkiiixxxybbbb4OLSFirstOrderConditionsThisminimizationproblemcanbesolvedusingmultivariablecalculas.Thisleadstok+1linearequationink+1unknown:niikkiiixxxy1221100ˆˆˆˆbbbbniikkiiiixxxyx12211010ˆˆˆˆbbbbniikkiiiixxxyx12211020ˆˆˆˆbbbbniikkiiiikxxxyx1221100ˆˆˆˆbbbb……5AFittedorPredictedValueForobservationi,thefittedvalueisTheresidualforobservationiisdefinedasinthesimpleregressioncase,Theproperties123Thepoint()isalwaysontheOLSregressionline:ikkiiixxxybbbbˆˆˆˆˆ22110iiiyyuˆˆkkixxxybbbbˆˆˆˆ22110yxxxk,,,,210ˆiu0ˆˆ0ˆiikiyuxu6InterpretingMultipleRegressiontioninterpretaahaseachisthat,ˆˆthatimpliesfixed,...,holdingso,ˆ...ˆˆˆso,ˆ...ˆˆˆˆ112221122110ribusceterispaxyxxxxxyxxxykkkkkbbbbbbbbb7A“PartiallingOut”Interpretation22011211122110ˆˆˆˆregressionestimatedthefromresidualstheareˆwhere,ˆˆˆthen,ˆˆˆˆi.e.,2wherecaseheConsidertxxrryrxxykiiiibbbb8“PartiallingOut”continuedPreviousequationimpliesthatregressingyonx1andx2givessameeffectofx1asregressingyonresidualsfromaregressionofx1onx2Thismeansonlythepartofxi1thatisuncorrelatedwithxi2arebeingrelatedtoyisowe’reestimatingtheeffectofx1onyafterx2hasbeen“partialledout”9SimplevsMultipleRegEstimatesampleintheeduncorrelatareandOR)ofeffectpartialno(i.e.0ˆ:unlessˆ~Generally,ˆˆˆˆregressionmultiplewiththe~~~regressionsimpletheCompare21221122110110xxxxxyxybbbbbbbb10Goodness-of-FitSSRSSESSTThen(SSR)squaresofsumresidualtheisˆ(SSE)squaresofsumexplainedtheisˆ(SST)squaresofsumtotaltheis:followingthedefinethenWeˆˆpart,dunexplaineanandpart,explainedanofupmadebeingasnobservatioeachofcanthinkWe222iiiiiiuyyyyuyy11Goodness-of-Fit(continued)Howdowethinkabouthowwelloursampleregressionlinefitsoursampledata?Cancomputethefractionofthetotalsumofsquares(SST)thatisexplainedbythemodel,callthistheR-squaredofregressionR2=SSE/SST=1–SSR/SST12Goodness-of-Fit(continued)22222ˆˆˆˆˆvaluestheandactualthebetweentcoefficienncorrelatiosquaredthetoequalbeingasofthinkalsocanWeyyyyyyyyRyyRiiiiii13MoreaboutR-squaredR2canneverdecreasewhenanotherindependentvariableisaddedtoaregression,andusuallywillincreaseBecauseR2willusuallyincreasewiththenumberofindependentvariables,itisnotagoodwaytocomparemodels14AssumptionsforUnbiasednessPopulationmodelislinearinparameters:y=b0+b1x1+b2x2+…+bkxk+uWecanusearandomsampleofsizen,{(xi1,xi2,…,xik,yi):i=1,2,…,n},fromthepopulationmodel,sothatthesamplemodelisyi=b0+b1xi1+b2xi2+…+bkxik+uiE(u|x1,x2,…xk)=0,implyingthatalloftheexplanatoryvariablesareexogenousNoneofthex’sisconstant,andtherearenoexactlinearrelationshipsamongthem15TooManyorTooFewVariablesWhathappensifweincludevariablesinourspecificationthatdon’tbelong?Thereisnoeffectonourparameterestimate,andOLSremainsunbiasedWhatifweexcludeavariablefromourspecificationthatdoesbelong?OLSwillusuallybebiased16OmittedVariableBias21111111022110~then,~~~estimatebutwe,asgivenismodeltruetheSupposexxyxxuxyuxxyiiibbbbbb17OmittedVariableBias(cont)iiiiiiiiiiiiiuxxxxxxxuxxxxuxxy1121122111221101122110becomesnumeratortheso,thatsomodel,truetheRecallbbbbbbbb18OmittedVariableBias(cont)2112112112111121121121~havewensexpectatiotaking0,)E(since~xxxxxEuxxuxxxxxxxiiiiiiiiiibbbbbb19OmittedVariableBias(cont)12112112111110212~~so~then~~~onofregressionheConsidertbbbExxxxxxxxxiii20SummaryofDirectionofBiasCorr(x1,x2)0Corr(x1,x2)0b20PositivebiasNegativebiasb20NegativebiasPositivebias21OmittedVariableBiasSummaryTwocaseswherebiasisequaltozerob2=0,thatisx2doesn’treallybelonginmodelx1andx2areuncorrelatedinthesampleIfcorrelationbetweenx2,x1andx2,yisthesamedirection,biaswillbepositiveIfcorrelationbetweenx2,x1andx2,yistheoppositedirection,biaswillbenegative22TheMoreGeneralCaseTechnically,canonlysignthebiasforthemoregeneralcaseifalloftheincludedx’sareuncorrelatedTypically,then,weworkthroughthebiasassumingthex’sareuncorrelated,asausefulguideevenifthisassumptionisnotstrictlytrue23VarianceoftheOLSEstimatorsNowweknowthatthesamplingdistributionofourestimateiscenteredaroundthetrueparameterWanttothinkabouthowspreadoutthisdistributionisMucheasiertothinkaboutthisvarianceunderanadditionalassumption,soAssumeVar(u|x1,x2,…,xk)=s2(Homoskedasticity)24VarianceofOLS(cont)Letxstandfor(x1,x2,…xk)AssumingthatVar(u|x)=s2alsoimpliesthatVar(y|x)=s2The4assumptionsforunbiasedness,plusthishomoskedasticityassumptionareknownastheGauss-Markovassu