Lecture 6 - Multiple Linear Regression and Multico

FNCE6290QuantitativeMethodsForFinanceYufengHan1Lecture6MultipleLinearRegressionandMulticollinearity1.MotivationInthelastlecture,wediscussedhowregressioncanhelpustoexplainhousepricesandfurthertopredicthouseprices.Weassumethatthesizeofthelivingarea(𝑠𝑞𝑓𝑡)determinesthehouseprices.Butitisclearthat𝑠𝑞𝑓𝑡isnottheonlydeterminingfactorofhouseprices;othervariablessuchasthenumberofbedroomsandbathscanalsoaffecttheprices.Itisintuitivelyclearthatbuyersusuallyprefermorebedroomstolessandmorebathstoless.Therefore,wewouldexpectapositivecorrelationbetweenthenumberofbedrooms(baths)andthehouseprice.Howdowealsoincorporatethesetwovariablesintotheregressionmodelwehadinthepreviouslecture?2.TheBasicModel2.1.ModelSetupRecallthesimplelinearregressionmodel:𝑌𝑖=𝛼+𝛽𝑋𝑖+𝑢𝑖Whatifseveralvariablesdetermine𝑌?𝑌𝑖=𝛽1+𝛽2𝑋2,𝑖+𝛽3𝑋3,𝑖+⋯+𝛽𝑘𝑋𝑘,𝑖+𝑢𝑖−𝑌againisthevariableunderinterestandthevariablewewanttoexplain.−Thechoiceofindependentvariables(the𝑋s)dependsoneconomic/financetheory.−𝑢𝑖hasthesameinterpretationasintheSimpleLinearRegressionModel.−𝛽1istheinterceptterm(thesameas𝛼intheSimpleLinearRegressionModel).−Therestofthe𝛽coefficientsarealsoknownas(partial)slopecoefficients.FNCE6290QuantitativeMethodsForFinanceYufengHan22.2.GeneralInterpretationoftheCoefficients2.2.1.IntercepttermTheinterceptiswhatweexpectthe𝑌variabletobeifall𝑋sareequalto0.Asdiscussedinthepreviouslecture,sometimesithasmeaningfulinterpretation,butmosttimes,itcapturestheaverageeffectofmodelmisspecification(e.g.,missingvariables).Therefore,wenormallywon’tfocusontheinterceptunlessithasagoodeconomicinterpretation(e.g.,theinterceptinFama-Frenchthree-factormodeldiscussedlater).2.2.2.SlopecoefficientsStatisticalInterpretation:Thesigntellsushowaspecific𝑋and𝑌arerelated.Supposethat𝛽20.Thenwecansaythat𝑋2and𝑌arepositivelycorrelated.Inotherwords,if𝑋2increases,𝑌willalsoincrease,andif𝑋2decreases,𝑌willdecrease.Ontheotherhand,if𝛽20,then𝑋2and𝑌moveinoppositedirections.Thesestatements,however,areonlytrueifwekeeptherestofthe𝑿variablesfixed-CeterisParibusInterpretation.EconomicInterpretation:Theeconomicinterpretationofeachslopecoefficient𝛽𝑗willdependontheactualproblem,butsomegeneralinterpretationexists.−Supposethat𝛽2ispositive.Thenwesaythat,keepingthevaluesofallother𝑿variablesthesame(allelseequal),oneunitincreasein𝑋2resultsin𝛽2unitsincreasein𝑌.−If𝛽2isnegativethenwesaythat,keepingthevaluesofallother𝑿variablesthesame(keepingeverythingelsefixed),oneunitincreasein𝑋2resultsin𝛽2unitsdecreasein𝑌.Youshouldknowtheunitsofmeasurementof𝑌and𝑋2inordertogivethecorrecteconomicinterpretationofthecoefficients.AsyoucanseethattheinterpretationsofthecoefficientsaresimilartotheonesintheSimpleLinearRegressionmodel.Butfortheslopecoefficients,theaddedclauseisthatweneedtofixothervariableswhenwechangeaspecific𝑋variable.Namely,weshouldonlychangeonevariableatatimetoisolatetheimpactofthatvariable.InotherFNCE6290QuantitativeMethodsForFinanceYufengHan3words,itiscriticallyimportanttofixallothervariablesconstantwhilechangethechosen𝑋variable.Otherwise,whenweincreasethechosen𝑋variable,let’ssay,𝑋2,othervariablesmayalsochangeasaresultpresumablybecausetheyarecorrelatedwith𝑋2.Sotheeffectof𝑋2inthiscaseisnotclear,i.e.,theeffectofX2onYiscompounded(orcontaminated)bythechangesofotherXvariablesandtheireffectsonY.3.OLSEstimationoftheMultipleRegressionModelWeneedtofindtheestimatesof𝛽1,𝛽2,𝛽3,⋯,𝛽𝑘.Assumethattheestimatesofthesecoefficientsare𝛽̂1,𝛽̂2,𝛽̂3,⋯,𝛽̂𝑘.Thefollowingrelationwillholdforeveryobservation.𝑌𝑖=𝛽̂1+𝛽̂2𝑋2.𝑖+𝛽̂3𝑋3.𝑖+⋯+𝛽̂𝑘𝑋𝑘.𝑖+𝑢̂𝑖AsintheSimpleLinearRegressionModel,thequestionis“Howdowefindthebestfit?”Andtheansweristhesame.Wewanttofind𝛽̂1,𝛽̂2,𝛽̂3,⋯,𝛽̂𝑘sothattheResidualSumofSquares(RSS)isminimized.min𝛽̂1,𝛽̂2,⋯,𝛽̂𝑘𝑅𝑆𝑆(𝛽̂1,𝛽̂2,⋯,𝛽̂𝑘)=min𝛽̂1,𝛽̂2,⋯,𝛽̂𝑘∑𝑢̂𝑖2𝑛𝑖=1=min𝛽̂1,𝛽̂2,⋯,𝛽̂𝑘∑(𝑌𝑖−𝛽̂1−𝛽̂2𝑋2𝑖−⋯−𝛽̂𝑘𝑋𝑘𝑖)2𝑛𝑖=13.1.1.NormalequationsInordertominimizetheResidualSumofSquare,weneedtosolve𝒌NormalEquations.Inthenormalequations,thereare𝑘unknownvariables-𝛽̂1,𝛽̂2,𝛽̂3,⋯,𝛽̂𝑘.3.1.2.SolutionstothenormalequationsSolving𝑘equationsforkunknownvariablesgivesustheOLSestimatesof𝛽1,𝛽2,⋯,𝛽𝑘.Theformulasofthesolutionsareirrelevanttousatthispoint.WewilluseStataorJMPorotherstatisticalsoftwaretoget𝛽̂1,𝛽̂2,𝛽̂3,⋯,𝛽̂𝑘.FNCE6290QuantitativeMethodsForFinanceYufengHan4Exercise1.HousePricesProblem𝑝𝑟𝑖𝑐𝑒=𝛽1+𝛽2𝑠𝑞𝑓𝑡+𝛽3𝑏𝑒𝑑𝑟𝑚𝑠+𝛽4𝑏𝑎𝑡ℎ𝑠+𝑢Theestimatedrelationbetweenpriceand𝑠𝑞𝑓𝑡,𝑏𝑒𝑑𝑟𝑚𝑠,and𝑏𝑎𝑡ℎ𝑠is:𝑝𝑟𝑖𝑐𝑒̂=129.06+0.15𝑠𝑞𝑓𝑡−21.59𝑏𝑒𝑑𝑟𝑚𝑠−12.19𝑏𝑎𝑡ℎ𝑠Questions:1.Comparetheinterceptandcoefficienton𝑠𝑞𝑓𝑡tothoseofthesimplelinearregression.Whyaretheydifferent?2.Whyarethecoefficientson𝑏𝑒𝑑𝑟𝑚𝑠and𝑏𝑎𝑡ℎ𝑠negative?3.Ifthelivingareaincreasesby100squarefeet,byhowmuchwillthehousepricegouponaveragekeepingeverythingelseconstant?4.Ifthenumberofbedroomsincreasesby1,byhowmuchwillthehousepricechangeonaveragekeepingeverythingelseconstant?5.Whatistheaveragepriceofahousewith1800squarefeet,4bedrooms,and2.75bath