Alexander J Shackma-Multiple Regression 2

MultipleRegression2SolvingforandbTheweightforpredictorxjwillbeafunctionof:Thecorrelationxjandy.Theextenttowhichxj’srelationshipwithyisredundantwithotherpredictorsrelationshipswithy(collinearity).Thecorrelationsbetweenyandallotherpredictors.ThecorrelationsbetweenxjandallotherpredictorsSolvingforandb:thetwovariablecase212121211XXXXYXYXrrrr•1=slopeforX1controllingfortheotherindependentvariableX2•2iscomputedthesameway.SwapX1s,X2s•Comparetobivariateslope:Whathappenstob1ifX1andX2aretotallyuncorrelated?YXYXr111XYssb22110XXYSolvingforandb:thetwovariablecaseSolvingforandbisrelativelysimplewithtwovariablesbutbecomesincreasinglycomplexwithmorevariablesandrequiresdifferentialcalculustoderiveformulas.Matrixalgebracanbeusedtosimplifytheprocess.MatrixEquationsR2=S(ryjj)whereeachryj=correlationbetweentheDVandthejthIVeachi=standardizedregressioncoefficientR2=RyjBjwhereRyj=rowmatrixofcorrelationsbetweentheDVandkIVs.Bj=columnmatrixofstandardizedregressioncoefficientsfortheIVs.Bj=Rjj-1RjyInotherwords,thematrixofstandardizedregressioncoefficientsissimplythecorrelationmatrixbetweentheDVandIVsdividedbythematrixofcorrelationsamongtheIVs.TestsofRegressionCoefficientserrorstandardestimatedtcoefficienregression1withpNdfSbtjbj0*jbNullHypothesis:TheXjpredictorisnotrelatedtoYwhentheotherpredictorsareheldconstant.FurtherInterpretationofRegressionCoefficientsRegressioncoefficientsinmultipleregression(unstandardizedandstandardized)areconsideredpartialregressioncoefficientsbecauseeachcoefficientiscalculatedaftercontrollingfortheotherpredictorsinthemodel.Testsofregressioncoefficientsrepresentatestoftheuniquecontributionofthatvariableinpredictingyoverandaboveallotherpredictorvariablesinthemodel.Assumptions1.Predictorsarelinearlyrelatedtocriterion.2.Normalityoferrors--residualsarenormallydistributedaroundzero3.Multivariatenormaldistribution--multivariateextensionofbivariatenormality—homoscedasticity.RegressiondiagnosticscheckontheseassumptionsRegressionDiagnosticsDetectingmultivariateoutliersDistance,leverage,andinfluenceEvaluatingCollinearityRegressionDiagnosticsMethodsforidentifyingproblemsinyourmultipleregressionanalysis--agoodideaforanymultipleregressionanalysisCanhelpidentifyviolationofassumptionsoutliersandoverlyinfluentialcases—casesyoumightwanttodeleteortransformimportantvariablesyou’veomittedfromtheanalysisThreeClassesofMRDiagnosticStatistics1.Distance--detectsoutliersinthedependentvariableandassumptionviolations--primarymeasureistheresidual(Y-Y^)orstandardizedresidual(i.e.,putintermsofzscores)orstudentizedresidual(i.e.,putintermsoft-scores)2.Leverage--identifiespotentialoutliersintheindependentvariables--primarymeasureistheleveragestatisticor“hat”diagnosticThreeClassesofMRDiagnosticStatistics(cont.)3.Influence--combinesdistanceandleveragetoidentifyunusuallyinfluentialobservations(i.e.,observationsorcasesthathaveabiginfluenceontheMRequation)--themeasurewewilluseisCook’sDDistanceAnalyzeresidualsPayattentiontostandardizedorstudentizedresiduals2.5;shouldn’tbemorethan5%ofcasesTellsyouwhichcasesarenotpredictedwellbyregressionanalysis--youcanlearnfromthisinitselfNecessarytotestMRassumptionshomoscedasticitynormalityoferrorsDistanceUnstandardizedResidualsThedifferencebetweenanobservedvalueandthevaluepredictedbythemodel.Themeanis0.StandardizedResidualsTheresidualdividedbyanestimateofitsstandarderror.Standardizedresidualshaveameanof0andastandarddeviationof1.StudentizedResidualsTheresidualdividedbyanestimateofitsstandarddeviationthatvariesfromcasetocase,dependingontheleverageofeachcase’spredictorvaluesindeterminingmodelfit.Theyhaveameanof0andastandarddeviationslightlylargerthan1.DistanceDeletedResidualsTheresidualforacasethatisexcludedfromthecalculationoftheregressioncoefficients.Itisthedifferencebetweenthevalueofthedependentvariableandtheadjustedpredictedvalue.StudentizedDeletedResidualsItisastudentizedresidualwiththeeffectoftheobservationdeletedfromthestandarderror.Theresidualcanbelargeduetodistance,leverage,orinfluence.Themeanis0andthevarianceisslightlygreaterthan1.Distance-exampleOpenmregression1/example2c.savRegressproblemsonpeak,week,andindexUnderstatisticsselectestimates,covariancematrix,andmodelfit.Savepredictedvaluesunstandardizedandsaveallresiduals(unstandardized,standardized,Studentized,deleted,andStudentizeddeleted)OkayDistance-exampleoutputInterpretb’sandbetas.Comparebetaswithcorrelations.ZeroordercorrelationsValiditycoefficientsWhyisthestandarderrorofestimatedifferentfromthestandarddeviationofunstandardizedresiduals?Notecasewisediagnosticscomparedtosavedvalues.Leverage(“hat”diagnostic;hatdiag)TellsyououtliersonXvariablesNotethatthiscandetectso-calledmultivariateoutliers,thatis,casesthatarenotoutliersonanyoneXvariablebutareoutliersoncombinationsofXvariables.Example:Someonewhois60inchestallandweighs190pounds.Guideline:Payattentiontocaseswithcenteredleveragethatstandsoutorisgreaterthan2*k/nforlargesamplesor3*k/nforsmallsamples(.04inthiscase).(SPS