逻辑回归-Logistic回归简介

PatternRecognition&MachineLearning模式识别与机器学习温雯[e-mail]温雯广东工业大学计算机学院2IntroductiontoLogisticRegressionSimpleandmultiplelinearregressionSimplelogisticregressionThelogisticfunctionEstimationofparameters温雯广东工业大学计算机学院3SimplelinearregressionAgeSBPAgeSBPAgeSBP22131411395212823128411715410524116461375614527106471115714128114481155815329123491335915730117491286315532122501836717633995113071172351215113377178401475114481217Table1Ageandsystolicbloodpressure(SBP)among33adultwomen温雯广东工业大学计算机学院4801001201401601802002202030405060708090SBP(mmHg)Age(years)adaptedfromColtonT.StatisticsinMedicine.Boston:LittleBrown,1974Age1.22281.54SBPSimplelinearregression温雯广东工业大学计算机学院5SimplelinearregressionRelationbetween2continuousvariables(SBPandage)Regressioncoefficientb1MeasuresassociationbetweenyandxAmountbywhichychangesonaveragewhenxchangesbyoneunitLeastsquaresmethodyxxβαy11Slope温雯广东工业大学计算机学院6MultiplelinearregressionRelationbetweenacontinuousvariableandasetoficontinuousvariablesPartialregressioncoefficientsbiAmountbywhichychangesonaveragewhenxichangesbyoneunitandalltheotherxisremainconstantMeasuresassociationbetweenxiandyadjustedforallotherxiExampleSBPversusage,weight,height,etcxβ...xβxβαyii2211温雯广东工业大学计算机学院7MultiplelinearregressionDependentIndependentvariablesPredictedPredictorvariablesResponsevariableExplanatoryvariablesOutcomevariableCovariablesxβ...xβxβαyii2211温雯广东工业大学计算机学院8LogisticregressionModelstherelationshipbetweenasetofvariablesxidichotomous(eat:yes/no)categorical(socialclass,...)continuous(age,...)anddichotomousvariableYDichotomous(binary)outcomemostcommonsituationinbiologyandepidemiology温雯广东工业大学计算机学院9Logisticregression(1)AgeCDAgeCDAgeCD220400540230411551240460581270470601280480600300491621300490651320501671330510711351511771380520811Table2Ageandsignsofcoronaryheartdisease(CD)温雯广东工业大学计算机学院10Howcanweanalysethesedata?Comparisonofthemeanageofdiseasedandnon-diseasedwomenNon-diseased:38.6yearsDiseased:58.7years(p0.0001)Linearregression?温雯广东工业大学计算机学院11Dot-plot:DatafromTable2AGE(years)SignsofcoronarydiseaseNoYes020406080100温雯广东工业大学计算机学院12Logisticregression(2)Table3Prevalence(%)ofsignsofCDaccordingtoagegroupDiseasedAgegroup#ingroup#%20-2950030-39611740-49722950-59745760-69548070-792210080-8911100温雯广东工业大学计算机学院13Dot-plot:DatafromTable302040608010002468Diseased%Age(years)温雯广东工业大学计算机学院14Thelogisticfunction(1)0.00.20.40.60.81.0Probabilityofdiseasexβxαβxαe1e)xP(y温雯广东工业大学计算机学院15ln()()PyxPyxx1Thelogisticfunction(2)logitofP(y|x){Pyxeexx()1温雯广东工业大学计算机学院16Thelogisticfunction(3)AdvantagesofthelogitSimpletransformationofP(y|x)LinearrelationshipwithxCanbecontinuous(Logitbetween-to+)Knownbinomialdistribution(Pbetween0and1)DirectlyrelatedtothenotionofoddsofdiseaseβxαP-1PlneP-1Pβxα温雯广东工业大学计算机学院17FittingequationtothedataLinearregression:LeastsquaresLogisticregression:MaximumlikelihoodLikelihoodfunctionEstimatesparametersaandbwithpropertythatlikelihood(probability)ofobserveddataishigherthanforanyothervaluesPracticallyeasiertoworkwithlog-likelihoodniiiiixyxylL1)(1ln)1()(ln)(ln)(温雯广东工业大学计算机学院18MaximumlikelihoodIterativecomputingChoiceofanarbitraryvalueforthecoefficients(usually0)Computingoflog-likelihoodVariationofcoefficients’valuesReiterationuntilmaximisation(plateau)ResultsMaximumLikelihoodEstimates(MLE)forandEstimatesofP(y)foragivenvalueofx温雯广东工业大学计算机学院19ReadingsLogistic回归模型