斯坦福大学机器学习课程讲义第四讲―― 多变量的线性回归模型表达

LinearRegressionwithmultiplevariablesMultiplefeaturesMachineLearningAndrewNgSize(feet2)Price($1000)210446014162321534315852178……Multiplefeatures(variables).AndrewNgSize(feet2)NumberofbedroomsNumberoffloorsAgeofhome(years)Price($1000)2104514546014163240232153432303158522136178……………Multiplefeatures(variables).Notation:=numberoffeatures=input(features)oftrainingexample.=valueoffeatureintrainingexample.AndrewNgHypothesis:Previously:AndrewNgForconvenienceofnotation,define.Multivariatelinearregression.LinearRegressionwithmultiplevariablesGradientdescentformultiplevariablesMachineLearningAndrewNgHypothesis:Costfunction:Parameters:(simultaneouslyupdateforevery)RepeatGradientdescent:AndrewNg(simultaneouslyupdate)GradientDescentRepeatPreviously(n=1):Newalgorithm:Repeat(simultaneouslyupdatefor)LinearRegressionwithmultiplevariablesGradientdescentinpracticeI:FeatureScalingMachineLearningAndrewNgE.g.=size(0-2000feet2)=numberofbedrooms(1-5)FeatureScalingIdea:Makesurefeaturesareonasimilarscale.size(feet2)numberofbedroomsAndrewNgFeatureScalingGeteveryfeatureintoapproximatelyarange.AndrewNgReplacewithtomakefeatureshaveapproximatelyzeromean(Donotapplyto).MeannormalizationE.g.LinearRegressionwithmultiplevariablesGradientdescentinpracticeII:LearningrateMachineLearningAndrewNgGradientdescent-“Debugging”:Howtomakesuregradientdescentisworkingcorrectly.-Howtochooselearningrate.AndrewNgExampleautomaticconvergencetest:Declareconvergenceifdecreasesbylessthaninoneiteration.0100200300400No.ofiterationsMakingsuregradientdescentisworkingcorrectly.AndrewNgMakingsuregradientdescentisworkingcorrectly.Gradientdescentnotworking.Usesmaller.No.ofiterationsNo.ofiterationsNo.ofiterations-Forsufficientlysmall,shoulddecreaseoneveryiteration.-Butifistoosmall,gradientdescentcanbeslowtoconverge.AndrewNgSummary:-Ifistoosmall:slowconvergence.-Ifistoolarge:maynotdecreaseoneveryiteration;maynotconverge.Tochoose,tryLinearRegressionwithmultiplevariablesFeaturesandpolynomialregressionMachineLearningAndrewNgHousingpricespredictionAndrewNgPolynomialregressionPrice(y)Size(x)AndrewNgChoiceoffeaturesPrice(y)Size(x)LinearRegressionwithmultiplevariablesNormalequationMachineLearningAndrewNgGradientDescentNormalequation:Methodtosolveforanalytically.AndrewNgIntuition:If1DSolvefor(forevery)AndrewNgSize(feet2)NumberofbedroomsNumberoffloorsAgeofhome(years)Price($1000)12104514546011416324023211534323031518522136178Size(feet2)NumberofbedroomsNumberoffloorsAgeofhome(years)Price($1000)2104514546014163240232153432303158522136178Examples:AndrewNgexamples;features.E.g.IfAndrewNgisinverseofmatrix.Octave:pinv(X’*X)*X’*yAndrewNgtrainingexamples,features.GradientDescentNormalEquation•Noneedtochoose.•Don’tneedtoiterate.•Needtochoose.•Needsmanyiterations.•Workswellevenwhenislarge.•Needtocompute•Slowifisverylarge.LinearRegressionwithmultiplevariablesNormalequationandnon-invertibility(optional)MachineLearningAndrewNgNormalequation-Whatifisnon-invertible?(singular/degenerate)-Octave:pinv(X’*X)*X’*yAndrewNgWhatifisnon-invertible?•Redundantfeatures(linearlydependent).E.g.sizeinfeet2sizeinm2•Toomanyfeatures(e.g.).-Deletesomefeatures,oruseregularization.