机器学习讲座

MachineLearninganditsapplicationRelationshipsamongAI，ML，DL人工智能、机器学习、深度学习的关系人工智能：机器展现的人类智能机器学习：实现人工智能的一种方法深度学习：实现机器学习的一种技术OutlineIntroductionofMachineLearningWhyDeep?Howtolearnit？ApplicationofdeeplearningProducts•BaiduEye•百度识图•GoogleGlass•AppleSiri•微软小冰Products（NLP）智能对话、百科、天气、星座、笑话、交通指南、餐饮点评等JIMI智能机器人售前咨询售后服务生活伴侣场景用户画像•提供个性化的产品服务人口属性：地域、年龄、性别、文化、职业、收入、生活习惯、消费习惯等产品行为：产品类别、活跃频率、产品喜好、产品驱动、使用习惯、产品消费等•京东的JIMI智能机器人——DNNLab首席科学家李成华：“用深度学习搞定80%的客服工作。”Products（NLP）Handwritingrecognition（LeNet-5）于1989年提出的CNN原型，成功应用于欧洲很多国家的手写支票识别。LandCoverClassification(f)SSAE(g)SSCNN(h)PSSCNN(i)GCNN(j)MSCNN(a)GroundTruth(b)SVM(c)AE(d)CNN(e)EPFDeepDream•Givenaphoto,machineaddswhatitsees……•Givenaphoto,machineaddswhatitsees……•Givenaphoto,makeitsstylelikefamouspaintings•Givenaphoto,makeitsstylelikefamouspaintings？ApplicationofdeeplearningMachineLearning≈LookingforaFunction•SpeechRecognition•PlayingGo•DialogueSystemf•ImageRecognitionfff“Howareyou”“Cat”“Hello”“Hi”(whattheusersaid)(systemresponse)“5-5”(nextmove)f1f1“cat”“dog”f2f2“money”“snake”FrameworkModelAsetoffunctionf1,f2f“cat”ImageRecognition:“cat”ImageRecognition:FrameworkModelAsetoffunctionf1,f2TrainingDatafBetter!“cat”“dog”functioninput:functionoutput:“monkey”GoodnessoffunctionfSupervisedLearningFrameworkAsetoffunctionf1,f2f“cat”ImageRecognition:ModelTrainingData“monkey”“cat”“dog”Usingf“cat”TrainingTestingStep1GoodnessoffunctionfStep2Pickthe“Best”Functionf*Step3Step1:defineasetoffunctionStep2:goodnessoffunctionStep3:pickthebestfunctionThreeStepsforDeepLearningDeepLearningissosimple……NeuralStep1:defineasetfunctionofNetworkStep2:goodnessoffunctionStep3:pickthebestfunctionThreeStepsforDeepLearningDeepLearningissosimple……HumanBrainsPlayGround的网址是：…………w1a1akaKbawkwKweightsNeuralNetworkNeuronza1w1akwkaKwKbAsimplefunctionzzActivationfunctionbiasNeuralNetworkzActivationfunctionbiasNeuron1-2-1weights12-114zzz11ezSigmoidFunction0.98zzzzNeuralNetworkDifferentconnectionsleadstodifferentnetworkstructureEachneuronscanhavedifferentvaluesofweightsandbiases.Weightsandbiasesarenetworkparameters𝜃FullyConnectFeedforwardNetworkzzz11ezSigmoidFunction1-11-2-1114-200.980.12FullyConnectFeedforwardNetwork1-21-1104-20.120.982-1-1-2-14-10.8630.110.620.8300-221-1FullyConnectFeedforwardNetwork1-2-11100.50.732-1-2-13-1-140.720.120.510.8500-22𝑓00=0.510.85𝑓1−1=0.620.8300Thisisafunction.Inputvector,outputvectorGivenparameters𝜃,defineafunctionGivennetworkstructure,defineafunctionset…………………………OutputLayerHiddenLayersInputLayerFullyConnectFeedforwardNetworkLayer1Inputx1x2xNLayer2LayerL………………Outputy1y2yMDeepmeansmanyhiddenlayersneuronOutputLayer(Option)•SoftmaxlayerastheoutputlayerOrdinaryLayery1z1y2z2y3z3z1z2z3Ingeneral,theoutputofnetworkcanbeanyvalue.Maynotbeeasytointerprety1eez2z22.70.05eeeez1•SoftmaxlayerastheoutputlayerSoftmaxLayereeez1ee3j1z1zjz33j1zj31z3-3200.88OutputLayer(Option)Probability:1𝑦𝑖0𝑖𝑦𝑖=13j13j1z2z3zjzj0.12y2e≈0y3e………………ExampleApplicationInputOutput16x16=256x1x2x256Ink→1Noink→0yy2y10Eachdimensionrepresentstheconfidenceofadigit.is1is2is00.10.70.2Theimageis“2”………………MachineExampleApplication•HandwritingDigitRecognitionx1x2x256y1y2“2”y10is1is2is0function……Input:256-dimvectoroutput:10-dimvectorNeuralNetworkWhatisneededisa………………………………ExampleApplicationInputOutputLayer1x1x2xNInputLayerLayer2LayerL……y1y2“2”y10is1is2is0……AfunctionsetcontainingthecandidatesforHandwritingDigitRecognition……OutputHiddenLayersLayerYouneedtodecidethenetworkstructuretoletagoodfunctioninyourfunctionset.FAQ•Q:Howmanylayers?Howmanyneuronsforeachlayer?•Q:Canthestructurebeautomaticallydetermined?TrialandErrorIntuition+NeuralofNetworkStep1:defineasetStep2:goodnessoffunctionStep3:pickthebestfunctionThreeStepsforDeepLearningDeepLearningissosimple……TrainingData•Preparingtrainingdata:imagesandtheirlabelsThelearningtargetisdefinedonthetrainingdata.“1”“3”“4”“1”“0”“2”“5”“9”Softmax………………LearningTarget16x16=256x1x2x256………………Ink→1Noink→0y1y2y10Thelearningtargetis……y1hasthemaximumvaluey2hasthemaximumvalueInput:Input:is1is2is0…………………………GivenasetofLossx1x2xN………………y2y10Loss𝑙00Losscanbethedistancebetweenthenetworkoutputandtargettargety1Ascloseas1possibleAgoodfunctionshouldmakethelossofallexamplesassmallaspossible.“1”parameters……………………TotalLossxRNNyR𝑦𝑅x1x2x3NNNNNNy1y2y3𝑦1𝑦2𝑦3Foralltrainingdata…𝐿=𝑟=1𝑅𝑙𝑟Findthenetworkparameters𝜽∗thatminimizetotallossLTotalLoss:𝑙1𝑙2𝑙3𝑙𝑅AssmallaspossibleFindafunctioninfunctionsetthatminimizestotallossLNeuralofNetworkStep1:defineasetfunctionStep2:goodnessoffunctionStep3:pickthebestfunctionThreeStepsforDeepLearningDeepLearningissosimple……GradientDescentTotalLoss𝐿Networkparameters