2019机器学习李宏毅Meta1-(v6)

MetaLearning(Part1)Hung-yiLeeIntroduction•Metalearning=LearntolearnLearningtask1Learningtask100Icanlearntask101betterbecauseIlearnsomelearningskillsLife-long:onemodelforallthetasksMeta:HowtolearnanewmodelLearningtask2……BeabetterlearnerTask1:speechrecognitionTask2:imagerecognitionTask100:textclassification…MetaLearningcatcatdogdogTrainingDataLearningAlgorithmDesignedbyDevelopers𝑓∗catTestingDataItisalsoafunction.𝐹𝐷𝑡𝑟𝑎𝑖𝑛𝑓∗=𝐹𝐷𝑡𝑟𝑎𝑖𝑛Canmachinefind𝐹fromdata?MetaLearningMachineLearning≈根據資料找一個函數f的能力MetaLearning≈根據資料找一個找一個函數f的函數F的能力catcatdogdogTrainingData𝐹=𝑓∗“Cat”𝑓=Step1:defineasetoffunctionStep2:goodnessoffunctionStep3:pickthebestfunctionMachineLearningisSimple就好像把大象放進冰箱……MetaLearningalgorithm𝐹Function𝑓MetaLearning•Defineasetoflearningalgorithm𝜃0𝜃1𝑔InitComputeGradientUpdateTrainingData𝜃2𝑔ComputeGradientUpdateTrainingData𝜃NetworkStructureLearningAlgorithm(Function𝐹)Whathappensintheredboxesisdecidedbyhumansuntilnow.Differentdecisionsintheredboxesleadtodifferentalgorithms.(limittogradientdescentbasedapproach)MetaLearning•Definingthegoodnessofafunction𝐹LearningAlgorithm𝐹catdogcatdogTask1𝑓1𝑙1TrainTestLearningAlgorithm𝐹Task2𝑓2𝑙2TrainTest𝐿𝐹=𝑙𝑛𝑁𝑛=1NtasksappleorangeappleorangeTestinglossfortasknaftertrainingMetaLearningMachineLearningTrainingTaskscatdogTraincatdogTestappleorangeappleorangecatdogcatdogbikecarbikecarTestingTasksTrainTestTrainTestTrainTestTask1Task2Widelyconsideredinfew-shotlearningSupportsetQuerysetSometimesyouneedvalidationtasksMetaLearning•Definingthegoodnessofafunction𝐹•Findthebestfunction𝐹∗𝐿𝐹=𝑙𝑛𝑁𝑛=1Testing:TaskNewLearningAlgorithm𝐹∗𝑓∗𝑙TrainTestbikecarbikecar𝐹∗=𝑎𝑟𝑔𝑚𝑖𝑛𝐹𝐿𝐹Omniglot•1623characters•Eachhas20examples–Few-shotClassification•N-waysK-shotclassification:Ineachtrainingandtesttasks,thereareNclasses,eachhasKexamples.•Splityourcharactersintotrainingandtestingcharacters•SampleNtrainingcharacters,sampleKexamplesfromeachsampledcharacters→onetrainingtask•SampleNtestingcharacters,sampleKexamplesfromeachsampledcharacters→onetestingtask20ways1shotEachcharacterrepresentsaclassTrainingset(Supportset)Testingset(Queryset)DemoofReptile:•MAML•ChelseaFinn,PieterAbbeel,andSergeyLevine,“Model-AgnosticMeta-LearningforFastAdaptationofDeepNetworks”,ICML,2017•Reptile•AlexNichol,JoshuaAchiam,JohnSchulman,OnFirst-OrderMeta-LearningAlgorithms,arXiv,2018TechniquesToday𝜃0𝜃1∇𝜃InitComputeGradientUpdateTrainingData𝜃2∇𝜃ComputeGradientUpdateTrainingData𝜃NetworkStructureLearningAlgorithm(Function𝐹)Onlyfocusoninitializationparameter𝐿𝜙=𝑙𝑛𝜃𝑛𝑁𝑛=1LossFunction:𝜃𝑛:modellearnedfromtask𝑛𝑙𝑛𝜃𝑛:lossoftask𝑛onthetestingsetoftask𝑛𝜃𝑛dependson𝜙MAML𝜙𝜙Howtominimize𝐿𝜙?𝜙←𝜙−𝜂∇𝜙𝐿𝜙GradientDescentModelPre-training𝐿𝜙=𝑙𝑛𝜙𝑁𝑛=1LossFunction:𝜃𝑛:modellearnedfromtask𝑛𝑙𝑛𝜃𝑛:lossoftask𝑛onthetestingsetoftask𝑛𝜃𝑛dependson𝜙𝐿𝜙=𝑙𝑛𝜃𝑛𝑁𝑛=1LossFunction:MAMLWidelyusedintransferlearningMAML𝜙ModelParameter𝑙1(Lossoftask1)𝑙2(Lossoftask2)𝐿𝜙=𝑙𝑛𝜃𝑛𝑁𝑛=1我們不在意𝜙在trainingtask上表現如何我們在意用𝜙訓練出來的𝜃𝑛表現如何𝜃1Small𝑙2𝜃2𝜃2Small𝑙1𝜃1𝑙1(Lossoftask1)𝑙2(Lossoftask2)ModelParameterModelPre-training𝐿𝜙=𝑙𝑛𝜙𝑁𝑛=1找尋在所有task都最好的𝜙𝜙並不保證拿𝜙去訓練以後會得到好的𝜃𝑛𝑙2𝜃2Howtominimize𝐿𝜙?𝜙←𝜙−𝜂∇𝜙𝐿𝜙GradientDescentModelPre-training𝐿𝜙=𝑙𝑛𝜙𝑁𝑛=1LossFunction:Find𝜙achievinggoodperformanceaftertraining潛力Find𝜙achievinggoodperformance現在表現如何𝜃𝑛:modellearnedfromtask𝑛𝑙𝑛𝜃𝑛:lossoftask𝑛onthetestingsetoftask𝑛𝜃𝑛dependson𝜙𝐿𝜙=𝑙𝑛𝜃𝑛𝑁𝑛=1LossFunction:MAMLWidelyusedintransferlearningMAML𝐿𝜙=𝑙𝑛𝜃𝑛𝑁𝑛=1𝜙←𝜙−𝜂∇𝜙𝐿𝜙𝜃＝𝜙－𝜀∇𝜙𝑙𝜙Consideringone-steptraining:𝜃0𝑔InitComputeGradientUpdateTrainingData𝜃NetworkStructureLearningAlgorithm(Function𝐹)Onlyfocusoninitializationparameter𝜙𝜙•Fast…Fast…Fast…•Goodtotrulytrainamodelwithonestep.•Whenusingthealgorithm,stillupdatemanytimes.•Few-shotlearninghaslimiteddata.ToyExampleEachtask:•Givenatargetsinefunction𝑦=𝑎𝑠𝑖𝑛𝑥+𝑏•SampleKpointsfromthetargetfunction•UsethesamplestoestimatethetargetfunctionSourceofimages𝑎and𝑏toformataskToyExampleModelPre-trainingMAMLSourceofimages://arxiv.org/abs/1703.03400WarningofMath∇𝜙𝑙𝜃=𝜕𝑙𝜃𝜕𝜙1𝜕𝑙𝜃𝜕𝜙2⋮𝜕𝑙𝜃𝜕𝜙𝑖⋮∇𝜙𝐿𝜙=∇𝜙𝑙𝑛𝜃𝑛𝑁𝑛=1=∇𝜙𝑙𝑛𝜃𝑛𝑁𝑛=1𝐿𝜙=𝑙𝑛𝜃𝑛𝑁𝑛=1𝜙←𝜙−𝜂∇𝜙𝐿𝜙𝜃＝𝜙－𝜀∇𝜙𝑙𝜙𝜙𝑖𝑙𝜃……𝜃1𝜃2𝜃𝑗𝜕𝑙𝜃𝜕𝜙𝑖=𝜕𝑙𝜃𝜕𝜃𝑗𝜕𝜃𝑗𝜕𝜙𝑖𝑗𝜕𝜃𝑗𝜕𝜙𝑖＝1－𝜀𝜕𝑙𝜙𝜕𝜙𝑖𝜕𝜙𝑗𝜕𝜃𝑗𝜕𝜙𝑖＝－𝜀𝜕𝑙𝜙𝜕𝜙𝑖𝜕𝜙𝑗≈1≈0∇𝜙𝐿𝜙=∇𝜙𝑙𝑛𝜃𝑛𝑁𝑛=1=∇𝜙𝑙𝑛𝜃𝑛𝑁𝑛=1𝐿𝜙=𝑙𝑛𝜃𝑛𝑁𝑛=1𝜙←𝜙−𝜂∇𝜙𝐿𝜙𝜃＝𝜙－𝜀∇𝜙𝑙𝜙𝜕𝑙𝜃𝜕𝜙𝑖=𝜕𝑙𝜃𝜕𝜃𝑗𝜕𝜃𝑗𝜕𝜙𝑖𝑗≈𝜕𝑙𝜃𝜕𝜃𝑖𝜃𝑗＝𝜙𝑗－𝜀𝜕𝑙𝜙𝜕𝜙𝑗𝑖≠𝑗:𝑖=𝑗:∇𝜙𝑙𝜃=𝜕𝑙𝜃𝜕𝜙1𝜕𝑙𝜃𝜕𝜙2⋮𝜕𝑙𝜃𝜕𝜙𝑖⋮∇𝜙𝐿𝜙=∇𝜙𝑙𝑛𝜃𝑛𝑁𝑛=1=∇𝜙𝑙𝑛𝜃𝑛𝑁𝑛=1𝐿𝜙=𝑙𝑛𝜃𝑛𝑁𝑛=1𝜙←𝜙−𝜂∇𝜙𝐿𝜙∇𝜙𝑙𝜃=𝜕𝑙𝜃𝜕𝜙1𝜕𝑙𝜃𝜕𝜙2⋮𝜕𝑙𝜃𝜕𝜙𝑖⋮𝜃＝𝜙－𝜀∇�