现代机器学习基于深度学习的图像特征提取

SARautoencoderconvolutionpooling1980ShallowLearningDeepLearning1980BackPropagationBPBPMulti-layerPerceptron90SVMSupportVectorMachinesBoostingLRLogisticRegressionSVMBoostingLR2000GoogleAdWordsCTRYahoo!2006GeoffreyHintonRuslanSalakhutdinov1.2.Layer-wisePre-training20062010DARPANECHubel-Wiesel2011GoogleDNN20~302012DNNImageNet2615DNNDrugeActivityGoogleGoogleDNNDNNGMM1.56102.1989YannLeCun()ConvolutionNeuralNetworksCNNCNNPoolingLaye5CNNHubel-WieselV1V2SimpleCellComplexCellCNNCNN201210GeoffreyHintonImageNetCNNHintondropoutGPU2012OCR2013+sparseautoencoder2.1sparseautoencoderDeeplearningsparseautoencodersparseautoencoderdeeplearningsparseautoencoder,sparseautoencoderyxy=xSparseautoencoder2.20sparseautoencoderKL0.050.050sigmoidKL0.05sigmoid-1+12.3-MatlabOctave.*123423sigmoid,1233.1autoencoder8*8patches10000sparseautoencoder36425641.zaazsigmoid2.zadelta3.azdelta12BPtrain.m110000patch204patchtrain.m23gradientchecking1gradientchecking6.5101e-111e-9L-BFGSW13.2Self-taughtlearningSelf-taughtlearningsparseautoencodersoftmaxregressionMNISTDataset5~9sparseautoencoder0~455~9softmaxregression98%96%3.3convolutionpooling96*9610010^6BPconvolutionconvolutionr*ca*bpatchsparseautoencoderkconvolution96*96,1008*8patch10010^310089*89*100=792100poolingpoolingconvolutionconvolutionconvolutionpoolingpoolingpoolingmaxpoolingaveragepoolingconvolutionpoolingconvolutionpoolingconvolutionpoolingpatcheswhiteningconvolutionwhiteningpoolingsoftmaxconvolutionconvolutionwhiteningconvolutionpoolingsoftmaxsoftmaxmatlabnmatlabconvolutioncnnConvolveconvolutionpatch(1000)patchconvutionpoolingcnnPoolpollingpoolingpoolingcnnPoolpoolingRGBconvolution3convolutionPooling4softmax1Accuracy:80.406%3.4deepnetwork2sparseautoencoderMINSTsoftmaxdeepnetwork1.sparseautoencoder12.1223.2softmaxsoftmax4.2softmax5.1232,1softmaxlbfssoftmaxsoftmax2sparseautoencodersoftmaxsoftmaxsoftmax1(Finetuning)200*784(b),200*200softmax200display_network20020*1016*25deeplearning200200196display_networkBeforeFinetuningTestAccuracy:92.190%AfterFinetuningTestAccuracy:97.670%4.1UniversalApproximationTheory4.24.3HadoopLatencyDNNSGDGPUDNNDNNGoogleDistBeliefGoogleGPUSGDDNN[1]BENGIOYLearningdeeparchitecturesforA1[J]FoundationsandTrendsinMachineLearning20092(1)1-124[2]D.Ciresan,U.Meier,J.Masci,andJ.Schmidhuber.Acommitteeofneuralnetworksfortrafficsignclassification.InNeuralNetworks(IJCNN),The2011InternationalJointConferenceon,pages19181921.IEEE,2011.10[3]HINTONGOSINDEROSTEHYAfastlearningalgorithmfordeepbeliefnets[J]NeuralComputation200618(7)1527-1554[4]Y.Boykov,O.Veksler,andR.Zabih.Fastapproximateenergyminimizationviagraphcuts.IEEETrans.PatternAnal.Mach.Intell.,23(11):12221239,2001.5[5]LECUNYBOTTOULBENGIOYeta1Gradientbasedlearningappliedtodocumentrecognition[J]ProceedingsoftheIEEE199886(11)2278-2324[6]D.Ciresan,U.Meier,J.Masci,andJ.Schmidhuber.Acommitteeofneuralnetworksfortrafficsignclassification.InNeuralNetworks(IJCNN),The2011InternationalJointConferenceon,pages19181921.IEEE,2011.10[7]C.Farabet,C.Couprie,L.Najman,andY.LeCun.Sceneparsingwithmultiscalefeaturelearning,puritytrees,andoptimalcovers.InProceedingsoftheInternationalConferenceonMachineLearning(ICML),June2012.2,6[8]J.CarreiraandC.Sminchisescu.CPMC:AutomaticObjectSegmentationUsingConstrainedParametricMin-Cuts.IEEETransactionsonPatternAnalysisandMachineIntelligence,2012.2[9]Y.BoykovandV.Kolmogorov.Anexperimentalcomparisonofmin-cut/max-flowalgorithmsforenergyminimizationinvision.IEEETrans.PatternAnal.Mach.Intell.,26(9):11241137,2004.5[10]Y.BoykovandM.P.Jolly.Interactivegraphcutsforoptimalboundary®ionsegmentationofobjectsinn-dimages.InProceedingsofInternationalConferenceofComputerVision(ICCV),volume1,pages105112,2001.11