基于迁移学习的深层卷积神经网络人脸属性分类的岛屿损失研究(IJIGSP-V12-N1-3)

I.J.Image,GraphicsandSignalProcessing,2020,1,18-29PublishedOnlineFebruary2020inMECS()DOI:10.5815/ijigsp.2020.01.03Copyright©2020MECSI.J.Image,GraphicsandSignalProcessing,2020,1,18-29IslandLossforImprovingtheClassificationofFacialAttributeswithTransferLearningonDeepConvolutionalNeuralNetworkShuvenduRoyDepartmentofComputerScienceandEngineering,KhulnaUniversityofEngineering&Technology,BangladeshEmail:[e-mail]:09August2019;Accepted:27August2019;Published:08February2020Abstract—Classificationtaskonthehumanfacialattributeishardbecauseofthesimilaritiesinbetweenclasses.Forexample,emotionclassificationandageestimationaretwoimportantapplications.Thereareverylittlechangesbetweenthedifferentemotionsofapersonandadifferentpersonhasadifferentwayofexpressingthesameemotion.Sameforageclassification.Thereislittledifferencebetweenconsecutiveages.Anotherproblemistheimageresolution.Smallimagescontainlessinformationandlargeimagerequiresalargemodelandlotsofdatatotrainproperly.Tosolvebothoftheseproblemsthisworkproposesusingtransferlearningonapre-trainedmodelcombiningacustomlossfunctioncalledIslandLosstoreducetheintra-classvariationandincreasetheinter-classvariation.Theexperimentshaveshownimpressiveresultsonbothoftheapplicationwiththismethodandachievedhigheraccuraciescomparedtopreviousmethodsonseveralbenchmarkdatasets.IndexTerms—IslandLoss,Transferlearning,Facialattributeclassification,CNN.I.INTRODUCTIONThehumanfacecanprovidelotsofinformationlikehisageoremotionalstate.So,theunderstandingfacialfeatureisanimportanttaskinmachineintelligence.Muchofthesetaskscanbetreatedasaclassificationtaskbutclassificationonfacialfeatureisharderthanotherclassificationinmostcases.Thisisbecausethereisalotofsimilaritiesindifferentclassesofthiskindofclassificationtask.Forexample,differentpeoplehavedifferentwaysofexpressingthesameemotionandthedifferencebetweensomeemotionsareverylittle.Similarly,peopleofdifferentgeographicareaorlifestylehasadifferentagingeffectonthefaceofthesameage.Expressionofemotionisimportantinsocialcommunication.Humanexpressesemotionthroughfacialexpression,speech,andbodymovement.Emotionrecognitionhasmanycommercialusesatconsumerlevelproductssuchasunderstandingusers’satisfactionformovieorservice.Manyotherreal-worldapplicationswillbebenefitedfromthisapplicationlikeeffect-awaregamedevelopmentandcallcenter.Thisisachallengingproblembecauseoftheverynatureofthehumanface.Themainreasonforthiscomplexityincludestheoverlappingoffacialexpressionindifferentpeopleanddifferentpeoplehaveadifferentwayofexpressingemotions.Emotionrecognitionisextensivelystudiedduetoitsnumerouspracticalapplication.Fromclassicalfeaturedetectiontorecentdeeplearningtechniques.Anytypeofaudio-visualsignal,e.g.image,video,text,speech,andbiosignalscanbeusedasinputtotheemotiondetectionsystem.However,visionisthemostimportantandmostmeaningfulsourceofinformationforunderstandinghumanemotion.Inthecaseofvision-basedemotionrecognitionsystemfactorslikethehumanfacialpose,contextandactionprovidevaluableinsight.Mostofthedatasets[1,2,3,4,5,6,7,8]forhumanfacialemotionrecognitionsystemiscollectedinlabconditionwherethesubjectsaretoldtocreateartificialemotionalwhichisfarfromnatural.So,whenitcomestothefactofunderstandingemotioninareal-lifesituation,thetaskbecomesverydifficultandtheaccuracydropsdrastically.Understandingemotioninanuncontrolledenvironmentstillremainsachallengeforresearchersduetofactssuchaslow-resolutionocclusion,cultureandagedifference.Thefacialemotionrecognitiontaskcanbedividedintotwocategoriesdependingonthesource:staticimageandvideosequences.Forsequence-basedstrategies,thesequenceofframeswithotherobjectandeventscanprovideusefulstrategiesforfindingfacialhighlights.Nonetheless,facialemotiondetectionfromastaticimageismorechallengingthanthatofvideoasthereisnoextrainformationavailable.However,insequence-basedmodels,anaturalfaceisusedasabaseline.Whichhastobedeterminedfirst.Atthatpoint,therecognitionisbasedonthebaselineface.Therefore,thedeterminationofthebaselineimageisveryimportant.Ifthebaselinefaceisnotdeterminedproperly,therecognitionwillnotbeaccurate.Amongtheearliersuccessfulmethods,appearance-basedapproacheswereverypopular.Thesemethodsarebasedonthelineardiscriminantanalysis(LDA)whichfindspatternsindata.ButtheseencountersdifficultiesIslandLossforImprovingtheClassificationofFacialAttributeswithTransferLearning19onDeepConvolutionalNeuralNetworkCopyright©2020MECSI.J.Image,GraphicsandSignalProcessing,2020,1,18-29Fig.1.ThearchitectureofaConvolutionalNeuralNetwork.Theexampleshowstheprocessingofa28×28sizeimage.IttakesanimageasinputandapplyconvolutionandMaxPollingoftheimageandfinallyconvertitintoonedimensionalvector.Fig.2.StepsoftrainingusingtheTransfermethod.Firstthemodelispre-trainedwithstandardlaegedatasettogetthepre-trainedstage.Thisisthenfine-tunedontaskspecificdata.withhighdimensionaldata.Severalmethodshavebeenproposedtosolvethehighdimensionalissues.Anovelregularizedmethodwasproposedin[9]tosolvethehighdimensionalproblem.Also,fuzzylogicwasi