KazemiCVPR14(图像换脸论文)

OneMillisecondFaceAlignmentwithanEnsembleofRegressionTreesVahidKazemiandJosephineSullivanKTH,RoyalInstituteofTechnologyComputerVisionandActivePerceptionLabTeknikringen14,Stockholm,Swedenfvahidk,[e-mail]ssestheproblemofFaceAlignmentforasingleimage.Weshowhowanensembleofregressiontreescanbeusedtoestimatetheface’slandmarkpositionsdirectlyfromasparsesubsetofpixelintensities,achievingsuper-realtimeperformancewithhighqualitypredictions.Wepresentageneralframeworkbasedongradientboostingforlearninganensembleofregressiontreesthatoptimizesthesumofsquareerrorlossandnaturallyhandlesmissingorpartiallylabelleddata.Weshowhowusingappropriatepriorsexploitingthestructureofimagedatahelpswithef-ficientfeatureselection.Differentregularizationstrategiesanditsimportancetocombatoverfittingarealsoinvesti-gated.Inaddition,weanalysetheeffectofthequantityoftrainingdataontheaccuracyofthepredictionsandexploretheeffectofdataaugmentationusingsynthesizeddata.1.IntroductionInthispaperwepresentanewalgorithmthatperformsfacealignmentinmillisecondsandachievesaccuracysupe-riororcomparabletostate-of-the-artmethodsonstandarddatasets.Thespeedgainsoverpreviousmethodsisacon-sequenceofidentifyingtheessentialcomponentsofpriorfacealignmentalgorithmsandthenincorporatingtheminastreamlinedformulationintoacascadeofhighcapacityregressionfunctionslearntviagradientboosting.Weshow,asothershave[8,2],thatfacealignmentcanbesolvedwithacascadeofregressionfunctions.Inourcaseeachregressionfunctioninthecascadeefficientlyestimatestheshapefromaninitialestimateandtheintensitiesofasparsesetofpixelsindexedrelativetothisinitialestimate.Ourworkbuildsonthelargeamountofresearchoverthelastdecadethathasresultedinsignificantprogressforfacealignment[9,4,13,7,15,1,16,18,3,6,19].Inparticular,weincorporateintoourlearntregressionfunctionstwokeyelementsthatarepresentinseveralofthesuccessfulalgo-rithmscitedandwedetailtheseelementsnow.Figure1.SelectedresultsontheHELENdataset.Anensembleofrandomizedregressiontreesisusedtodetect194landmarksonfacefromasingleimageinamillisecond.Thefirstrevolvesaroundtheindexingofpixelintensi-tiesrelativetothecurrentestimateoftheshape.Theex-tractedfeaturesinthevectorrepresentationofafaceimagecangreatlyvaryduetobothshapedeformationandnui-sancefactorssuchaschangesinilluminationconditions.Thismakesaccurateshapeestimationusingthesefeaturesdifficult.Thedilemmaisthatweneedreliablefeaturestoaccuratelypredicttheshape,andontheotherhandweneedanaccurateestimateoftheshapetoextractreliablefeatures.Previouswork[4,9,5,8]aswellasthiswork,useanit-erativeapproach(thecascade)todealwiththisproblem.Insteadofregressingtheshapeparametersbasedonfea-turesextractedintheglobalcoordinatesystemoftheimage,theimageistransformedtoanormalizedcoordinatesystembasedonacurrentestimateoftheshape,andthenthefea-turesareextractedtopredictanupdatevectorfortheshapeparameters.Thisprocessisusuallyrepeatedseveraltimesuntilconvergence.Thesecondconsidershowtocombatthedifficultyofthe1inference/predictionproblem.Attesttime,analignmental-gorithmhastoestimatetheshape,ahighdimensionalvec-tor,thatbestagreeswiththeimagedataandourmodelofshape.Theproblemisnon-convexwithmanylocaloptima.Successfulalgorithms[4,9]handlethisproblembyassum-ingtheestimatedshapemustlieinalinearsubspace,whichcanbediscovered,forexample,byfindingtheprincipalcomponentsofthetrainingshapes.Thisassumptiongreatlyreducesthenumberofpotentialshapesconsideredduringinferenceandcanhelptoavoidlocaloptima.Recentwork[8,11,2]usesthefactthatacertainclassofregressorsareguaranteedtoproducepredictionsthatlieinalinearsub-spacedefinedbythetrainingshapesandthereisnoneedforadditionalconstraints.Crucially,ourregressionfunc-tionshavethesetwoelements.Alliedtothesetwofactorsisourefficientregressionfunctionlearning.Weoptimizeanappropriatelossfunc-tionandperformfeatureselectioninadata-drivenmanner.Inparticular,welearneachregressorviagradientboosting[10]withasquarederrorlossfunction,thesamelossfunc-tionwewanttominimizeattesttime.Thesparsepixelset,usedastheregressor’sinput,isselectedviaacombinationofthegradientboostingalgorithmandapriorprobabilityonthedistancebetweenpairsofinputpixels.Thepriordistri-butionallowstheboostingalgorithmtoefficientlyexplorealargenumberofrelevantfeatures.Theresultisacascadeofregressorsthatcanlocalizethefaciallandmarkswheninitializedwiththemeanfacepose.Themajorcontributionsofthispaperare1.Anovelmethodforalignmentbasedonensembleofregressiontreesthatperformsshapeinvariantfeatureselectionwhileminimizingthesamelossfunctiondur-ingtrainingtimeaswewanttominimizeattesttime.2.Wepresentanaturalextensionofourmethodthathan-dlesmissingoruncertainlabels.3.Quantitativeandqualitativeresultsarepresentedthatconfirmthatourmethodproduceshighqualitypredic-tionswhilebeingmuchmoreefficientthanthebestpreviousmethod(Figure1).4.Theeffectofquantityoftrainingdata,useofpartiallylabeleddataandsynthesizeddataonqualityofpredic-tionsareanalyzed.2.MethodThispaperpresentsanalgorithmtopreciselyestimatethepositionoffaciallandmarksinacomputationallyeffi-cientway.Simila