Comparison of skin color detection and tracking me

1BirgittaMartinkauppi*^,MaricorSoriano**,andMattiPietikäinen**^DepartmentofElectricalandInformationEngineering,P.O.Box4500,FIN-90014,UniversityofOulu,Finland.,currentlyat:InFotonicsCenter,P.O.Box111,FIN-80101,UniversityofJoensuu,Finland.**NationalInstituteofPhysics,UniversityofthePhilippines,1101Diliman,QuezonCity,Philippines.*DepartmentofElectricalandInformationEngineering,P.O.Box4500,FIN-90014,UniversityofOulu,Finland.Abstract—Whenskinareaslikefacesandhandsareimagedundernaturalenvironmentstheircolorappearanceisfrequentlyaffectedbyvariationsinilluminationintensityandchromaticity.Incolor-basedskintrackinganddetection,changingintensityisoftendismissedeitherwiththeuseofnormalized,intensity-invariantcolorcoordinatesorbyadditionallymodelingpossibleskinintensities.Chromaticityvariationsarerarelyconsideredalthoughtheyarecommoninpractice.Inmostapproachesconsideringchromaticitytheexperimentsaredonewithasmallorundefinedvariationrange.Itisdifficulttocomparedifferentapproachesandassesstheirapplicabilityrangeforthisreason.Toimprovethesituation,weevaluatetheperformanceoffourstate-of-the-artmethodsunderdrasticbutpracticallycommonilluminationchanges.Theeffectofilluminationchromaticityforskinisclearlydefined,andbasedonitwedrawconclusionabouttheperformanceoftheseapproaches.Indexterms—colorvision,RGBfaceimages,facedetection,facetracking,performanceevaluation1.INTRODUCTIONFacesandhandsareimportantforhumansocialinteraction:facesareusedtorecognizeotherpersonsandhandgesturescanbeusedasameansofcommunication.Theyalsoprovideinformationaboutactions,forexamplewhenapersonismovingtowardsComparisonofskincolordetectionandtrackingmethodsundervaryingillumination2adoororliftsanobject.Itisthereforenotsurprisingthatmanymachinevisionapplications,suchashuman-computerinteraction,seekdataoutthem.Inthefirststage,facesandhandsneedtobefoundandseparatedfromtherestoftheimagecontent.Althoughmanydifferentcueshavebeenutilizedinthedetection,colorinformationisamongthemostpowerfulandpopularones;colorcuemayhavealowcomputationalcost,simplicity,highdiscriminativepowerandrobustnessagainstgeometricaltransformationslikerotation,scaling,transferandshapechangesunderauniform,stableilluminationfield.Whenusingcolorasacue,onemusttakeintoaccountitsrelationshiptoilluminationconditionsasitissensitivetotheseandwhitebalancingofthecamera.First,thesamescenemayappearwithslightlydifferentcolorbetweenimagestakenunderilluminantswithdifferentspectralpowerdistributioneventhoughthecamerawaswhitebalancedeachtimetotheprevailingilluminant.Second,ifthecameraisnotcalibratedtotheilluminationvariation,especiallyinthecaseofchangingilluminationchromaticity,thecolorappearancecanvarydrastically.Forexample,skincolorcanappearreddishorbluishandthedynamicrangeofthecameracanbemoreeasilyexceeded[1].Insomecases,eventhesurroundingscanproducestrongcolorcasts.Theeffectofvaryingilluminationchromaticityisrarelyconsideredbyskindetectionandtrackingalgorithms,possiblybecauseitisalsogenerallyignoredintestdatabasesusedforthem.Becauseingeneraltheimageorvideodatabasesconsideringilluminationchromaticityhavesmallandoftenunspecifiedvariation,wehavecreatedaFaceVideoDatabase[2]whichhasvideosontime-varyingilluminationchromaticitytakenwithseveralcameras,faceimagestakenunderdifferentbutknownwhitebalancingilluminationandcurrentilluminationconditionsandspectralinformationofskinandlightsources.Thespectralsensitivitiesofthecamerasandtheirsettingsareassumedtobeunknownasthisiscommoninpractice.Therefore,thecameraindependentskincolordatabase(e.g.skinreflectancesandSPDsoflightsourcesareknownlikein[1])wouldbeuseless.Whenthevideosweretaken,automaticwhitebalancingwasturnedoffandtheadjustmentofthecamerawasfixedafterinitialsettings.Thefaceareasaremanuallymarkedfromframesbyxy-coordinates(“boundingareas”)whichcanbeusedasgroundtruthsfortheevaluationofanalgorithm.ThedatabaseisavailableviatheInternet.Totackleproblemsofvaryingappearance,amassivenumberofstudieshavebeenmadeaboutrelationsbetweencolorappearanceandillumination.Thesuggestedclassic“solution”forvaryingcolorappearancesiscolorconstancy,whichmeanseithercorrectingcolorsoftheimageorusinginvariantcolorfeatures.Thecolorconstancyalgorithmsareoftenonlyforlimitedusebecauseoftheirrestrictingassumptions,likeunsaturatedchannels,andrequirements,likeanalwayspresentreferencecolor.Inaddition,theyareusuallydesignedforflatsurfacesandauniform,globalilluminationfield,anexceptionbeingRetinex(seeaboutRetinexdrawbacksin[3]).However,realscenesareperceivedinthree-dimensionalspaceandtheilluminationfieldmayvaryoverthesceneanditsobjects.Thesuitabilityofcolorconstancyalgorithmsformachinevisionpurposesismoredubiousbecausetheyhaveoftenbeendesignedtoproducepleasingcolorappearancesforhumans.ThisisclearlydemonstratedbyFunt3etal.[4]whoshowedthefailureofthesealgorithmsincolorindexing.Despitethis,someauthorsclaimilluminationinvariantorrobusttechniquesevenwhentheirtechniqueonlytolerateschangesinilluminationdirection,intensityorboth.Insomepapersitisnotevenclearforareaderwhichkindsofilluminationchangesthepaperissupposedt