[2009 CVIU] Performance evaluation of local colour

PerformanceevaluationoflocalcolourinvariantsGertjanJ.Burghoutsa,*,Jan-MarkGeusebroekbaTNOObservationSystems,ElectroOptics,OudeWaalsdorperweg63,2597AK,TheHague,TheNetherlandsbIntelligentSystemsLabAmsterdam,InformaticsInstitute,UniversityofAmsterdam,Kruislaan403,1098SJAmsterdam,TheNetherlandsarticleinfoArticlehistory:Received30September2007Accepted8July2008Availableonline24July2008Keywords:LocaldescriptorsColourSIFTabstractInthispaper,wecomparelocalcolourdescriptorstogrey-valuedescriptors.WeadopttheevaluationframeworkofMikolayzcykandSchmid.Wemodifytheframeworkinseveralways.Wedecomposetheevaluationframeworktotheleveloflocalgrey-valueinvariantsonwhichcommonregiondescriptorsarebased.Wecomparethediscriminativepowerandinvarianceofgrey-valueinvariantstothatofcolourinvariants.Inaddition,weevaluatetheinvarianceofcolourdescriptorstophotometriceventssuchasshadowandhighlights.Wemeasuretheperformanceoveranextendedrangeofcommonrecordingcon-ditionsincludingsignificantphotometricvariation.Wedemonstratetheintensity-normalizedcolourinvariantsandtheshadowinvariantstobehighlydistinctive,whiletheshadowinvariantsaremorerobusttobothchangesoftheilluminationcolour,andtochangesoftheshadingandshadows.Overall,theshadowinvariantsperformbest:theyaremostrobusttovariousimagingconditionswhilemaintain-ingdiscriminativepower.WhenpluggedintotheSIFTdescriptor,theyshowtooutperformothermeth-odsthathavecombinedcolourinformationandSIFT.TheusefulnessofC-colour-SIFTforrealisticcomputervisionapplicationsisillustratedfortheclassificationofobjectcategoriesfromtheVOCchal-lenge,forwhichasignificantimprovementisreported.2008ElsevierInc.Allrightsreserved.1.IntroductionManycomputervisiontasksdependheavilyonlocalfeatureextractionandmatching.Objectrecognitionisatypicalcasewherelocalinformationisgatheredtoobtainevidenceforrecognitionofpreviouslylearnedobjects.Recently,muchemphasishasbeenplacedonthedetectionandrecognitionoflocally(weakly)affineinvariantregions[1–5].Therationalehereisthatplanarregionstransformaccordingtowellknownlaws.Successfulmethodsrelyonfixingalocalcoordinatesystemtoasalientimageregion,resultinginanellipsedescribinglocalorientationandscale.Aftertransformingthelocalregiontoitscanonicalform,imagedescrip-torsshouldbewellabletocapturetheinvariantregionappear-ance.AspointedoutbyMikolajczykandSchmid[6],thedetectionofellipticregionsvariescovariantlywiththeimage(weakperspective)transformation,whilethenormalizedimagepatterntheycoverandtheimagedescriptorsderivedfromthemaretypicallyinvarianttothegeometrictransformation.Recogni-tionperformanceisfurtherenhancedbydesigningimagedescrip-torstobephotometricinvariant,suchthatlocalintensitytransformationsduetoshadingandvariationinilluminationhavenoorlimitedeffectontheregiondescription.State-of-the-artmethodsinobjectrecognitionnormalizemeanintensityandstan-darddeviationoftheintensityimage[2,6,7].Moreover,imagemeasurementsusingaGaussianfilteranditsderivativesisbecom-ingincreasinglypopularasawayofdetectingandcharacterizingimagecontentinageometricandphotometricinvariantway.Gaussianfiltershaveinterestingpropertiesfromanimageprocess-ingpointofview,amongothers,theirrobustnesstonoise[8],theirrotationalsteerability[9],andtheirapplicabilityinmulti-scalesettings[10].ManyoftheintensitybaseddescriptorsproposedinliteraturearebasedonGaussian(derivative)measurements[1,11–14].Hence,asonecontributionofthispaper,weaimtoeval-uatetheGaussianderivativeperformanceindependentofthedescriptor.AwellengineeredexponentofintensitydescriptorsisLowe’sSIFTdescriptor[2].Indeed,forgrey-valuedescriptors,thedetectionofaffineregionscombinedwiththeSIFTdescriptorisdemonstratedtobebetterthanmanyalternatives[1].Hence,asasecondcontributionofthispaper,weaimtoextentthisdescrip-tortocolour,andwewillevaluateitsperformancewithrespecttophotometricvariationanddiscriminativepower.Inthispaper,weconsidertheextensiontocolour-baseddescriptors.colourhashighdiscriminativepower;inmanycases,objectscanwellberecognizedmerelybytheircolourcharacteris-tics[15–20].However,photometricinvarianceislesstrivialtoachieve,astheaccidentalilluminationandrecordingconditionsaf-fecttheobservedcoloursinacomplicatedway.Photometricinvariancehasbeenintensivelystudiedforcolourfeatures1077-3142/$-seefrontmatter2008ElsevierInc.Allrightsreserved.doi:10.1016/j.cviu.2008.07.003*Correspondingauthor.E-mailaddresses:[e-mail](G.J.Burghouts),[e-mail](J.-M.Geusebroek).ComputerVisionandImageUnderstanding113(2009)48–62ContentslistsavailableatScienceDirectComputerVisionandImageUnderstandingjournalhomepage:[16,17,21–24].Geusebroeketal.[25]derivedasetofcolourinvari-antfeaturesbasedontheGaussianderivativeframework,facili-tatedbyKoenderink’sGaussiancolourmodel.Theimportantresearchquestionisifcolour-baseddescriptorsindeedimproveupontheirgrey-basedcounterpartsinpractise.Theanswerde-pendsonthestabilityofthenon-linearcombinationsofGaussianderivativesnecessarytoachieveasimilarlevelofinvarianceasimplementedingrey-valuedescriptors.Forinstance,thevaluesofphotometricinva