Salient-region-detection-and-segmentation

SalientRegionDetectionandSegmentationRadhakrishnaAchanta,FranciscoEstrada,PatriciaWils,andSabineSÄusstrunkSchoolofComputerandCommunicationSciences(I&C),EcolePolytechniqueF¶ed¶eraledeLausanne(EPFL),{radhakrishna.achanta,francisco.estrada,patricia.wils,sabine.susstrunk}@epfl.ch:Salientregions,low-levelfeatures,segmentation1IntroductionIdentifyingvisuallysalientregionsisusefulinapplicationssuchasobjectbasedimageretrieval,adaptivecontentdelivery[11,12],adaptiveregion-of-interestbasedimagecompression,andsmartimageresizing[2].Weidentifysalientre-gionsasthoseregionsofanimagethatarevisuallymoreconspicuousbyvirtueoftheircontrastwithrespecttosurroundingregions.Similarde¯nitionsofsaliencyexistinliteraturewheresaliencyinimagesisreferredtoaslocalcontrast[9,11].Ourmethodfor¯ndingsalientregionsusesacontrastdetermination¯lterthatoperatesatvariousscalestogeneratesaliencymapscontaining\saliencyvaluesperpixel.Combined,theseindividualmapsresultinour¯nalsaliencymap.Wedemonstratetheuseofthe¯nalsaliencymapinsegmentingwholeobjectswiththeaidofarelativelysimplesegmentationtechnique.Thenoveltyofourapproachliesin¯ndinghighqualitysaliencymapsofthesamesizeandresolutionastheinputimageandtheiruseinsegmentingwholeobjects.Themethodise®ectiveonawiderangeofimagesincludingthoseofpaintings,videoframes,andimagescontainingnoise.Thepaperisorganizedasfollows.TherelevantstateoftheartinsalientregiondetectionispresentedinSection2.OuralgorithmfordetectionofsalientregionsanditsuseinsegmentingsalientobjectsisexplainedinSection3.Theparametersusedinouralgorithm,theresultsofsaliencymapgeneration,seg-mentation,andcomparisonsagainstthemethodofIttietal.[9]aregiveninSection4.Finally,inSection5conclusionsarepresented.2AuthorsSuppressedDuetoExcessiveLength2ApproachesforSaliencyDetectionTheapproachesfordetermininglow-levelsaliencycanbebasedonbiologicalmodelsorpurelycomputationalones.Someapproachesconsidersaliencyoverseveralscaleswhileothersoperateonasinglescale.Ingeneral,allmethodsusesomemeansofdetermininglocalcontrastofimageregionswiththeirsur-roundingsusingoneormoreofthefeaturesofcolor,intensity,andorientation.Usually,separatefeaturemapsarecreatedforeachofthefeaturesusedandthencombined[8,11,6,4]toobtainthe¯nalsaliencymap.Acompletesurveyofallsaliencydetectionandsegmentationresearchisbeyondthescopeofthispaper,herewediscussthoseapproachesinsaliencydetectionandsaliency-basedsegmentationthataremostrelevanttoourwork.MaandZhang[11]proposealocalcontrast-basedmethodforgeneratingsaliencymapsthatoperatesatasinglescaleandisnotbasedonanybiologicalmodel.Theinputtothislocalcontrast-basedmapisaresizedandcolorquan-tizedCIELuvimage,sub-dividedintopixelblocks.Thesaliencymapisobtainedfromsummingupdi®erencesofimagepixelswiththeirrespectivesurroundingpixelsinasmallneighborhood.Thisframeworkextractsthepointsandregionsofattention.Afuzzy-growingmethodthensegmentssalientregionsfromthesaliencymap.Huetal.[6]createsaliencymapsbythresholdingthecolor,intensity,andorientationmapsusinghistogramentropythresholdinganalysisinsteadofascalespaceapproach.Theythenuseaspatialcompactnessmeasure,computedastheareaoftheconvexhullencompassingthesalientregion,andsaliencydensity,whichisafunctionofthemagnitudesofsaliencyvaluesinthesaliencyfeaturemaps,toweightheindividualsaliencymapsbeforecombiningthem.Ittietal.[9]havebuiltacomputationalmodelofsaliency-basedspatialat-tentionderivedfromabiologicallyplausiblearchitecture.Theycomputesaliencymapsforfeaturesofluminance,color,andorientationatdi®erentscalesthatag-gregateandcombineinformationabouteachlocationinanimageandfeedintoacombinedsaliencymapinabottom-upmanner.ThesaliencymapsproducedbyItti'sapproachhavebeenusedbyotherresearchersforapplicationslikeadaptingimagesonsmalldevices[3]andunsupervisedobjectsegmentation[5,10].SegmentationusingItti'ssaliencymaps(a480x320pixelimagegeneratesasaliencymapofsize30x20pixels)oranyothersub-sampledsaliencymapfromadi®erentmethodrequirescomplexapproaches.Forinstance,aMarkovrandom¯eldmodelisusedtointegratetheseedvaluesfromthesaliencymapalongwithlow-levelfeaturesofcolor,texture,andedgestogrowthesalientobjectregions[5].KoandNam[10],ontheotherhand,useaSupportVectorMachinetrainedonthefeaturesofimagesegmentstoselectthesalientregionsofinterestfromtheimage,whicharethenclusteredtoextractthesalientobjects.Weshowthatusingoursaliencymaps,salientobjectsegmentationispossiblewithoutneedingsuchcomplexsegmentationalgorithms.Recently,Frintropetal.[4]usedintegralimages[14]inVOCUS(VisualObjectDetectionwithaComputationalAttentionSystem)tospeedupcom-putationofcenter-surrounddi®erences