Content-based image retrieval color and edges

1、Content-basedImageRetrieval:ColorandEdgesRobertS.GrayDepartmentofComputerScienceDartmouthCollegeHanover,NH03755E-mail:robert.s.gray@dartmouth.eduAbstractOneofthetoolsthatwillbeessentialforfutureelectronicpublishingisapowerfulimageretrievalsystem.Theauthorshouldbeabletosearchanimagedatabaseforimagesthatconveythedesiredinformationormood;areadershouldbeabletosearchacorpusofpublishedworkforimagesthatarerelevanttohisorherneeds.Mostcommercialimageretrievalsystemsassociatekeywordsortextwitheachimagean。

2、drequiretheusertoenterakeywordortextualdescriptionofthedesiredimage.Thistext-basedapproachhasnumerousdrawbacks{associatingkeywordsortextwitheachimageisatedioustask;someimagefeaturesmaynotbementionedinthetextualdescription;somefeaturesare\nearlyimpossibletodescribewithtext;andsomefeaturescanbedescribedinwidelydierentways[Na93a].Inaneorttoovercometheseproblemsandimproveretrievalperformance,researchershavefocusedmoreandmoreoncontent-basedimageretrievalinwhichretrievalisaccomplishedbycomparingimag。

3、efeaturesdirectlyratherthantextualdescriptionsoftheimagefeatures.Featuresthatarecommonlyusedincontent-basedretrievalincludecolor,shape,textureandedges.Inthispaperwedescribeasimplecontent-basedsystemthatretrievescolorimagesonthebasisoftheircolordistributionsandedgecharacteristics.Thesystemusestworetrievaltechniquesthathavebeendescribedintheliterature{i.e.histogramintersectiontocomparecolordistributionsandsketchcomparisontocompareedgecharacteristics.Theperformanceofthesystemisevaluatedandvariousex。

4、tensionstotheexistingtechniquesareproposed.1IntroductionOneofthetoolsthatwillbeessentialforfutureelectronicpublishingisapowerfulimageretrievalsystem.Theauthorshouldbeabletosearchanimagedatabaseforimagesthatconveythedesiredinformationormood;thereadershouldbeabletosearchacorpusofpublishedworkforimagesthatarerelevanttohisorherneeds.Mostcommercialimageretrievalsystemsassociatekeywordsortextwitheachimageinthecorpusandrequiretheusertoenterakeywordortextualdescriptionofthedesiredimage.Standardtextretri。

5、evaltechniquesarethenusedtoidentifytherelevantimages.Unfortunatelythistext-basedapproachtoimageretrievalhasnumerousdrawbacks[Na93a].Associatingkeywordsortextwitheachimageisatediousandtime-consumingtasksinceitmustbedonemanuallyoratbestsemi-automatically;imageprocessingtechnologyisnotadvancedenoughtoallowtheautomaticconstructionoftextualimagedescriptionsexceptinwell-denedandtightlyfocuseddomains.Someimagefeaturesmaynotbementionedinthetextualdescriptionduetodesigndecisionorindexererror;theseimagef。

6、eaturesdonotexistfromthestandpointoftheretrievalsystemandanyquerythatmentionsthemwillfail.Somefeaturesare\nearlyimpossibletodescribewithtext[Na93a];forexamplemanytexturesandshapesdefyeasydescription.Finallydierentindexers{oreventhesameindexer{maydescribethesamefeaturewithdierenttermsordierentfeatureswiththesameterm;thesearethestandardtextretrievalproblemsofsynonymyandpolysemy.Inaneorttoovercometheproblemsofthetext-basedapproachandimproveretrievalperformance,re-searchershavefocusedmoreandmore。

7、oncontent-basedimageretrievalinwhichretrievalisaccomplishedbycomparingimagefeaturesdirectlyratherthantextualdescriptionsoftheimagefeatures.Itishopedthatcontent-basedtechniquescanprovidethebasisforpowerful\querybyexampleretrievalsystems.Forexampletheusermightprovideasamplepictureandrequestsimilarpictures,apictureofanobjectandrequestpicturesthatcontaintheobject,asetofcolorsandrequestimagesthatcontainthosecolors,andsoon.Featuresthatarecommonlyusedincontent-basedimageretrievalincludecolor,shape,text。

8、ureandedges.1PartiallysupportedbyAFOSRcontractF49620-93-1-0266andAFOSR/DARPA89-05361ImagesColorextractionQueryprocessingformulationGUIforqueryedgemapsEdgeextractionHistogramsandFigure1:Thearchitectureoftheimageretrievalsystem{themainmodulesareedgeextraction,colorextraction,queryprocessinganduserinterface.Inthispaperwedescribeasimplecontent-basedsystemthatretrievescolorimagesonthebasisoftheircolordistributionsandedgecharacteristics.Thesystemisfullyautomaticasnomanualinterventionisrequiredduringth。

9、eindexingprocess.Fullautomation{alongwitheectiveretrievalperformance{istheprimarygoalofthesystemsinceamanualorsemi-automaticindexingprocessiserror-proneandtime-intensive.Thesystemdoesnotdevelopanynovelretrievaltechniquesbutinsteadusesexistingtechniquesthathavebeendescribedintheliterature{i.ehistogramintersection[SB91,Swa93]isusedtocomparecolordistributionsandsketchcomparison[HK92,KKOH92]isusedtocompareedgecharacteristics.Itishopedthatthesystemwillhighlightpotentialavenuesofresearchandserveasate。

10、stbedforfuturework.Tothisend,theperformanceofthesystemisevaluatedandvariousextensionstotheexistingretrievaltechniquesareproposed.Thenextsectiondescribestheimplementationofthesystem.Theremainingsectionsdiscusstheweaknessesofthecurrentimplementationandmethodsforaddressingtheseweaknesses.2ImplementationThesystemisimplementedasfourmodules{edgeextraction,colorextraction,queryprocessinganduserinterface.Thecolorandedgeextractionmodulesconstructasetofhistogramsandanedgemapforeachimage.Nomanualinterventio。