Color-Transfer-between-Images

Oneofthemostcommontasksinimageprocessingistoalteranimage’scolor.Oftenthismeansremovingadominantandundesirablecolorcast,suchastheyellowinphotostakenunderincandescentillumination.Thisarticledescribesamethodforamoregeneralformofcolorcorrectionthatborrowsoneimage’scolorcharacteristicsfromanoth-er.Figure1showsanexampleofthisprocess,whereweappliedthecolorsofasunsetphotographtoadaytimecomputergraphicsrendering.Wecanimaginemanymethodsforapplyingthecolorsofoneimagetoanother.Ourgoalistodosowithasimplealgorithm,andourcorestrategyistochooseasuitablecolorspaceandthentoapplysimpleoper-ationsthere.Whenatypicalthreechannelimageisrepresentedinanyofthemostwell-knowncolorspaces,therewillbecorrelationsbetweenthedifferentchannels’val-ues.Forexample,inRGBspace,mostpixelswillhavelargevaluesfortheredandgreenchannelifthebluechannelislarge.Thisimpliesthatifwewanttochangetheappearanceofapixel’scolorinacoherentway,wemustmodifyallcolorchannelsintandem.Thiscomplicatesanycolormodificationprocess.Whatwewantisanorthogonalcolorspacewithoutcorrelationsbetweentheaxes.Recently,Rudermanetal.developedacolorspace,calledlαβ,whichminimizescorrelationbetweenchan-nelsformanynaturalscenes.2Thisspaceisbasedondata-drivenhumanperceptionresearchthatassumesthehumanvisualsystemisideallysuitedforprocessingnaturalscenes.Theauthorsdiscoveredlαβcolorspaceinthecontextofunderstandingthehumanvisualsys-tem,andtoourknowledge,lαβspacehasneverbeenappliedotherwiseorcomparedtoothercolorspaces.There’slittlecorrelationbetweentheaxesinlαβspace,whichletsusapplydifferentoperationsindifferentcolorchannelswithsomeconfidencethatundesirablecross-channelartifactswon’toccur.Additionally,thiscolorspaceislogarithmic,whichmeanstoafirstapproxima-tionthatuniformchangesinchannelintensitytendtobeequallydetectable.3DecorrelatedcolorspaceBecauseourgoalistomanipulateRGBimages,whichareoftenofunknownphosphorchromaticity,wefirstshowareasonablemethodofconvertingRGBsignalstoRudermanetal.’sperception-basedcolorspacelαβ.BecauselαβisatransformofLMSconespace,wefirstconverttheimagetoLMSspaceintwosteps.ThefirstisaconversionfromRGBtoXYZtristimulusvalues.Thisconversiondependsonthephosphorsofthemonitorthattheimagewasoriginallyintendedtobedisplayedon.Becausethatinformationisrarelyavailable,weuseadevice-independentconversionthatmapswhiteinthechromaticitydiagram(CIExy)towhiteinRGBspaceandviceversa.Becausewedefinewhiteinthechromaticitydiagramasx=X/(X+Y+Z)=0.333,y=Y/(X+Y+Z)=0.333,weneedatransformthatmapsX=Y=Z=1toR=G=B=1.Toachievethis,wemodifiedtheXYZitu601-1(D65)standardconver-sionmatrixtohaverowsthataddupto1.TheInterna-tionalTelecommunicationsUnionstandardmatrixis(1)BylettingMitux=(111)Tandsolvingforx,weobtainavectorxthatwecanusetomultiplythecolumnsofmatrixMitu,yieldingthedesiredRGBtoXYZconversion:(2)ComparedwithMitu,thisnormalizationprocedurecon-XYZRGB=051410323901604026510670200641002410122808444.........Mitu=043060341501784022200706700713002020129509394.........0272-1716/01/$10.00©2001IEEEAppliedPerception34September/October2001Weuseasimplestatisticalanalysistoimposeoneimage’scolorcharacteristicsonanother.Wecanachievecolorcorrectionbychoosinganappropriatesourceimageandapplyitscharacteristictoanotherimage.ErikReinhard,MichaelAshikhmin,BruceGooch,andPeterShirleyUniversityofUtahColorTransferbetweenImagesstitutesasmalladjustmenttothematrix’svalues.Onceindevice-independentXYZspace,wecancon-verttheimagetoLMSspaceusingthefollowingconversion:4(3)Combiningthesetwomatricesgivesthefollowingtrans-formationbetweenRGBandLMSconespace:(4)Thedatainthiscolorspaceshowsagreatdealofskew,whichwecanlargelyeliminatebyconvertingthedatatologarithmicspace:2L=logLM=logMS=logS(5)Usinganensembleofspectralimagesthatrepresentsagoodcross-sectionofnaturallyoccurringimages,Rud-ermanetal.proceedtodecorrelatetheseaxes.Theirmotivationwastobetterunderstandthehumanvisualsystem,whichtheyassumedwouldattempttoprocessinputsignalssimilarly.Wecancomputemaximaldecor-relationbetweenthethreeaxesusingprincipalcompo-nentsanalysis(PCA),whicheffectivelyrotatesthem.Thethreeresultingorthogonalprincipalaxeshavesim-pleformsandareclosetohavingintegercoefficients.Movingtothosenearbyintegercoefficients,Rudermanetal.suggestthefollowingtransform:(6)IfwethinkoftheLchannelasred,theMasgreen,andtheSasblue,wecanseethatthisisavariantofmanyopponent-colormodels:4Achromatic∝r+g+bYellow–blue∝r+g–bRed–green∝r–g(7)Thusthelaxisrepresentsanachromaticchannel,whiletheαandβchannelsarechromaticyellow–blueandred–greenopponentchannels.Thedatainthisspacearesymmetricalandcompact,whileweachievedecorrelationtohigherthansecondorderforthesetofnaturalimagestested.2Flanaganetal.5mentionedthiscolorspaceearlierbecause,inthiscolorspace,theachromaticaxisisorthogonaltotheequiluminantplane.Ourcolor-correctionmethodoperatesinthislαβspacebecausedecorrelationletsustreatthethreecolorchannelsseparately,simplifyingthemethod.Aftercolorprocessing,whichweexplaininthenextsection,wemusttransfertheresultbacktoRGBtodis-playit.Forconvenience,herearetheinverseopera-tions.Weconvertfr