一种基于深度细节网络的图像除雨算法英文论文

RemovingrainfromsingleimagesviaadeepdetailnetworkXueyangFu1JiabinHuang1DeluZeng2YueHuang1XinghaoDing1∗JohnPaisley31KeyLaboratoryofUnderwaterAcousticCommunicationandMarineInformationTechnology,MinistryofEducation&SchoolofInformationScienceandEngineering,XiamenUniversity,China2SchoolofMathematics,SouthChinaUniversityofTechnology,China3DepartmentofElectricalEngineering&DataScienceInstitute,ColumbiaUniversity,USAAbstractWeproposeanewdeepnetworkarchitectureforremov-ingrainstreaksfromindividualimagesbasedonthedeepconvolutionalneuralnetwork(CNN).Inspiredbythedeepresidualnetwork(ResNet)thatsimplifiesthelearningpro-cessbychangingthemappingform,weproposeadeepde-tailnetworktodirectlyreducethemappingrangefrominputtooutput,whichmakesthelearningprocesseasier.Tofur-therimprovethede-rainedresult,weuseaprioriimagedo-mainknowledgebyfocusingonhighfrequencydetaildur-ingtraining,whichremovesbackgroundinterferenceandfocusesthemodelonthestructureofraininimages.Thisdemonstratesthatadeeparchitecturenotonlyhasbenefitsforhigh-levelvisiontasksbutalsocanbeusedtosolvelow-levelimagingproblems.Thoughwetrainthenetworkonsyntheticdata,wefindthatthelearnednetworkgeneralizeswelltoreal-worldtestimages.Experimentsshowthattheproposedmethodsignificantlyoutperformsstate-of-the-artmethodsonbothsyntheticandreal-worldimagesintermsofbothqualitativeandquantitativemeasures.WediscussapplicationsofthisstructuretodenoisingandJPEGarti-factreductionattheendofthepaper.1.IntroductionUnderrainyconditions,theimpactofrainstreaksonim-agesandvideoisoftenundesirable.Inadditiontoasubjec-tivedegradation,theeffectsofraincanalsoseverelyaffecttheperformanceofoutdoorvisionsystems,suchassurveil-lancesystems.Effectivemethodsforremovingrainstreaks∗Correspondingauthor:[e-mail]-portedinpartbytheNationalNaturalScienceFoundationofChinagrants61571382,81671766,61571005,81671674,U1605252,61671309and81301278,GuangdongNaturalScienceFoundationgrant2015A030313007,FundamentalResearchFundsfortheCentralUniversi-tiesgrants20720160075and20720150169,andtheCCF-Tencentresearchfund.X.FuconductedportionsofthisworkatColumbiaUniversityunderChinaScholarshipCouncilgrantNo.[2016]3100.Figure1:Anexamplereal-worldrainyimageandourresult.areneededforawiderangeofpracticalapplications.How-ever,whenanobject’sstructureandorientationissimilarwiththatofrainstreaks,itishardtosimultaneouslyre-moverainandpreservestructure.Toaddressthisdifficultproblem,wedevelopanend-to-enddeepnetworkarchitec-tureforremovingrainfromindividualimages.Figure1showsanexampleofareal-worldtestimageandourresult.Todate,manymethodshavebeenproposedforremovingrainfromimages.Thesemethodsfallintotwocategories:video-basedmethodsandsingle-imagebasedmethods.Webrieflyreviewtheseapproachesandthendiscussthecontri-butionsofourproposedframework.1.1.RelatedworkForvideo-basedmethods,raincanbemoreeasilyidenti-fiedandremovedusinginter-frameinformation[3,4,10,21,28,35].Manyofthesemethodsworkwell,butaresignif-icantlyaidedbythetemporalcontentofvideo.Inthispa-perweinsteadfocusonremovingrainfromasingleimage.Thistaskissignificantlymorechallengingsincemuchlessinformationisavailablefordetectingandremovingrain.Single-imagebasedmethodshavebeenproposedtodealwiththischallengingproblem.Forexample,in[20]kernelregressionandanon-localmeanfilteringareusedtodetectandremoverainstreaks.In[6],theauthorsproposeagen-eralizedmodelinwhichadditiverainisassumedtobelowrank.Ingeneral,however,successhasbeenlessnoticeablethaninvideo-basedalgorithmsandthereisstillmuchroom3855Figure2:Theproposedframeworkforsingle-imagerainremoval.Middleimagesshowabsolutevalueforbettervisualization.forimprovement.Severalmethodsusingpatch-basedmodelinghavealsobeenproposedandrepresentthecurrentstate-of-the-art[5,14,15,18,24,25,31].Forexample,in[25]theauthorsusediscriminativesparsecodingtorecoveracleanimagefromarainyimage.Recently,[24]proposedamethodbasedonGaussianmixturemodelsinwhichpatch-basedpriorsareusedforbothacleanlayerandarainlayer.Theauthorsshowhowmultipleorientationsandscalesofrainstreakscanbeaccountedforbysuchpre-trainedGaussianmixturemodels.1.2.OurcontributionsAsmentioned,removingrainfromasingleimageissig-nificantlymoredifficultthanfromvideo.Thisisbecausemostexistingmethodsseparaterainstreaksfromimagesusinglow-levelimagefeatures[14,18,24,25].Whenanobject’sstructuresandorientationsaresimilarwiththatofrainstreaks,thesemethodshaveadifficulttimesimultane-ouslyremovingrainandpreservingstructure.Towardsfix-ingthisproblem,wedesignarain-removalmethodbasedontheconvolutionalneuralnetwork(CNN)[22,23].ThedeepCNNhasnotonlyachievedsuccessonhigh-levelvi-siontasks[12,13]buthasalsobeenextendedtoproblemssuchasimagedenoising[34,36],super-resolution[7,19],reducingartifactsofcompression[8],imageinpainting[26]andimagedehazing[27].CNNsareeffectiveatincreasingtheabilityofamodeltoexploreandcaptureavarietyofimagecharacteristics[30].Inthispaper,wedesignanewnetworkarchitectureforsingle-imagerainremoval.Asshownbythedeepresidualnetwork(ResNet)[12],directlyreducingthemappingrangefrominputtooutputcanmakethelearningprocesssig