Adaptive-Loop-Filtering-for-Video-Coding

934IEEEJOURNALOFSELECTEDTOPICSINSIGNALPROCESSING,VOL.7,NO.6,DECEMBER2013AdaptiveLoopFilteringforVideoCodingChia-YangTsai,Member,IEEE,Ching-YehChen,TomooYamakage,InSukChong,Yu-WenHuang,Chih-MingFu,TakayukiItoh,TakashiWatanabe,TakeshiChujoh,MartaKarczewicz,andShaw-MinLei,Fellow,IEEEAbstract—Adaptiveloopfilteringforvideocodingistominimizethemeansquareerrorbetweenoriginalsamplesanddecodedsam-plesbyusingWiener-basedadaptivefilter.TheproposedALFislocatedatthelastprocessingstageforeachpictureandcanbere-gardedasatooltocatchandfixartifactsfrompreviousstages.Thesuitablefiltercoefficientsaredeterminedbytheencoderandexplicitlysignaledtothedecoder.Inordertoachievebettercodingefficiency,especiallyforhighresolutionvideos,localadaptationisusedforlumasignalsbyapplyingdifferentfilterstodifferentre-gionsorblocksinapicture.Inadditiontofilteradaptation,filteron/offcontrolatcodingtreeunit(CTU)levelisalsohelpfulforim-provingcodingefficiency.Syntax-wise,filtercoefficientsaresentinapicturelevelheadercalledadaptationparameterset,andfilteron/offflagsofCTUsareinterleavedatCTUlevelintheslicedata.Thissyntaxdesignnotonlysupportspictureleveloptimizationbutalsoachievesalowencodinglatency.SimulationresultsshowthattheALFcanachieveonaverage7%bitratereductionfor25HDsequences.Theruntimeincreasesare1%and10%forencodersanddecoders,respectively,withoutspecialattentiontooptimiza-tioninC++code.IndexTerms—ALF,adaptiveloopfiltering.I.INTRODUCTIONRECENTLYHigh-EfficiencyVideoCoding(HEVC)[1]hasbeendevelopedbytheJointCollaborativeTeamonVideoCoding(JCT-VC)formedbyISO/IECMovingPictureExpertsGroup(MPEG)andITU-TVideoCodingExpertsGroup(VCEG)forthenextgenerationvideocodingstandard.TheHEVCtargetis50%bitratereductionincomparisonwithAdvancedVideoCoding(AVC),ornamelyH.264[2],underthesamevisualquality.Inordertoimprovecodingefficiency,manyexistingtoolsinH.264/AVCandnewtoolshavebeenstudiedinVCEG.In-loopfilterisoneofthesetools.Deblockingfilter[3]isadoptedinH.264/AVCasanin-loopfilterthatisnon-linearfilteringwithManuscriptreceivedJanuary30,2013;revisedMay13,2013;acceptedJune13,2013.DateofpublicationJuly04,2013;dateofcurrentversionNovember18,2013.Theguesteditorcoordinatingthereviewofthismanuscriptandap-provingitforpublicationwasProf.YunHe.C.-Y.Tsai,C.-Y.Chen,Y.-W.Huang,C.-M.Fu,andS.LeiarewithMe-diaTek,Inc.,Hsinchu30078,Taiwan(e-mail:[e-mail];[e-mail];[e-mail];[e-mail];[e-mail]).T.Yamakage,T.Itoh,T.Watanabe,andT.ChujoharewithToshibaCorpo-ration,Kawasaki212-8582,Japan(e-mail:[e-mail];[e-mail];[e-mail];[e-mail]).I.S.ChongandM.KarczewiczarewithQualcommInc.,SanDiego,CA92121USA(e-mail:[e-mail];[e-mail]).Colorversionsofoneormoreofthefiguresinthispaperareavailableonlineatfinedlow-passcharacteristicstoreduceblockingartifactsatboundariesoftransformblocks.Inaddition,apost-filterhintSupplementalEnhancementInformation(SEI)messagethatre-ducesthecodingerrorofoutputpicturesbyWienerfilteringisadoptedinH.264/AVC[4].Inordertoreducecodingerrorofbothoutputpicturesandreferencepicturesfornon-boundarysamplesaswellasboundarysamples,whichwillbeusedformotioncompensationofsucceedingpictures,adaptiveloopfiltering(ALF)techniqueswereproposed[5]–[8].ALFintroducestheWienerfilterinthecodingloop.ItisalsopossibletointroduceaWienerfilterforthemotionsub-pixelinterpolation,whichiscalledadaptiveinterpolationfilter(AIF)[9].SincethecodingerrorisreducedbyALF,themotionsub-pixelinterpolationfiltercanconcen-trateoncreatingthesub-pixelpositionpixelsbypreservingthehigherfrequencysignals.Furthermore,sinceALFisin-loopfilteringprocess,oncethefilteringisperformed,theeffectwillbepropagatedtosucceedingpictures,whichresultsinlesserrorneededtobefilteredout,comparedtopost-filtering.ThisimpliesthatALFisamoreefficienttoolthanpost-filteringintermsofpowerconsumptionandcodingefficiency.Consideringthein-loopfilteringeffect,applyingtheWienerfiltertoallsamplesinapicturemaynotbeappropriateduetorepetitivefilteringinin-loopprocess.Inordertoavoidover-smoothingbyrepetitiveALF,ablock-basedfilteron/offcontrolmethodisused.InJanuary2010,aJointCallforProposals(CfP)onVideoCompressionTechnologywasissuedbyMPEGandVCEG.InresponsetotheCfP,27proposalsweresubmitted.AlmostthreequartersoftheproposalsincludedALFasacodingtool.There-fore,ALFwasincludedinthereferencesoftwareofHEVCfromthebeginning,whichwasfirstcalledTestModelunderConsideration(TMuC)andthennamedHEVCTestModel(HM).Fig.1showstheencoderblockdiagramofHM-7.0,whichcontainsthreein-loopfilteringblocks:deblockingfilter(DF),sampleadaptiveoffset(SAO)[10],andALF.UnlikeDF,whichusespredefinedfilters,SAOandALFutilizetheoriginalsamplesofthecurrentpicturetoreducethemeansquareerrorsbetweentheoriginalsamplesandthereconstructedsamplesbyaddinganoffsetandbyapplyingafiniteimpulseresponse(FIR)filter,respectively,withcodedsideinformationsignalingtheoffsetsandfiltercoefficients.ALFislocatedatthelas