Research-and-Application-on-BP-Neural-Network-Algo

ResearchandApplicationonBPNeuralNetworkAlgorithmZhaoYanBohaiUniversity,Jinzhou,[e-mail]:neuralnetwork;BPalgorithm;basicidea;model;improvement;applicationAbstract.Intheartificialneuralnetwork,BPneuralnetworkisamultilayerfeedforwardneuralnetworkwhichisusedwidely.BPneuralnetworkusesaclassicBPalgorithm,anditisinaccordancewiththeerrorback-propagationalgorithmforlearningandtraining.ThispaperfirstanalyzesthebasicideaofBPalgorithm,andusingBPneuralnetworkmodelandtheflowcharttoillustrate;then,introducesthedisadvantageofBPalgorithm,hasslowconvergenceandeasytofallintolocalminimumpointandotherdefects,anddescribesthecurrentimprovementsmethods,suchasaddingmomentumitem,introducevariablestepmethodandotheroptimizationmethods,whicheffectivelyimprovetheconvergenceofBPalgorithm,toavoidfallingintolocalminimumpoint;finally,describesindetailtheapplicationofBPneuralnetworkinfacerecognition.EffectiveresearchonBPneuralnetwork,whichcanbefurtherdevelopmentandapplicationofBPnetworksplayanimportantrole.IntroductionArtificialneuralnetwork(ANN),alsoknownasneuralnetworks(NN),iscomposedofalargenumberofprocessingelements(neurons)madeextensiveinterconnectionnetwork,istheabstraction,simplificationandsimulationofhumanbrain,whichreflectsthebasiccharacteristicsofthehumanbrain,whichissimilartothehumanbraincanbesummarizedintwoaspects:throughthelearningprocessusingneuralnetworktoacquireknowledgefromexternalenvironment;internalneurons(synapticweights)usedtostoreacquireknowledgeandinformation[1].Artificialneuralnetworksisabranchofartificialintelligencesciencedevelopedrapidlyinrecentyears,iswidelyusedinrecentyearsonceagaindemonstrateditsactivevitality.BPneuralnetworkinthefieldofneuralnetworksiscurrentlyoneofthemostwidelyusedmodels,whichisoneoffeedforwardnetworksmostcommonlyusednetworkcanachievethemappingtransformation.BP(backpropagation)neuralnetworkisalsoknownasmulti-layerfeedforwardnetwork,whichisinaccordancewiththeerrorback-propagationalgorithmforlearningandtraining,itdoesnotrequirepriorknowledgeaboutthemathematicalequationofexpressionmapping,willbeabletotrainandstorelargeamountsofinputandoutputmodemappingrelationship[2].BecauseBPneuralnetworkusesaclassicBPalgorithm,whileBPalgorithmisbasedongradientsteepestdescentmethodwithsquarederrorastheobjectivefunction,whichenablestheuseofneuralnetworkalgorithmshavetheabilityoflearningandmemory.Andintheory,BPneuralnetworkcanapproximateanycontinuousnonlinearfunction:asimplethreelayerBPneuralnetworkcanachieveanyonefromndimensiontomdimensionalmapping,anditsthinkingclear,easyprogramming,simplestructure,highprecision,strongoperability,soithasbeenverywidelyusedinmanyfields.Themainapplicationfieldshavepatternrecognition,intelligentcontrol,faultdiagnosis,imagerecognitionprocessing,optimalcalculation,informationprocessing,financialforecasting,marketanalysisandbusinessmanagement,andsoon.TheBasicIdeaofBPAlgorithmBPneuralnetworkhasnotonlythenodesininputlayer,outputlayernode,andthereareoneormorelayerhiddennodes.Forinputinformation,thefirstforwardspreadtothenodesofthehiddenlayer,throughtheactivationfunctionofeachunit(alsoknownasactionfunction,conversionfunction)tooperate,aftertheoperation,theoutputinformationofhiddennodedisseminatetotheInternationalIndustrialInformaticsandComputerEngineeringConference(IIICEC2015)©2015.Theauthors-PublishedbyAtlantisPress1444outputnode,andfinallygivestheoutputresults.BPneuralnetworkmodelisshowninFig.1.BPnetworkusesagradientdescentmethod,gradientdescentmethodisbasedonthegradientoftheerrorfunctionforeverytwonodesweights,andcalculatetheweightcontributionoftheerrorfunction,andthenaccordingtothegradientinformationtomodifytheweightsinordertoachievethepurposeoflearning.BPnetworkcanhavemultiplehiddenlayers,withhhiddenlayer,accordingtofeedforwardorder,thehiddenlayernodesism1,m2…mh;eachhiddenlayeroutputisy1,y2…yh;weightmatrixforeachlayerisw1,w2…wh+1,theneachweightadjustmentformulais:outputlayer:nkmjyooodhhjkkkkhjhjhjk…=…=−−==∆++,2,1;,2,1,0,)1()(yw11hhd,hhiddenlayer:hhhjhjhjnkhjkkhjhjhijmjmiyyyw…=…=−==∆−−=+−∑,2,1;,2,1,0,)1()(yw1111o1dhhd,Accordingtotheruletoanalogizecanobtainthefirstlayerweightsadjustmentcalculationformula[3].ThemainideaofBPalgorithmisthelearningprocessintosignalforwardpropagationanderrorbackpropagationintwostages[4].Intheforwardpropagationstage,theinputinformationfromtheinputlayerthroughthehiddenlayertotheoutputlayeristransmitted,generatinganoutputsignalatanoutputterminal.Intheprocessofthesignalpassedalongthenetworkweightvalueisfixed,thestateofeachlayerneurononlyaffectsthelowerlayerneuronstate.Ifthedesiredoutputcannotgetintheoutputlayer,thereisanerrorbetweentheactualoutputsvalueandthedesiredoutputvalue,andthenturnedbackpropagationprocess.Intheback-propagationphase,errorsignalreturnsalongtheoriginalconnectionpath,bymodifyingtheweightsofeachlayerneurons,successivelytotheinputlayerpropagationtocalculate,andthenthroughtheforwardpropagationprocess,repeateduseofthesetwoprocesses,suchthattheerrorsignalisminimized.Infact,whenerrorreachedthedesiredrequirements,thenetwor