Feed-forward neural nets as models for time series

Feed-forwardNeuralNetsasModelsforTimeSeriesForecastingZaiyongTangBUS351Dept.ofDecision&InformationSciencesUniversityofFlorida,Gainesville,FL32611Phone:[phone]Email:[e-mail]301CSE,DepartmentofComputer&InformationSciencesUniversityofFlorida,Gainesville,FL32611Phone:[phone]Email:[e-mail]:Feed-forwardneuralnetworks,applicationForecasting:TimeseriesAbstractWehavestudiedneuralnetworksasmodelsfortimeseriesforecasting,andourresearchcomparestheBox-Jenkinsmethodagainsttheneuralnetworkmethodforlongandshorttermmemoryseries.Ourworkwasinspiredbypreviouslypublishedworksthatyieldedinconsistentresultsaboutcomparativeperformance.Wehavesinceexperimentedwith16timeseriesofdieringcomplexityusingneuralnetworks.TheperformanceoftheneuralnetworksiscomparedwiththatoftheBox-Jenkinsmethod.Ourexperimentsindicatethatfortimeserieswithlongmemory,bothmethodsproducedcomparableresults.However,forserieswithshortmemory,neuralnetworksoutperformedtheBox-Jenkinsmodel.Becauseneuralnetworkscanbeeasilybuiltformultiple-step-aheadforecasting,theypresentabetterlongtermforecastmodelthantheBox-Jenkinsmethod.Wediscussedtherepresentationability,themodelbuildingprocessandtheapplicabilityoftheneuralnetapproach.Neuralnetworksappeartoprovideapromisingalternativefortimeseriesforecasting.TR91-008ComputerandInformationSciences,UniversityofFlorida1IntroductionAtimeseriesisasequenceoftime-ordereddatavaluesthataremeasurementsofsomephysicalprocess.Forexample,onecanconsidertime-dependentsalesvolumeorrainfalldatatobeexamplesoftimeseries.Timeseriesforecastinghasanespeciallyhighutilityforpredictingeconomicandbusinesstrends.Manyforecastingmethodshavebeendevelopedinthelastfewdecades(Makridakis1982).TheBox-Jenkinsmethodisoneofthemostwidelyusedtimeseriesforecastingmethodsinpractice(BoxandJenkins1970).Recently,articialneuralnetworksthatserveaspowerfulcomputationalframeworkshavegainedmuchpopularityinbusinessapplicationsaswellasincomputerscience,psychologyandcognitivescience.Neuralnetshavebeensuccessfullyappliedtoloanevaluation,signaturerecog-nition,timeseriesforecasting(DuttaandShekhar1988;ShardaandPatil1990),classicationanalysis(FisherandMcKusick1989;SingletonandSurkan1990),andmanyotherdicultpat-ternrecognitionproblems(Simpson1990).Whileitisoftendicultorimpossibletoexplicitlywritedownasetofrulesforsuchpatternrecognitionproblems,aneuralnetworkcanbetrainedwithrawdatatoproduceasolution.Concerningtheapplicationofneuralnetstotimeseriesforecasting,therehavebeenmixedreviews.Forinstance,LapedesandFarber(1987)reportedthatsimpleneuralnetworkscanoutperformconventionalmethods,sometimesbyordersofmagnitude.Theirconclusionsarebasedontwospecictimeserieswithoutnoise.Weigendetal.(1990)appliedfeed-forwardneuralnetstoforecastingwithnoisyreal-worlddatafromsunspotsandcomputationalecosystems.Aneuralnetwork,trainedwithpastdata,generatedaccuratefuturepredictionsandconsistentlyout-performedtraditionalstatisticalmethodssuchastheTAR(thresholdautoregressive)model(TongandLim1980).ShardaandPatil(1990)conductedaforecastingcompetitionbetweenneuralnetworkmodelsandatraditionalforecastingtechnique(namelytheBox-Jenkinsmethod)using75timeseriesofvariousnature.TheyconcludedthatsimpleneuralnetscouldforecastaboutaswellastheBox-Jenkinsforecastingsystem.Fishwick(1989)demonstratedthat,foraballisticstrajectoryfunctionapproximationproblem,theneuralnetworkusedoeredlittleTR91-008ComputerandInformationSciences,UniversityofFlorida2competitiontothetraditionallinearregressionandsurfaceresponsemodel.1Thepotentialofneuralnetsasmodelsfortimeseriesforecastinghasnotbeenstudiedsys-tematically.ShardaandPatil(1990)reportedthattheperformanceofneuralnetsandtheBox-Jenkinsmethodareonpar.Outofthe75seriestheytested,neuralnetsperformedbetterthantheBox-Jenkinsmethodfor39series,andworsefortheother36series.Onewouldaskwhenneuralnetsarebetterthantheothermethod?Cantheperformanceofneuralnetsbeimproved?Neuralnetmodelsaregenerallyregardedas\blackboxes.Fewresearchershaveexploredindetailneuralnetsastimesseriesforecastingmodels.Wewouldliketoaskwhyneuralnetscanbeusedasforecastingmodels,andwhyshouldtheybeabletocompetewithconventionalmethodssuchastheBox-Jenkinsmethod.Wewilltrytoanswerthequestionsraisedabovebyperformingaseriesofforecastingexper-imentsandanalysis.Dierentneuralnetstructuresandtrainingparameterswillbeused.TheperformanceofneuralnetsiscomparedwiththatoftheBox-Jenkinsmethod.Inthefollowingsection,wegiveabriefreviewofthetwoapproaches.Thesubsequentsectionpresentsthefore-castingexperimentresults.WediscusstheissuesinapplyingneuralnetsinforecastinginSection3.Thelastsectionpresentsasummaryofthestudyandourconclusions.1MethodologyBackgroundWhiletheBox-Jenkinsmethodisafairlystandardtimeseriesforecastmethod,neuralnetsasforecastmodelsarerelativelynew.Thus,inthefollowing,wepresentonlyabriefaccountoftheBox-Jenkinsmethodandgiveamoredetaileddescriptionoftheneuralnetapproach.Completetreatmentsofthetwomethodscanbefoundin(Ho1983)and(Rumelhart,McClellandandthePDPResearchGroup1986).1Welaterdeterminedt