Engineering applications of the self-organizing ma

TEUVOKOHONEN,FELLOW,IEEE,ERKKIOJA,SENIORMEMBER,IEEE,OLLISTMULA,SENIORMEMBER,IEEE,ARIVISA,SENIORMEMBER,IEEE,ANDJAR1UNGASInvitedPaperTheself-organizingmap(SOM)methodisanew,powerjfulsoftwaretoolforthevisualizationofhigh-dimensionaldata.Itconvertscomplex,nonlinearstatisticalrelationshipsbetweenhigh-dimensionaldataintosimplegeometricrelationshipsonalow-dimensionaldisplay.Asittherebycompressesinformationwhilepreservingthemostimportanttopologicalandmetricrelationshipsoftheprimarydataelementsonthedisplay,itmayalsobethoughttoproducesomekindofabstractions.Thesetwoaspects,visualizationandabstraction,occurinanumberofcomplexengineeringtaskssuchasprocessanalysis,machineperception,control,andcommunication.Thetermself-organizingmapsignifiesaclassofmappingsdefinedbyerror-theoreticconsiderations.Inpracticetheyresultincertainunsupervised,competitivelearningprocesses,computedbysimple-lookingSOMalgorithms.ThefirstSOMalgorithmswereconceivedaround1981-1982,andthepopularityofthemoreadvancedSOMmethodsisgrowingatasteadypace.ManyindustrieshavefoundtheSOM-basedsoftwaretoolsuseful.ThemostimportantpropertyoftheSOM,orderlinessoftheinpikt-outputmapping,canbeutilizedformanytasks:reductionoftheamountoftrainingdata,speedinguplearning,nonlinearinterpolation,andextrapolation,generalization,andeffectivecompressionofinformationforitstransmission.~I.INTRODUCTIONA.ObjectivesTheself-organizingmap(SOM)isaneural-networkmodelandalgorithmthatimplementsacharacteristicnonlinearprojectionfromthehigh-dimensionalspaceofsensoryorotherinputsignalsontoalow-dimensionalarrayofneurons.TheSOMisabletomapastructured,high-dimensionalsignalmanifoldontoamuchlower-dimensionalnetworkinanorderlyfashion.Themappingtendstopreservethetopologicalrelationshipsofthesignaldomains.Duetothisorder,theimageofthesignalspaceManuscriptreceivedOctober1,1995,revisedMarch14,1996TheauthorsarewithHelsinkiUniversityofTechnology,NeuralNet-PublisherItemIdentifierS0018-9219(96)07176-9worksResearchCentre,FIN-02150Espoo,Finland1358tendstomanifestclustersofinputinformationandtheirrelationshipsonthemap.Accordingly,themostimportantapplicationsoftheSOMareinthevisualizationofhigh-dimensionalsystemsandprocessesanddiscoveryofcategoriesandabstractionsfromrawdata.Thelatteroperationiscalledtheexploratorydataanalysisor“datamining.”Inengineering,themoststraightforwardapplicationsoftheSOMareintheidentificationandmonitoringofcomplexmachineandprocessstates,otherwiseverydif-ficulttoperceiveandinterpret.TheSOMcanalsobeutilizedforthedevelopmentofnewpattemclassificationandtargetrecognitionsystems,wherebycategorizationoftheinputsignalstatesisperformedbyit.Incombinationwithreinforcementlearning,theSOMcanbeeffectivelyusedtoimplementvariouscontrolfunctions.Moreover,theorderedsignalmappinghasbeenfoundveryusefulforcertaintasksintelecommunications,suchasincreasingemortoleranceinsignaldetectionandimagetransmission.B.TheSOMAlgorithmTheSOMmaybedescribedasanonlinear,ordered,andsmoothmappingofhigh-dimensionalinputdatadomainsontotheelementsofaregular,low-dimensionalarray.AphysicalanalogyoftheSOMisanordereddecoderarrayinwhichoneandonlyoneofthedecodersrespondstoaparticulardomainofinputsignals.Itisthelocationofthedecoderandnottheexactmagnitudeofitsresponsethatcharacterizesinputinformation.Unlikeindigital-computertechnology,however,theinputsignalsetscanusuallyberegardedasrealvectorsinametricspace,typicallyahigh-dimensionalEuclideanspace.ThismappingisimplementedbytheSOMalgorithminthefollowingway.Assumethatthesetofinputvariables{E,}isdefinableasarealvectorx=[El,&,...,[,ITE!Rn.WitheachelementintheSOMarrayweassociateaparametricrealvectorm,=[pzl,pLz2,...,pLznITERn.The“image”ofaninputvectorxontheSOMarray0018-9219/96$05.0001996IEEEPROCEEDINGSOFTHEIEEE,VOL84,NO10,OCTOBER1996isdefinedbythedecoderfunction:assumingageneraldistancemeasurebetweenxandm,denotedd(x,m,),theimageissupposedtohavethearrayindexcdefinedasc=argmin{d(x,2m,)}.(1)Ourtaskistodefinethem,insuchawaythatthemappingisorderedanddescriptiveofthedistributionofx.Onemaytrytodeterminethem,byanoptimizationprocess,followingtheideaoftheclassicalvectorquanti-zation(VQ)[49],[53],[114].Inthelatter,afinitesetofcodebookvectors{m,}isplacedintothespaceofthexsignalstoapproximatethem.Letp(x)betheprobabilitydensityfunctionofx,andletm,bethecodebookvectorthatisclosesttoxinthesignalspace,i.e.,d(x,m,)issmallest.TheVQshallthenminimizetheaverageexpectedquantizationerrorfunctionE=/f[d(x,mc)lp(x)dx(2)wherefissomemonotonicallyincreasingfunctionofthedistanced.Noticethattheindexcisalsoafunctionofxandallthem,,wherebytheintegrandofEisnotcontinuouslydifferentiable:cchangesabruptlywhencrossingaborderinthesignalspaceatwhichtwocodebookvectorshavethesamevalueofdistancefunction.MinimizationofEmayleadtoacomplicatedtreatment[97].Thesetofvalues{m,}thatminimizesEisthesolutionoftheVQproblem,andthesignalspaceismappedontothesetofcodebookvectors.However,theindexingofthesevaluescanbemadeinanarbitraryway,wherebythismappingisstillunordered.Oneoftheauthors[97]hasshownthatifEismodifiedinsuchawaythatthequantizationerrorissmoothedloc