Pattern Recognition Mechanisms

PONTIFICIAEACADEMIAESCIENTIARVMSCRIPTAVARIA54STUDYWEEKonPATTERNRECOGNITIONMECHANISMSApril25-29,1983EDITEDBYCARLOSCHAGAS-RICARDOGATTASS-CHARLESGROSSEXAEDIBVSACADEMICISINCIVITATEVATICANA—MCMLXXXVTHEANALYSISOFMOVINGVISUALPATTERNSJ.ANTHONYMOVSHON*,EDWARDH.ADELSON**,MARTINS.GIZZI*andWILLIAMT.NEWSOME***INTRODUCTIONThereisabundantevidencethattheorientationofcontoursisafeatureofconsiderableimportancetothevisualsystem.Bothpsycho-physicalandelectrophysiologicalstudiessuggestthattheretinalimageistreatedrelativelyearlyinthevisualprocessbyorientationally-tunedspatialfilters(seeHubelandWiesel,1962;CampbellandKulikowski,1966,amongmanyothers).Orientationalfilteringundoubtedlyplaysaroleintheanalysisofthestructureofavisualpattern,butthevisualsystemhasothertasks,mostobviouslythatofextractinginformationaboutthemotionofobjects.Asimpleanalysisrevealsthatseparatingatwo-dimensionalimageintoitsone-dimensional(thatis,oriented)com-ponentspresentsproblemsforasystemconcernedwithextractingobjectmotion.Hereweoutlinetheproblem,proposeanovelformalsolutiontoit,andconsidertheapplicationsofthissolutiontoavarietyofper-ceptualandelectro-physiologicalphenomena.Theambiguityofmotionofone-dimensionalpatterns.Themotionofasingleextendedcontourdoesnotbyitselfallowonetodeterminethemotionofthesurfacecontainingthatcontour.Theproblemisillustrat-edinFig.1.Thethreesectionsofthefigureeachshowasurfacecontain-inganobliquegratinginmotionbehindacircularaperture.InFig.1Athesurfacemovesupandtotheleft;inFig.1Bitmovesup;inFig.1C,it*DepartmentofPsychology,NewYorkUniversity,NewYork,NY10003,USA.**Presentaddress:RCADavidSarnoffResearchLaboratories,Princeton,NJ08540,USA.***LaboratoryofSensorimotorResearch,NationalEyeInstitute,NationalInstitutesofHealth,Bethesda,MD20205,USA.Presentaddress:DepartmentofNeurobiologyandBehavior,StateUniversityofNewYork,StonyBrook,NY11794,USA.118PONTIFICIAEACADEMIAESCIENTIARVMSCRIPTAVARTA-54Fig.1.Threedifferentmotionsthatproducethesamephysicalstimulus.movestotheleft.Notethatinallthreecasestheappearanceofthemovinggrating,asseenthroughthewindow,isidentical:thebarsappeartomoveupandtotheleft,normaltotheirownorientation,asifproducedbythearrangementshowninFig.1A.Thefactthatasinglestimuluscanhavemanyinterpretationsderivesfromthestructureofthestimulusratherthanfromanyquirkofthevisualsystem.Anymotionparalleltoagra-ting'sbarsisinvisible,andonlymotionnormaltothebarscanbedetected.Thus,therewillalwaysbeafamilyofrealmotionsintwodimensionsthatcangiverisetothesamemotionofanisolatedcontourorgrating(Wohlgemuth,1911,Wallach,1935;FennemaandThompson,1979;MarrandUllman,1981).PATTERNRECOGNITIONMECHANISMS119Wemustdistinguishattheoutsetbetweenwhatwetermone-dimensional(1-D)andtwo-dimensional(2-D)patterns.A1-Dpatternisonelikeanextendedgrating,edge,orbar:itisuniformalongoneaxis.Ingeneral,suchapatternwouldhavetoextendinfinitelyalongitsaxistobetruly1-Dbutforthepresentpurposesitissufficientthatthepatternextendbeyondthebordersofthereceptivefieldofaneuronbeingstud-ied,orbeyondtheedgeofaviewingaperture.Theessentialpropertyisthat,whenal-Dpatternismovedparalleltoitsownorientation,itsappearancedoesnotchange.Byconvention(andinagreementwithitsappearance),wewillrepresenttheprimarymotionofa1-Dpatternashavingthevelocitynormaltoitsorientation.2-Dpatternsarenotinvariantwithtranslationalonganysingleaxis;theyincluderandomdotfields,plaids,andnaturalscenes.Suchpatternschangenomatterhowtheyaremoved,andtheirmotionisnotambiguousinthesamewayasthemotionofa1-Dpatternis.Inthispaperweareconcernedonlywithuniformlinearmotion.Forcertainotherkindsofmotion(e.g.rotationorcurvilinearmotion,ormotionindepth),analogousambiguitiesexistandcanbedescribedandsolvedinamannersimilartotheonewepresenthere(butseealsoHildreth,1983).Thedisambiguationofmotion.Ifthemotionofa1-Dpatternsuchasanedgeisambiguous,howisitpossibletodeterminethemotionofanobjectatall?Itturnsoutthat,althoughasinglemovingcontourcannotofferauniquesolution,twomovingcontours(whichbelongtothesameobject)can,aslongastheyarenotparallel.AsFig.1shows,thereisafamilyofmotionsconsistentwithagiven1-Dstimulus.Naturally,thisisalsotrueofthe1-Delementsofa2-Dstimulus.ConsiderthediamondsshowninFig.2A.Theleft-handdiamondmovestotheright;theright-handdiamondmovesdown.Notethatinbothcases,inthelocalregionindicatedoneachdiamondbythesmallcircle,thebordermovesdown-wardandtotheright.ThemovingedgeinFig.2B,whichcouldrepresentamagnifiedviewofthecircledregionsofthediamonds'bordersinFig.2A,canbegeneratedbyanyofthemotionsshownbythearrows.Motionparalleltotheedgeisnotvisible,soallmotionsthathavethesamecom-ponentofmotionnormaltotheedgearepossiblecandidatesforthetruemotiongivingrisetotheobservedmotionoftheedge.Wemaymapthissetofpossiblemotionsasalocusinvelocityspace,asshowninFig.2B.Velocitiesareplottedasvectorsinpolarcoordinates,startingattheorigin.Thelengthofthevectorcorrespondstothespeedofthemotion,andthe120PONTIFICIAEACADEMIESCIENTIARVMVARIA-54Fig.2.A.Twomovingdiamonds.Thelocalregionscircledoneachdiamond’sborderhaveidenticalmotions.B.Asinglemovingcontour,withtherepresentationofitspossi