Statistical Mechanics and Visual Signal Processing

arXiv:cond-mat/9401072v130Jan1994PUPT-1435cond-mat/9401072STATISTICALMECHANICSANDVISUALSIGNALPROCESSINGMARCPOTTERS(1,2)ANDWILLIAMBIALEK(1)(1)NECResearchInstitute,4IndependenceWay,Princeton,NewJersey08540(2)DepartmentofPhysics,PrincetonUniversity,Princeton,NewJersey08544January1994Thenervoussystemsolvesawidevarietyofproblemsinsignalprocessing.Inmanycasestheperformanceofthenervoussystemissogoodthatitappo-rachesfundamentalphysicallimits,suchasthelimitsimposedbydiffractionandphotonshotnoiseinvision.Inthispaperweshowhowtousethelanguageofstatisticalfieldtheorytoaddressandsolveproblemsinsignalprocessing,thatisproblemsinwhichonemustestimatesomeaspectoftheenvironmentfromthedatainanarrayofsensors.Inthefieldtheoryformulationtheop-timalestimatorcanbewrittenasanexpectationvalueinanensemblewheretheinputdataactasexternalfield.Problemsatlowsignal-to-noiseratiocanbesolvedinperturbationtheory,whilehighsignal-to-noiseratiosaretreatedwithasaddle-pointapproximation.Theseideasareillustratedindetailbyanexampleofvisualmotionestimationwhichischosentomodelaproblemsolvedbythefly’sbrain.Inthisproblemtheoptimalestimatorhasarichstructure,adaptingtovariousparametersoftheenvironmentsuchasthemean-squarecontrastandthecorrelationtimeofcontrastfluctuations.Thisstructureisinqualitativeaccordwithexistingmeasurementsonmotionsensitiveneuronsinthefly’sbrain,andwearguethattheadaptivepropertiesoftheoptimalesti-matormayhelpresolveconlfictsamongdifferentinterpretationsofthesedata.Finallyweproposesomecrucialdirecttestsoftheadaptivebehavior.11IntroductionImaginewalkingalongabusycitystreet.Aswewalk,wearealmostunawareofthemyriadtaskswhichourbrainsareperforming:Weuseacombinationofvisualandvestibularsignalstokeepourselvesupright,sensorsinourfeetandlegmuscleshelpadjustourstridetotheterrain,welistenforcarsandpeoplebehindus,visionhelpsusrecognizethecaf´eweareapproachingandidentifiesourfriendwhowillmeetusthere,andallthesesensescombinetoprovideuswithatrajectorywhichreachesourgoalandavoidsobstacles.Allofthesetasksinvolvesignalprocessing,andwehavethequalitativeimpressionthatwe(andotheranimals)arequitegoodatsolvingtheseproblems.Thegoalofthispaperistoshowthatstatisticalmechanicsprovidesthenaturallanguageforformulatingandsolvingsignalprocessingproblems,andthatthestructureofthestatisticalmechanicsmodelsprovidesapredictivetheoryofhowrealbrainssolvethecorrespondingproblems.1.1PhysicallimitsandbiologicalsignalprocessingSignalprocessingisinessenceaphysicsproblem.Asanexample,invisionthepreciseformulationofanytaskmustbeginwiththefactthatimagesareblurredbydiffractionandcorruptedbyphotonshotnoise.Theseirreduciblelimitationsinthequalityoftheinputsignallimitthereliabilitywithwhichthebrain(oranydevice)canestimatewhatisreallygoingoninthevisualenvironment—wecanneverbetrulycertainofwhatwearelookingat,norcanweknowitsexacttrajectory,andsoon.Therearephysicallimitationsfromthehardwareaswell—cellsofacertainsizeareboundtogenerateelectricalnoise,signalsarepassedfromonepointtoanotheralongcablesoflimitedinformationcapacity,....Remarkably,thesephysicallimitationstothereliabilityofcomputationareactuallyrelevanttotheoperationofrealbrains.Indeedtherearemanycaseswheretheperformanceofthenervoussystemisessentiallyequaltothelimitthatonecalculatesfromfirstprinciples[2,4];examplesrangefromphotoncountingintoadsandhumans[1,13]tovisualmotionestimationinflies[7,44]toacousticcodinginfrogsandcrickets[39]andechodelayestimationinbats[48].Theseobservationsstronglysuggestthatatheoryofoptimalsignalprocessingwillhelpusunderstandthecomputationalstrategiesofbrains.1.2ChoosingaproblemOfthemanyexamplesofsignalprocessinginthenervoussystem,ourdiscussionismotivatedmostdirectlybytheproblemofmotionestimationinthefly’svisualsystem.Inmanyinsectsthevisualsystemproducesmovementsignalswhichareusedtocontroltheflightmusclesandtherebystabilizeflight.Inthecaseofflies,visuallyguidedflightbehaviorhasbeenstudiedbothbymeasuringflightpathsduringnaturalbehaviors[30,51,52,53]andbyexaminingthetorquesproduced2byflieshangingfromatorsionbalanceinresponsetomovementsofcontrolledpatternsacrossthevisualfield[20,36].Theinputtothemotioncomputationcomesfromasingleclassofphotoreceptorcellswhicharearrayedintheregularlatticeofthecompoundeye,andthesignalandnoisepropertiesofthesecellsareextremelywellcharacterized.Theoutputofthemovementcomputationcanbemonitoredinahandfulofidentifiedcellsofthelobulacomplex[15,19],anddestructionofindividuallobulaplateneuronsproducescleardeficitsinthefly’sopto-motorbehavior[18].Thefactthattheseneuronsare“identified”hasatechnicalmeaning:cellsofessentiallyidenticalmorphologyoccurineachfly,andthesecellshaveresponsestovisualstimuliwhicharequantitativelyreproduciblefromindividualtoindividual.Thusthecellscanbenamedandnumberedbasedoneitherstructureorfunction;underfavorableconditionsonecanrecordfromthecellH1,whichcodeswidefield,horizontalmovementsofthevisualfield,forperiodsofuptofivedays[43].Theaccessibilityofquantitativemeasurementsateachofseverallayersinthenervoussystem—photoreceptors,secondorderneurons,motion-sensitivecells,flightbehavior—makestheflyanearlyi