EGG-脑电情绪识别

ComparativeanalysisofphysiologicalsignalsandElectroencephalogram(EEG)formultimodalemotionrecognitionusinggenerativemodelsCristianA.Torres-Valencia,Hern´anF.Garc´ıa-Arias,MauricioA.´AlvarezL´opezandAlvaroA.Orozco-Guti´errezAbstract—MultimodalEmotionrecognition(MER)isanap-plicationofmachinelearningweredifferentbiologicalsignalsareusedinordertoautomaticallyclassifyadeterminedaffectivestate.MERsystemshasbeendevelopedfordifferenttypeofapplicationsfrompsychologicalevaluation,anxietyassessment,human-machineinterfacesandmarketing.Thereareseveralspacesofclassificationproposedinthestateofartfortheemotionrecognitiontask,themostknownarediscreteanddimensionalspacesweretheemotionsaredescribedintermsofsomebasicemotionsandlatentdimensionsrespectively.Theuseofdimensionalspacesofclassificationallowsahigherrangeofemotionalstatestobeanalyzed.ThemostcommondimensionalspaceusedforthispurposeistheArousal/Valencespacewereemotionsaredescribedintermsoftheintensityoftheemotionthatgoesfrominactivetoactiveinthearousaldimension,andfromunpleasanttopleasantinthevalencedimension.TheuseofphysiologicalsignalsandtheEEGiswellsuitedforemotionrecognitionduetothefactthatanemotionalstatesgeneratesresponsesfromdifferentbiologicalsystemsofthehumanbody.Sincetheexpressionofanemotionisadynamicprocess,weproposetheuseofgenerativemodelsasHiddenMarkovModels(HMM)tocapturededynamicsofthesignalsforfurtherclassificationofemotionalstatesintermsofarousalandvalence.ForthedevelopmentofthisworkaninternationaldatabaseforemotionclassificationknownasDatasetforEmotionAnalysisusingPhysiologicalsignals(DEAP)isused.TheobjectiveofthisworkistodeterminewhichofthephysiologicalandEEGsignalsbringsmorerelevantinformationintheemotionrecognitiontask,severalexperimentsusingHMMsfromdifferentsignalsandcombinationsofthemareperformed,andtheresultsshowsthatsomeofthosesignalsbringsmorediscriminationbetweenarousalandvalencelevelsastheEEGandtheGalvanicSkinResponse(GSR)andtheHeartrate(HR).I.INTRODUCTIONEmotionsarethereactionsorperceptionsthatapersonhasofanspecificsituation.Thisreactionsareobservableinthebehaviorofthesubjectandaffectdifferentbiologicalsystemsinthehumanbody[1].Havingthisinmind,multimodalemotionrecognitionisthetaskofassessingtheemotionalstateofanindividualusingdatafromdifferentbiologicalsignalsthatcanbeobtainedfromthesubject[2].Thenumberofsignalsthatarecombinedinordertoassesemotionsdeterminethemodeofthesystem,beingunimodalforonedatasourceandmultimodalemotionrecognitioniswhenseveralsourcesarecombined[3].TheauthorsC.A.Torres-Valencia,H.F.Garc´ıa-Arias,M.A.´AlvarezandA.A.Orozco-Guti´errezarewiththeResearchGroupofAutomatic,DepartmentofElectricalEngineering,FacultyofElectricalEngineering,UniversidadTecnol´ogicadePereira,Pereira,Colombia,{cristian.torres,hernan.garcia,malvarez,aaog}@utp.edu.coPleasantUnpleasantCalmExcitedArousalValenceHappyGladPleasedExcitedAmusedAstonishedAlarmedAfraidTenseAngryFrustratedDistressedMiserableSadDepressedBoredDroopyTiredSleepyRelaxedCalmSatisfiedContentFig.1.Arousal/ValencespacedescriptionofemotionsEmotionrecognitionhasbeenanimportantresearchfieldinrecentyears,combiningdifferentareasofknowledgesuchasneurology,psychologyandengineering.Theapplicationsofautomaticemotionrecognitionvaryfrommarketing,anxietytreatmenttocomplexhuman-machineinterfaces[4][5].Thecategoriesinwhichemotionscanbediscriminatedarediscreteclassificationspacesanddimensionalspaces.Discretespacesallowstheevaluationofafewbasicemotionsasfear,happi-ness,sadness,definedbyEckmanandaremoresuitableforunimodalsystems[6].Sincethereareseveralemotionalstatesthatcanbeassessed,adimensionalspaceallowsthedefinitionofahighernumberofemotionsfromthecombinationoflatentdimensionsthatallowsabetterunderstandingofthedifferentstates.Thearousalandvalencespacearethemostuseddimen-sionalspaceforemotionrecognitionwhichdefineemotionsintermsoftheactivationornonactivationandthepositivenessornegativenessofanemotionalstate[7].Thisdimensionsareusuallyusedfordescribingemotionsusingmachinelearningalgorithmsthatasseseachdimensionindependently[7].Clas-sificationandregressionmethodologiesarewellstudiedinthestateofartofthisfield.Theuseofthearousal/valencespaceallowsthedescriptionofbasicemotionsandseveralsubtleemotionalstatesthatcanberepresentedbyacombinationofthistwodimensions[5].Althoughtherearesomeemotionsthatcanbeoverlapped,themajorityofemotionalstatescanbewelldiscriminatedusingthisclassificationspace.Figure1showsanexampleoftherepresentationofemotionsfromthetwolatentdimensionarousalandvalence.GenerativemodelsasHiddenMarkovModels(HMM)areamachinelearningalgorithmthatallowstherepresentationofadynamicalsignalintostatesthatrepresentaparticularclass[8].978-1-4799-7666-9/14/$31.00c2014IEEESincetheemotionsareintendtobeassessedfrombiologicalsignals,theHMM’sarewellsuitedtothistaskforcapturingthedynamicsofthesignalsthatrepresentseachemotionalstate.TheuseofHMM’swithGaussianmixturesallowstheuseofprobabilistictheoryinordertoadjusttheparametersofanHMMintoaspecificclassificationproblem.Inrecentyears,thestudyofemotionshasderivedintheconstructionoffewdatabas