ICA——独立成分分析

IndependentComponentAnalysisContentWhatisICA?NongaussianityMeasurement—KurtosisICAByMaximizationofNongaussianityGradientandFastICAAlgorithmsUsingKurtosisMeasuringNongaussianitybyNegentropyFastICAUsingNegentrophyIndependentComponentAnalysisWhatisICA?MotivationExample:threepeoplearespeakingsimultaneouslyinaroomthathasthreemicrophones.Denotethemicrophonesignalsbyx1(t),x2(t),andx3(t).Theyaremixturesofsourcess1(t),s2(t),ands3(t).Thegoalistoestimatetheoriginalspeechsignalsusingonlytherecordedsignals.Thisiscalledthecocktail-partyproblem.111213212223311231231233233123()()()()(()()())()()()()xtxtxtaaaaastststststststsatasataTheCocktail-PartyProblemTheoriginalspeechsignalsThemixedspeechsignalsTheCocktail-PartyProblemTheoriginalspeechsignalsTheestimatedsourcesTheProblem)()()()()()()()()()()()(333232131332322212123132121111tsatsatsatxtsatsatsatxtsatsatsatxAsxFindthesourcess1(t),s2(t)ands3(t),andthecoefficientsaij’sfromtheobservedsignalsx1(t),x2(t),andx3(t).Itturnsoutthattheproblemcanbesolvedjustbyassumingthatthesourcessi(t)arenongaussianandstatisticallyindependent.)()()()()()()()()()()()(333232131332322212123132121111txbtxbtxbtstxbtxbtxbtstxbtxbtxbtsBxxAs1ApplicationsCocktailpartyproblem:separationofvoicesormusicorsoundsSensorarrayprocessing,e.g.radarBiomedicalsignalprocessingwithmultiplesensors:EEG,ECG,MEG,fMRITelecommunications:e.g.multiuserdetectioninCDMAFinancialandothertimeseriesNoiseremovalfromsignalsandimagesFeatureextractionforimagesandsignalsBrainmodellingBasicICAModelnitsatsatsatxniniii,,2,1),()()()(2211Mixingsignals(observable)Latentvariables)(1tx)(2tx),(21xxp1x2x)(1xp)(2xpAsxTheBasicAssumptionsTheindependentcomponentsareassumedstatisticallyindependent.Theindependentcomponentsmusthavenongaussiandistributions.Forsimplicity,weassumethattheunknownmixingmatrixAissquare.AsxAssumptionI:StatisticalIndependenceBasically,randomvariablesy1,y2,…,ynaresaidtobeindependentifinformationonthevalueofyidoesnotgiveanyinformationonthevalueofyjforij.Mathematically,thejointpdfisfactorizableinthefollowingway:p(y1,y2,…,yn)=p1(y1)p2(y2)…pn(yn)Notethatuncorrelatednessdoesnotnecessaryimplyindependence.AsxAssumptionII:NongaussianDistributionsNotethatinthebasicmodelwedonothavetoknowwhatthenongaussiandistributionsoftheICslooklike.AsxAssumptionIII:MixingMatrixissquareInotherwords,thenumberofindependentcomponentsisequaltothenumberofobservedmixtures.Thissimplifiesourdiscussioninthefirststage.However,inthebasicICAmodel,thisisnorestrictionaslongasoriginallythenumberofobservationsxiisatleastaslargeasthenumberofsourcessj.AsxAmbiguitiesofICAWecannotdeterminethevariances(energies)ofIC’s.ThisalsoimpliesE[x]=0(centeringofx)andsignofsiisunimportant.WecannotdeterminetheorderofIC’s.Asx11niiiiisxaTherefore,weassume1][2isE0][isEPsAPx1wherePisanypermutationmatrix.IllustrationofICAotherwisesspii03||321)(210105AMixingAsxWhiteningIsOnlyHalfofICAwhiteningVxzWhiteningMatrixAsxWhiteningIsOnlyHalfofICAVxz)(izpBywhitening,wehaveE[zzT]=I.This,however,doesn’timplyzi’sareindependent,i.e.,wemayhaveniiinzpzzzp121)(),,,(Uncorrelatednessisrelatedtoindependence,butisweakerthanindependence.IndependentComponentAnalysisVxzCentrallimittheoremimplicitlytellsusthattheadditiveofcomponents,makesthedistributiontobecome‘more’Gaussian.Therefore,nongaussianityisanimportantcriterionforICA.DegaussianishencethecentralthemeinICA.IndependentComponentAnalysisNongaussianityMeasurement—KurtosisMomentsdxxpxxEjjj)(][Thejthmoment:Mean:dxxpmxxEjxjj)()(])[(1Thejthcentralmoment:Variance:][1xEmx])[(222xxmxESkewness:])[()(33xmxExskewMomentGeneratingFunctionThemomentgeneratingfunctionMX(t)ofarandomvariableXisdefinedby:X~N(,2)Z~N(0,1)dxxpeeEtMtxtXX)(][)(2/22)(ttXeetM2/2)(tZetM!3][!2][!1][1][)(3322tXEtXEtXEeEtMtXXStandardNormalDistributionN(0,1)2/2)(tZetM0][12kZE!2!2][2kkZEkk!32!22!12136242ttt!3][!2][!1][1][)(3322tXEtXEtXEeEtMtXXZeroforalloddmoments1][2ZE3][4ZEKurtosisKurtosisofazero-meanrandomvariableXisdefinedbyNormalizekurtosis:224])[(3][)(XEXEXkurt3])[(][)(~224XEXEX0)(ZkurtGaussianityGaussianSupergaussianSubgaussian||2)(.,.xexpge2/21..,()2xegpxe],[,21)(.,.aaxaxpgeKurtosisforSupergaussian||2)(xexpConsiderLaplacianDistribution:dxexXEx||222][dxexx02220][XEdxexXEx||442][dxexx044242242324)(Xkurt4123)(~X0224])[(3][)(XEXEXkurtKurtosisforSupergaussian||2)(xexpConsiderLaplacianDistribution:dxexXEx||222][dxexx02220][XEdxexXEx||442][dxexx044242242324)(Xkurt4123)(~X0224])[(3][)(XEXEXkurt3])[(][)(~224XEXEXKurtosisforSubgassianConsiderUniformDistribution:dxxaXEaa2221][32a0][XEdxxaXE4421][54a224335)(aaXkurt1522a56)(~X0],[,21)(aaxaxp224])[(3][)(XEXEXkurt3])[(][)(~224XEXEXNongaussianityMeasurementByKurtosisKurtosis,orratherisabsolutevalue,hasbeenwidelyus