Soli-雷达传感器

GoogleProjectSoliPRESENTER:WENGUANGMAOMotivationFreeairgesturesforHCIinwearable,mobile,ubiquitouscomputingVisionbasedapproachesLatencyObstructionPowerOtherRFbasedapproachesLargescalegestureProjectSoliGesturerecognitionsystemMicro-gesturesuchaspitchandrubMillimeterwave60GHzLowpower,smallformfactor300mW12mm*12mmProjectSoliSignalprocessingdesignRange-DopplerimageSoli:UbiquitousGestureSensingwithMillimeterWaveRadar,SIGGRAPH2016HardwaredesignSolisensor:asinglechipintegratingantennas,RFfrontend,A/Dconverter,VCOAHighlyIntegrated60GHz6-ChannelTransceiverWithAntennainPackageforSmartSensingandShort-RangeCommunications,IEEEJournalonSolid-StateCircuits,2016GestureclassificationdesignRandomforestCNN+RNNInteractingwithSoli:ExploringFine-GrainedDynamicGestureRecognitionintheRadio-FrequencySpectrum,UIST2016SystemOverviewHardwareHardwareSpecs#ofTx2#ofRx4Txpower2–5dBmRange10mCarrier60GHzSignalFMCW/DSSSBandwidth7GHzBeamwidth150degPowerconsump.300mWSignalModelScatteringcentermodelsSendFMCWchirporPNsequenceperiodicallyTreatuser’shandasmultiplereflectorsVariousdistanceVariousreflectionReceivedSamplesAllreflectedpathssuperimposeatthereceiverreceivedsamples𝑠𝑟𝑎𝑤(𝑡′)𝑡′=𝑡+𝑇⋅𝑝𝑝istheperiodfortransmittedsignals0𝑡𝑝,fasttime𝑇,slowtime𝑠𝑟𝑎𝑤𝑡′=𝑠𝑟𝑎𝑤(𝑡,𝑇)Frame:consistingofmultipleT40msGesture:consistingofmultipleframesFastTimeReceivedsamples𝑠𝑟𝑎𝑤(𝑡,𝑇)Preprocessing:mixingforFMCWorcorrelationforPNsequence𝑠𝑟𝑒𝑐𝑡,𝑇Given𝑇,𝑠𝑟𝑒𝑐𝑡,𝑇describesthepathdelayprofileat𝑇Fasttime𝑡Timeduringachirp/PNsequenceAssumeuser’shandisstationaryoverfasttimeFasttimereflectsthepropagationdelay(range)ofeachpathSlowTimeSlowtime𝑇TimeoverdifferentperiodsCapturethemotion(Dopplerfrequency)ofeachpathovertimeApplyFFTon𝑠𝑟𝑒𝑐(𝑡,𝑇)treating𝑡asaconstanttogetDopplerfrequencyforeachpathUsingsamplesineachframe𝑆(𝑡,𝑓)Range-DopplerImageTwopaths(duetodifferentpartsofthehand)areresolvableSeparationinrangeDifferenceinvelocityUseRange-DopplerImage(RDI)tocapturethefinger-leveldynamicsRange-DopplerImageDeriveRDI:𝑆𝑡,𝑓=𝑆2𝑟𝑐,2𝑣𝜆=𝑅𝐷(𝑟,𝑣)Row:DopplervelocityColumn:rangePixel:intensityofthepathwithspecificrangeandDopplervelocityconsiderRDIovertime:𝑅𝐷(𝑟,𝑣,𝑇)FeaturesRangeprofile:𝑅𝑃𝑟,𝑇=𝑅𝐷(𝑟,𝑣,𝑇)𝑣Dopplerprofile:D𝑃𝑟,𝑇=𝑅𝐷(𝑟,𝑣,𝑇)𝑟Velocityprofilecenter/anditsvariationovertimeVelocityprofiledispersionTotalinstantaneousenergy:ET=𝑅𝑃(𝑟,𝑇)𝑟anditsvariationovertimeFeaturesRange-DopplerImagematrixDownsampletoreducethedimensionalityAverageovermultiplechannelConsiderthevariationoverchannelsConsiderthevariationovertimeRawIQsamplesDerivativeovertimeSumofderivativeMaximumchannelangleFeaturesfasttime-frequencyspectrogram𝑆𝑃𝑡,𝑓,𝑇=𝑠𝑟𝑎𝑤𝑥,𝑇𝑒−𝑗2𝜋𝑓𝑥𝑡+𝑡𝑤𝑖𝑛𝑡𝑑𝑥DescribethespectrumofthereceivedsignalovertimeThree-dimensionalspatialprofileTreatuser’shandasasinglepointProvidedbybasicfunctionofradarsensorGestureSetsActiongesturesvssigngesturesGestureswithamotioncomponentHardtodescribeAssociateagesturetoatoolwhichrequiresthegesturetooperatePitchvsbuttonRubvsdialGestureClassificationFeaturevector:785elementsRandomForestclassifier:Effectiveformulti-classclassificationLowcomputationaltestingcostSmallmodelsizeGestureClassificationUsingtemporalfilteringLeveragetemporalcorrelationBayesianfilter𝑃𝑔𝑘𝑥~𝑃𝑥𝑔𝑘𝑃(𝑔𝑘)𝑃𝑔𝑘𝑇=𝑧𝑘𝑇𝜔𝑛𝑛𝑃(𝑥𝑇−𝑛|𝑔𝑘𝑇−𝑛)ResultsAccuracyPer-frame:73.6%(raw),78.2%(temporalfiltered)Per-gesture:86.9%(raw),92.1%(temporalfiltered)ComputationspeedSnapdragon400(QuadCortexA7at1.6GHz):2880gesturerecognitionpersecondRaspberryPi2(QuadCortexA7at900MHz):1480gesturerecognitionpersecondDeepLearningbasedGestureRecognitionConvolutionalNeuralNetwork(CNN)NomanualfeatureextractionprocedureUserawRDIastheinputUseCNNtolearnfeaturesautomaticallyRecurrentNeuralNetwork(RNN)UsepreviousresultsasthecurrentinputCapturethetemporalcorrelationwhenperformingagestureGradientvanishingandexplodingproblemUseLongshort-termmemory(LSTM)NetworkArchitectureGestureSetsTraining11gestures,10users,25timespergesture2750gesturesequencesNVIDIAGeForceTITANXGPUTrainingtime?Testingtime:150gesturerecognitionpersecond265MBGPUmemoryduringrunningtime689MBdiskneededtostorethetrainedmodelComparedApproachesStandaloneshallowCNNStandalonedeepCNNStandaloneRNNCNN+RNN(proposed)ClassificationAccuracyTemporalEvolutionofPerformancePairwiseClassificationGesture1(pairwise/entire)Gesture2(pairwise/entire)Pinchindexorpinky84.5%/67.7%75.9%/71.1%Swipefastorslow95.5%/84.8%92.2%/98.4%Pushandpull97.5%/98.6%59.0%/89.9%