GTC-黄仁勋演讲PPT

LEAPSINVISUALCOMPUTINGJEN-HSUNHUANG,CO-FOUNDER&CEO|GTC2015FOURANNOUNCEMENTSANewGPUandDeepLearningAVeryFastBoxandDeepLearningRoadmapRevealandDeepLearningSelf-DrivingCarsandDeepLearningAMAZINGYEARINVISUALCOMPUTING©2015IndustrialLight&Magic.AllRightsReserved.10XGROWTHINGPUCOMPUTING2008150,000CUDADownloads4,000AcademicPapers60UniversitiesTeaching77SupercomputingTeraflops6,000TeslaGPUs27CUDAApps2008150,000CUDADownloads4,000AcademicPapers60UniversitiesTeaching77SupercomputingTeraflops6,000TeslaGPUs27CUDAApps20153MillionCUDADownloads10XGROWTHINGPUCOMPUTING20153MillionCUDADownloads10XGROWTHINGPUCOMPUTING319CUDAApps2008150,000CUDADownloads4,000AcademicPapers60UniversitiesTeaching77SupercomputingTeraflops6,000TeslaGPUs27CUDAApps20153MillionCUDADownloads800UniversitiesTeaching10XGROWTHINGPUCOMPUTING319CUDAApps2008150,000CUDADownloads4,000AcademicPapers60UniversitiesTeaching77SupercomputingTeraflops6,000TeslaGPUs27CUDAApps20153MillionCUDADownloads800UniversitiesTeaching10XGROWTHINGPUCOMPUTING319CUDAApps2008150,000CUDADownloads4,000AcademicPapers60UniversitiesTeaching77SupercomputingTeraflops6,000TeslaGPUs27CUDAApps60,000AcademicPapers20153MillionCUDADownloads800UniversitiesTeaching10XGROWTHINGPUCOMPUTING319CUDAApps2008150,000CUDADownloads4,000AcademicPapers60UniversitiesTeaching77SupercomputingTeraflops6,000TeslaGPUs27CUDAApps60,000AcademicPapers450,000TeslaGPUs20153MillionCUDADownloads60,000AcademicPapers800UniversitiesTeaching54,000SupercomputingTeraflops10XGROWTHINGPUCOMPUTING450,000TeslaGPUs319CUDAApps2008150,000CUDADownloads4,000AcademicPapers60UniversitiesTeaching77SupercomputingTeraflops6,000TeslaGPUs27CUDAApps8BillionTransistors3,072CUDACores7TFLOPSSP/0.2TFLOPSDP12GBMemoryTITANXTHEWORLD’SFASTESTGPU01234567TITANXFORDEEPLEARNINGTrainingAlexNetDays16-coreXeonCPUTITANTITANBlackcuDNNTITANXcuDNN~43…8BillionTransistors3,072CUDACores7TFLOPSSP/0.2TFLOPSDP12GBMemoryTITANXTHEWORLD’SFASTESTGPU$999FOURANNOUNCEMENTSANewGPUandDeepLearningAVeryFastBoxandDeepLearningRoadmapRevealandDeepLearningSelf-DrivingCarsandDeepLearningASHORTHISTORYOFDEEPLEARNINGConvolutionalNeuralNetworksforHandwrittenDigitalRecognitionLECUN,BOTTOU,BENGIO,HAFFNER,1998ImageNetClassificationwithNVIDIAGPUsKRIZHEVSKY,HINTON,ETAL.,201219952000200520102015Accuracy%2010201420122011201374%84%DNNCV72%“DelvingDeepintoRectifiers:SurpassingHuman-LevelPerformanceonImageNetClassification”—Microsoft:4.94%,Feb.6,2015“DeepImage:ScalingupImageRecognition”—Baidu:5.98%,Jan.13,2015“BatchNormalization:AcceleratingDeepNetworkTrainingbyReducingInternalCovariantShift”—Google:4.82%,Feb.11,2015IMAGENETCHALLENGEAccuracy%2010201420122011201374%84%DNNCV72%THEBIGBANGDEEPLEARNINGVISUALIZEDGPU-ACCELERATEDDEEPLEARNINGSTART-UPSDetectingMitosisinBreastCancerCells—IDSIAPredictingtheToxicityofNewDrugs—JohannesKeplerUniversityUnderstandingGeneMutationtoPreventDisease—UniversityofTorontoDEEPLEARNINGREVOLUTIONIZINGMEDICALRESEARCH“AutomatedImageCaptioningwithConvNetsandRecurrentNets”—AndrejKarpathy,Fei-FeiLiDIGITSDEEPGPUTRAININGSYSTEMFORDATASCIENTISTSDesignDNNsVisualizeactivationsManagemultipletrainingsUSERINTERFACEVisualizeLayersConfigureDNNProcessDataGPUGPUHWCloudGPUClusterMulti-GPUTheanoTorchMonitorProgressCaffecuDNN,cuBLASCUDAMonitorProgressDIGITSConfigureDNNProcessDataVisualizeLayersTestImageDIGITSDEVBOXWorld’sfastestGPUMaxGPUoutofaplugMulti-GPUtraining&inference“I’veneverseenAlexNetrunthisfast…TitanXisamonster,CrazyFast”DIGITSDEVBOX—EARLYRESULTS“DIGITSmakesitwayeasiertodesignthebestnetworkforthejob”0x1x2x3x4x124Multi-GPUscalingonTorchAlexNetVGG—SimonOsinderoA.I.Architech—SoumithChintalaResearchEngineerDIGITSDEVBOXAvailableMay2015$15,000FOURANNOUNCEMENTSANewGPUandDeepLearningAVeryFastBoxandDeepLearningRoadmapRevealandDeepLearningSelf-DrivingCarsandDeepLearningSGEMM/W2012201420082010201648361202460201872TeslaFermiKeplerMaxwellPascalMixedPrecision3DMemoryNVLinkVoltaGPUROADMAPPascal2xSGEMM/WFrameBufferCapacity(GB)2012201420082010201640301002050201860TeslaFermiKeplerMaxwellPascalMixedPrecision3DMemoryNVLinkVoltaGPUROADMAPPascal2.7xMemoryCapacityHGEMM/W201220142008201020169672240481202018144TeslaFermiKeplerMaxwellPascalMixedPrecision3DMemoryNVLinkVoltaGPUROADMAPPascal4xMixedPrecisionSTREAMGB/s2012201420082010201660045015003007502018900TeslaFermiKeplerMaxwellPascalMixedPrecision3DMemoryNVLinkVoltaGPUROADMAPPascal3xBandwidthPASCAL10XMAXWELLCONVOLUTIONFULLYCONNECTEDFULLYCONNECTEDCONVOLUTION(compute)(bandwidth)(bandwidth)(compute)WEIGHTUPDATE(interconnect)4x(FP16)6x6x4x10xMixedPrecision3DMemoryNVLINKforwardbackwardMixedPrecision3DMemory5x2x*VeryroughestimatesFOURANNOUNCEMENTSANewGPUandDeepLearningAVeryFastBoxandDeepLearningRoadmapRevealandDeepLearningSelf-DrivingCarsandDeepLearningTODAY’SADASPLANACTCPUWARNFPGACVASICSENSEBRAKENEXT-GENERATIONADASPLANACTCPUWARNFPGACVASICSENSEBRAKESTEERACCELERATENVIDIADRIVEPXSELF-DRIVINGCARCOMPUTERPLANACTCPUWARNFPGACVASICDNNSENSEBRAKESTEERIMAGENETCHALLENGEAccuracy%2010201420122011201374%84%DNNCV72%ACCELERATENVIDIADRIVEPXSELF-DRIVINGCARCOMPUTERPLANACTCPUWARNFPGACVASICDNNS