英特尔研究院院长吴甘沙分享十大前沿技术

大数据的十个技术前沿 吴甘沙英特尔中国研究院 LegalDisclaimersINFORMATIONINTHISDOCUMENTISPROVIDEDINCONNECTIONWITHINTELPRODUCTS.NOLICENSE,EXPRESSORIMPLIED,BYESTOPPELOROTHERWISE,TOANYINTELLECTUALPROPERTYRIGHTSISGRANTEDBYTHISDOCUMENT.EXCEPTASPROVIDEDININTEL'STERMSANDCONDITIONSOFSALEFORSUCHPRODUCTS,INTELASSUMESNOLIABILITYWHATSOEVERANDINTELDISCLAIMSANYEXPRESSORIMPLIEDWARRANTY,RELATINGTOSALEAND/ORUSEOFINTELPRODUCTSINCLUDINGLIABILITYORWARRANTIESRELATINGTOFITNESSFORAPARTICULARPURPOSE,MERCHANTABILITY,ORINFRINGEMENTOFANYPATENT,COPYRIGHTOROTHERINTELLECTUALPROPERTYRIGHT.AMissionCriticalApplicationisanyapplicationinwhichfailureoftheIntelProductcouldresult,directlyorindirectly,inpersonalinjuryordeath.SHOULDYOUPURCHASEORUSEINTEL'SPRODUCTSFORANYSUCHMISSIONCRITICALAPPLICATION,YOUSHALLINDEMNIFYANDHOLDINTELANDITSSUBSIDIARIES,SUBCONTRACTORSANDAFFILIATES,ANDTHEDIRECTORS,OFFICERS,ANDEMPLOYEESOFEACH,HARMLESSAGAINSTALLCLAIMSCOSTS,DAMAGES,ANDEXPENSESANDREASONABLEATTORNEYS'FEESARISINGOUTOF,DIRECTLYORINDIRECTLY,ANYCLAIMOFPRODUCTLIABILITY,PERSONALINJURY,ORDEATHARISINGINANYWAYOUTOFSUCHMISSIONCRITICALAPPLICATION,WHETHERORNOTINTELORITSSUBCONTRACTORWASNEGLIGENTINTHEDESIGN,MANUFACTURE,ORWARNINGOFTHEINTELPRODUCTORANYOFITSPARTS.Intelmaymakechangestospecificationsandproductdescriptionsatanytime,withoutnotice.Designersmustnotrelyontheabsenceorcharacteristicsofanyfeaturesorinstructionsmarkedreservedorundefined.Intelreservestheseforfuturedefinitionandshallhavenoresponsibilitywhatsoeverforconflictsorincompatibilitiesarisingfromfuturechangestothem.Theinformationhereissubjecttochangewithoutnotice.Donotfinalizeadesignwiththisinformation.Theproductsdescribedinthisdocumentmaycontaindesigndefectsorerrorsknownaserratawhichmaycausetheproducttodeviatefrompublishedspecifications.Currentcharacterizederrataareavailableonrequest.ContactyourlocalIntelsalesofficeoryourdistributortoobtainthelatestspecificationsandbeforeplacingyourproductorder.CopiesofdocumentswhichhaveanordernumberandarereferencedinthisdocumentorotherIntelliterature,maybeobtainedbycalling1-800-548-4725,orgoto:清华⼤大数据产业联合会 LegalDisclaimersIntelprocessornumbersarenotameasureofperformance.Processornumbersdifferentiatefeatureswithineachprocessorfamily,notacrossdifferentprocessorfamilies:Goto:LearnAboutIntel®ProcessorNumbersIntel,Intellogo,andXeonaretrademarksofIntelCorporationintheU.S.and/orothercountries.ContactyourlocalIntelsalesofficeoryourdistributortoobtainthelatestspecificationsandbeforeplacingyourproductorder.Copyright©2014,IntelCorporation.Allrightsreserved.*Othernamesandbrandsmaybeclaimedasthepropertyofothers.指数时代：颠覆成为常态 4 t x 清华⼤大数据产业联合会 t+1 x2 清华⼤大数据产业联合会 摩尔定律：指数社会的“基因” 晶体管数性能成本功耗存储带宽像素…… 数据 5 清华⼤大数据产业联合会 大数据：指数社会的“蛋白质”50Billion=35ZB2020感知-互联-智能 数据化生存 Xf(,t)价值为大 清华⼤大数据产业联合会 大数据的技术挑战 7 35ZB2020f(,t)全集采样 实时和全时 ⻅见微知著 人与工具 清华⼤大数据产业联合会 十个技术前沿 8膨胀宇宙 巴别之难 数据有价 软硬兼施 多快好省 天下三分 分久必合 精益求精 人机消⻓长 智能之争 数据 计算 分析 IntelConfidential—DoNotForward1.膨胀宇宙 清华⼤大数据产业联合会 清华⼤大数据产业联合会 TB PB EB ZB YB DBMS MPP NewSQL NoSQL DFS Globally-DistributedDB ArrayDBMS 清华⼤大数据产业联合会 TB PB EB ZB YB DBMS MPP NewSQL NoSQL DFS Cache,in-memoryFS ErasureCoding 清华⼤大数据产业联合会 TB PB EB ZB YB DBMS MPP NewSQL NoSQL DFS Flashstorage 清华⼤大数据产业联合会 TB PB EB ZB YB DBMS MPP NewSQL NoSQL DFS Flashstorage DRAMstorage RDMA 清华⼤大数据产业联合会 HeterogeneousStorage TB PB EB ZB YB DBMS MPP NewSQL NoSQL DFS Flashstorage DRAMstorage 清华⼤大数据产业联合会 HeterogeneousStorage TB PB EB ZB YB DBMS MPP NewSQL NoSQL DFS Flashstorage DRAMstorage NVRAMstorage 清华⼤大数据产业联合会 TB PB EB ZB YB ？ Tapesystem HeterogeneousStorage DBMS MPP NewSQL NoSQL DFS Flashstorage DRAMstorage NVRAMstorage IntelConfidential—DoNotForward2.巴别之难 清华⼤大数据产业联合会 清华⼤大数据产业联合会 Datatalks,butindifferentlanguages DataCuration DataWrangling DB-hard 清华⼤大数据产业联合会 19Courtesy:MichaelStonebraker DataTamer 清华⼤大数据产业联合会 20Courtesy:MichaelStonebraker 清华⼤大数据产业联合会 21Courtesy:MichaelStonebraker 清华⼤大数据产业联合会 Activework 22Datainterpretationviaautomatedlearning§ Discoveryofstructure§ Entityresolution§ TransformationFrom(semi-)structuredtounstructuredDataandmetadataorganization§ ApacheUIMA(UnstructuredInformationManagementArchitecture) 清华⼤大数据产业联合会 IntelConfidential—DoNotForward3.数据有价 清华⼤大数据产业联合会 清华⼤大数据产业联合会 安全和定价 数据权利的定义安全• 系统的安全• 数据的安全• 数据使用的安全定价清华⼤大数据产业联合会 数据权利的定义 25数据拥有权数据隐私权数据许可权（撤销，转移）数据审计权数据分红权清华⼤大数据产业联合会 系统的安全 26Hadoop用户和服务鉴权（基于Kerberos）HDFS文件和数据块权限控制数据高度分布、去中心化场景下的安全：分布式安全架构§ 如blockchain 清华⼤大数据产业联合会 数据的安全 27静态数据安全§ 加密：HADOOP-10150§ 访问控制：ApacheAccumulo，Hbase动态数据安全§ 动态审计能力：数据泄露防护（DataLeakagePrevention）个人对数据的控制§ DoNotTrack，MAC地址混淆，个人数据删除§ OpenPDS（personaldatastore）清华⼤大数据产业联合会 数据的安全 28数据脱敏/匿名化§ 去标识符，但基于准标识符（