05 BDTC2015Apache Kylin 大数据可视化实践

ApacheKylin大数据可视化实践仲俭(Jason,Zhong)[e-mail]|@ApacheKylinAboutme仲俭(Jason,Zhong)•SoftwareEngineer,eBayAnalyticsDataInfrastructure•ApacheKylinPMCApacheKylin简介ApacheKylinRESTAPI基于ApacheKylin的可视化ApacheKylin与Zeppelin的整合及二次开发FAQAgendaˈkiːˈlɪn/麒麟--n.(inChineseart)amythicalanimalofcompositeformExtremeOLAPEngineforBigDataApacheKylinisanopensourceDistributedAnalyticsEnginedesignedtoprovideSQLinterfaceandmulti-dimensionalanalysis(OLAP)onHadoopsupportingextremelylargedatasets,originalcontributedfromeBayInc.•于2014年10月正式开源•于2014年11月加入Apache孵化器•于2015年11月正式毕业成为Apache顶级项目@ApacheKylin…ANSISQL•Industry–百度地图–中国移动–京东–美团–唯品会–网易–Yahoo日本–Expedia–Tableau–Infoworks.io–…CaseCubeSizeRawRecordsSessionAnalysis30TB91+billionrowsTrafficAnalysis30TB28+billionrowsTransactionAnalysis560GB1.2+billionrows•eBay–2productionplatforms–20+usecases–90%ilequeries5seBay案例•DSSNous–NRTSEODashboard–NearReal-timeandHistoricaltogether•eCG–GADeepDive–PilotCase–Extremechallengeforstorageandbuildingtime–EnabledTableau•eCS–TrackingReport–HugestCube:91+Brawrecordsinonecube京东案例OLAP分析单个Cube最大维度16个，最大数据条数100亿，最大存储空间400G，95%的查询响应时间在15秒以内原始明细数据查询(京东已经贡献回社区)单个Cube最大维度8个，最大数据条数4亿，最大存储空间800G。30个Cube占用总空间4T左右，查询QPS在50左右，所有查询平均响应时间200ms，查询QPS在200左右平均响应时间可以保持在1s以内ApacheKylin简介ApacheKylinRESTAPI基于ApacheKylin的可视化ApacheKylin与Zeppelin的整合及二次开发FAQAgendaCubeListcubesRebuildcubeDisablecubePurgecubeEnablecubeRestAPIListJobResumejobDiscardjobGetjobsteplogQueryLoginQueryListqueryabletablesHiveMetadataGetHiveTableGetHiveTable(ExtendInfo)GetHiveTablesLoadHiveTableBuildJobPUT:BasicxxxxJD124xxxGFxxxSDFContent-Type:application/json;charset=UTF-8{startTime:0,endTime:1388563200000,buildType:BUILD}CubeNameApacheKylin简介ApacheKylinRESTAPI基于ApacheKylin的可视化ApacheKylin与Zeppelin的整合及二次开发FAQAgenda://kylin.ioHowtouseRestAPIinJavascriptApacheKylin简介ApacheKylinRESTAPI基于ApacheKylin的可视化ApacheKylin与Zeppelin的整合及二次开发FAQAgendaKylinInterpreterZeppelinWebpublicclassKylinInterpreterextendsInterpreter{publicInterpreterResultinterpret(….){…HttpClient…format…}}KylinServerSQLRESTRegistIntepreterPropertyParseDataFromFrontendUserKylinQueryApiandFormatZeppelinVisulizationApacheKylin简介ApacheKylinRESTAPI基于ApacheKylin的可视化ApacheKylin与Zeppelin的整合及二次开发FAQAgenda仲俭|[e-mail]微信公众号：ApacheKylin