能源部云计算模式和SDM的作用Beyond Workflows DOE Cloud Computi

yournamehereBeyondWorkflows-DOECloudComputingParadigmandtheSDMRoleandFutureMladenA.Vouk,NagizaSmatova,PaulBreimyer,PierreMoualem,MeiNagappan,andthewholeSPAteam(listavailableseparately)ScientificDataManagementCenter–ScientificProcessAutomationGroupNCStateUniversity,Raleigh,NC276951yournamehereOverview•ScientificWorkflowtechnology–Asuccessstoryfromthepast7yearsintheSDMcenter(atechnologyusedinproductionorotherwisebyapplicationpeople)–Developedcomponents:Workflows,Provenance,“Dashboard”,other•DOESDM“Cloud”-VisionforthefutureoftheSDMcentre–Integrationofcomponents-IntelligentAnalyticsandSocialNetworks,Component-based“cloud”,IntegratedServices(serviceorientedarchitecture)•Sustainablescience-LongtermapproachforthesurvivalofSDMcentertechnology(BeyondSciDACandlonger)–IntegrationofResearch,Engineering,Transfer-of-Technology,Partnerships,Results(ROI,TOC)2yournamehereScientificProcessAutomation•Akeydifferentiatingelementofasuccessfulinformationtechnology(IT)isitsabilitytobecomeatrue,valuable,andeconomicalcontributortocyberinfrastructure.•AnIT-assistedworkflowrepresentsaseriesofstructuredactivitiesandcomputationsthatariseininformationassistedproblemsolving.•Scientificprocessautomationprinciples,aswellasproductionlevelpilots,isSDM’sKeyContributionoverlast7years–SmokeyMountainsretreat.•FromNCState:numerouspublications,3graduatedPhDand4MSwiththesisstudents,severalinprogress,severalgenerationsofsoftware.3yournamehere4EnvironmentComputationsOrchestration(Kepler)Data,DataBasesProvenance…StorageAnalyticsControlPanels(Dashboard)&DisplayNetworkingLocal/Remote…“Cloud”ServicesAnalyticsAnalyticsComputationsOrchestration(Kepler)Data,DataBasesProvenance…Storageyournamehere5WorkflowFrameworkProvenance,Tracking&Meta-Data(DBsandPortals)ControlPlane(lightdataflows)ExecutionPlane(“HeavyLifting”Computationsandflows)SynchronousorAsynchronousKepleryournamehere66OutNetwork/”Cloud”Bsubcode_run------------wherecode_runisascript--------------code_run#!/bin/cshsource/usr/local/lsf/conf/cshrc.lsf#BSUB-W5#BSUB-n100mpiexec./code#BSUB-o/share/vouk/WFLOW/code.out.%J#BSUB-e/share/vouk/WFLOW/code.err.%J#BSUB-Jcodevouk-------------------------InActor/ProcessinaBroaderSenseyournamehere7ModularFrameworkSupercomputers+AnalyticsNodesKeplerDashStorageMeta-Dataabout:Processes,Data,Workflows,System,Apps&EnvironmentOrchestrationAuthDataStoreRecAPIDispAPIManagementAPIAccessTrustyournamehereReadMore…•SinghM.P.andM.A.Vouk,NetworkComputing,inJohnG.Webster(editor),EncyclopediaofElectricalandElectronicsEngineering,JohnWiley&Sons,NewYork,Vol.14,pp.114-132,1999•SKlasky,MBeck,VBhat,EFeibush,BLudäscher,MParashar,AShoshani,DSilverandMVouk,Datamanagementonthefusioncomputationalpipeline,SciDAC2005,JournalofPhysics:ConferenceSeries16(2005),510-520,doi:10.1088/1742-6596/16/1/070•IlkayAltintas,OscarBarney,ZhengangCheng,TerenceCritchlow,BertramLudaescher,SteveParker,ArieShoshaniandMladenVouk,Acceleratingthescientificexplorationprocesswithscientificworkflows,sciDAC2006,JournalofPhysics:ConferenceSeries46(2006),468-478,doi:10.1088/1742-6596/46/1/065•M.A.Vouk,I.AltintasR.Barreto,J.Blondin,Z.Cheng,T.Critchlow,A.Khan,S.Klasky,J.Ligon,B.Ludaescher,P.A.Mouallem,S.Parker,N.Podhorszki,A.Shoshani,C.Silva,AutomationofNetwork-BasedScientificWorkflows,Proc.oftheIFIPWoCo9onGrid-basedProblemSolvingEnvironemnts:ImplicationsforDevelopmentandDeploymentofNumericalSoftware,IFIPWG2.5onNumericalSoftware,Prescott,AZ,2006,printedinIFIP,Vol239,Grid-BasedProblemSolvingEnvironments,eds.GaffneyPWandPoolJCT(Boston:Springer),pp.35-61,2007•Klasky,S.;Barreto,R.;Kahn,A.;Parashar,M.;Podhorszki,N.;Parker,S.;Silver,D.;Vouk,M.A.Collaborativevisualizationspacesforpetascalesimulations,ProceedingsoftheCTS2008-InternationalSymposiumonCollaborativeTechnologiesandSystems,pp203-211,DigitalObjectIdentifier10.1109/CTS.2008.4543933,10-23May2008•More…•“Cloud”computing–buildsondecadesofresearchinvirtualization,distributedcomputing,utilitycomputing,grids,andmorerecentlynetworking,webandsoftwareservices.•Itimpliesaseamlessserviceorientedandcomponent-basedarchitecture-deliveryofanintegratedandorchestratedsuiteofon-demandfunctionstoanend-userthroughcompositionofbothlooselyandtightlycoupledfunctions,orservices-oftennetwork-based,reducedinformationtechnologyoverheadfortheend-user,serviceorchestration,virtualizationofresources,greatflexibility,reducedtotalcostofownership,different“flavors”.•IntelligentAnalyticsandKnowledge-CreatingSocialNetworks,Component-based“Clouds”,Seamless/IntegratedServices•NecessaryinthecontextofPeta-andExa-sciences,data,etc.9yournamehere10“AnalyticsCloudKnowledgecreation&Integration,SocialNetworking,Provenance,Tracking&Meta-Data(DBsandPortals)ExecutionPlane-“Heavyduty”in-cloudComputations,FlowsServicesW/FEngineConcept-drivenAnalyticsW/FGenerationWizardRun-timeManagerandSchedulerSynchronous&AsynchronousServicesWorkflowcontrolplaneAnalyticsEnabledResourcesSupercomputersClustersSupercomputersActiveStorageOther“cloud”devicesyournamehereComponents•Reusability(elementscanbere