亚马逊技术白皮书：AWS 高性能计算简介

AnIntroductiontoHighPerformanceComputingonAWSScalable,Cost-EffectiveSolutionsforEngineering,Business,andScienceAugust2015AmazonWebServices–AnIntroductiontoHighPerformanceComputingonAWSAugust2015Page2of22©2015,AmazonWebServices,Inc.oritsaffiliates.Allrightsreserved.NoticesThisdocumentisprovidedforinformationalpurposesonly.ItrepresentsAWS’scurrentproductofferingsandpracticesasofthedateofissueofthisdocument,whicharesubjecttochangewithoutnotice.CustomersareresponsibleformakingtheirownindependentassessmentoftheinformationinthisdocumentandanyuseofAWS’sproductsorservices,eachofwhichisprovided“asis”withoutwarrantyofanykind,whetherexpressorimplied.Thisdocumentdoesnotcreateanywarranties,representations,contractualcommitments,conditionsorassurancesfromAWS,itsaffiliates,suppliersorlicensors.TheresponsibilitiesandliabilitiesofAWStoitscustomersarecontrolledbyAWSagreements,andthisdocumentisnotpartof,nordoesitmodify,anyagreementbetweenAWSanditscustomers.AmazonWebServices–AnIntroductiontoHighPerformanceComputingonAWSAugust2015Page3of22ContentsAbstract4  Introduction4  WhatIsHPC?5  GridsandClusters7  AWideSpectrumofHPCApplicationsintheCloud8  MappingHPCApplicationstoAWSFeatures10  LooselyCoupledGridComputing10  TightlyCoupledHPC10  Data-IntensiveComputing11  FactorsthatMakeAWSCompellingforHPC12  ScalabilityandAgility12  GlobalCollaborationandRemoteVisualization13  ReducingorEliminatingRelianceonJobQueues13  FasterProcurementandProvisioning14  SampleArchitectures15  GridComputingintheCloud15  ClusterComputingintheCloud16  RunningCommercialHPCApplicationsonAWS17  SecurityandGovernanceforHPC17  World-ClassProtection18  Built-InSecurityFeatures18  Conclusion20  Contributors20  FurtherReading21  Notes22  AmazonWebServices–AnIntroductiontoHighPerformanceComputingonAWSAugust2015Page4of22AbstractThispaperdescribesarangeofhighperformancecomputing(HPC)applicationsthatarerunningtodayonAmazonWebServices(AWS).Youwilllearnbestpracticesforclouddeployment,forclusterandjobmanagement,andforthemanagementofthird-partysoftware.ThiswhitepapercoversHPCusecasesthatincludehighlydistributed,highlyparallelgridcomputingapplications,aswellasmoretraditionalclustercomputingapplicationsthatrequireahighlevelofnode-to-nodecommunications.WealsodiscussHPCapplicationsthatrequireaccesstovarioustypesofhighperformancedatastorage.Thiswhitepapercoverscostoptimization.Inparticular,wedescribehowyoucanleverageAmazonElasticComputeCloud(EC2)SpotInstances1andstorageoptionssuchasAmazonSimpleStorageService(S3),AmazonElasticBlockStore(EBS),andAmazonGlacierforincreasedperformanceandsignificantcostsavingswhenmanaginglarge,scalableHPCworkloads.IntroductionAmazonWebServices(AWS)provideson-demandscalabilityandelasticityforawidevarietyofcomputationalanddata-intensiveworkloads,includingworkloadsthatrepresentmanyoftheworld’smostchallengingcomputingproblems:engineeringsimulations,financialriskanalyses,moleculardynamics,weatherprediction,andmanymore.UsingtheAWSCloudforhighperformancecomputingenablespublicandprivateorganizationstomakenewdiscoveries,createmorereliableandefficientproducts,andgainnewinsightsinanincreasinglydata-intensiveworld.OrganizationsofallsizesuseAWS.GlobalenterprisesuseAWStohelpmanageandscaletheirproductdevelopmentandmanufacturingefforts,toevaluatefinancialrisks,andtodevelopnewbusinessinsights.ResearchandacademicinstitutionsuseAWStoruncalculationsandsimulationsatscalesthatwerepreviouslyimpractical,acceleratingnewdiscoveries.InnovativestartupsuseAWStodeploytraditionalHPCapplicationsinnewandinnovativeways,especiallythoseapplicationsfoundinscienceandengineering.AWSalsoAmazonWebServices–AnIntroductiontoHighPerformanceComputingonAWSAugust2015Page5of22providesuniquebenefitsforentirelynewcategoriesofapplicationsthattakeadvantageofthevirtuallylimitlessscalabilitythatcloudhastooffer.UsingAWS,youcanfocusondesign,simulation,anddiscovery,insteadofspendingtimebuildingandmaintainingcomplexITinfrastructures.AWSprovidesarangeofservices:fromvirtualserversandstoragethatyoucanaccesson-demand,tohigherlevelcomputinganddataservicessuchasmanageddatabasesandsoftwaredevelopmentanddeploymenttools.AWSalsoprovidesservicesthatenableclusterautomation,monitoring,andgovernance.WhatIsHPC?OnewaytothinkofHPCistocompareHPCrequirementstorequirementsforatypicalserver.HPCapplicationsrequiremoreprocessorcores–perhapsvastlymore–thanthecoresavailableinatypicalsingleserver,andHPCapplicationsalsorequirelargeramountsofmemoryorhigherstorageI/Othanisfoundinatypicalserver.MostHPCapplicationstodayneedparallelprocessing—eitherbydeployinggridsorclustersofstandardserversandcentralprocessingunits(CPUs)inascale-outmanner,orbycreatingspecializedserversandsystemswithunusuallyhighnumbersofcores,largeamountsoftotalmemory,orhighthroughputnetworkconnectivitybetweentheservers,andfromserverstohigh-performancestorage.Thesesystemsmightalsoincludenon-traditionalcomputeprocessing,forexampleusinggraphicalprocessingunits(GPUs)orotheracceleratorsattachedtotheservers.ThesespecializedHPCsystems,whendeployedatlargescale,aresometimesrefe