Adaptive Scheduling on Power-Aware Managed Data-Ce

报告人：肖文华2013.9.26AdaptiveSchedulingonPower-AwareManagedData-CentersUsingMachineLearningAppearin12thIEEEInt.Conf.onGRIDComputing2011Authors:JosepLl.Berralet.al,France1IntroductionEnergySavinginCloudSelf-management:-Applythewell-knownconsolidationstrategyChallenges:–Howtodetermineoptimalplaceforajob/VM?–Requiredresourcesforthejob/VM?–ResultingQoSinthenewlocation?Contributions:–Mathematicmodelofthecloud/data-center–LearningandpredictingresourcesofjobsandQoSelements–SolvethemodelasBenefit-CostoptimizationTarget:CompromisebetweenenergyandQoS2Dividingtheproblemintoparts…Modelthedata-center–CreateamathematicmodeltorepresenttheData-CenterPredictrelationsandgoals–Relevantvariablesfordecisionmakingonlyavailableaposteriori–MLcreatesamodelfrompastexamplesSolvingthemodel–Complementthemodelwithlearnedrelations–Findappropriatealgorithmtooptimizethemodel3Modelingthedata-centerModelsummarization–Thegoal:Profit=Benefitsforrunningjobs/VMs–powercosts–Outputs:Scheduleoptimizingprofits–Constraints:maintainingtheconsistenceofDCandoperations–Parameters&variables:hosts,jobs/VMsandSLAsinDC3Modelingthedata-centerMathematicalModelUnknownvariablesTosettheseunknownvariablesPastmethod:humanexpertsThispaper:learningfromthepastbehavior(,())()revenueiiiifJobSLAJobJobSLAJob4LearningrelationsHowmuchCPUwilleachjob(webserviceinVM)demand?-Giventhepastbehavior(knownload,resourcesconsumedetc)-LearningLoadvsResourcesrelation:load~CPU,MEMM5Plearningalgorithm:decisiontree+LinReg-Useittofillparametersinthemodel!Howgoodwillthejob(webserviceinVM)behave?-Giventheresourcesassignedandhostcharacteristics-LearningContextvsResponseTime:givenCPU+CPUcontext~RTLinearRegressionalgorithm-UseittoadjustCPUassignationinsolvingtime!4LearningrelationsInputvariables:-Workload(numberofrequests,bytesperrequest,),CPUandMEMconsumed,CPUandMEMdemanded,CPUandMEMconsumedinthewholesystem,networkused,averagetimeperrequestconsumed,responsetimeOutPut:(1)(CPU,MEM)~WorkloadCPUcanberelateddirectlywithloadinagiventimeunit.Predictingthenecessarymemorymusttakeintoaccountnotonlythecurrentstateofthesystem,butalsoitspasthistory.(2)RT(Reponsetime)~(workload,cpuneeded,cpuobtained).Themostrelevantattributes:Fourloadattributes(requestspersecond,bytesperrequest,networkspeedandtimeperrequest),plusthetotalusedCPUfromphysicalmachine,thegivenCPUtotheVMandtherequiredCPUfromthewebservicejob5SolvingthemodelModeltobeIntegerLinearProgram（ILP）-Thefunctionsofrevenue,powercost,power,SLAarelinearfunctions,andthelearnedfunctionsofresourcesandresponsetimetobealsolinear.-Outputschedule[H,J]isintegerbinarymatrix-NP-completeproblemLinearFunctions(e.gGUROBIILPsolver)-Whileallfunctionsinvolvedarewrittenaslinear:ILP-Ifsolutionfound:wehaveanoptimalsolution-However,Solvingisexpensiveintime/space!Heuristicandapproximatealgorithms-Solvingthemodelusingfirst-fit/best-fitalgorithms-Solvingtimeandspacecanbeadjusted-itcanadjusttonon-linearfunctions-However,Optimalsolutioncanbemissed!(buthowmuchmissed?)5SolvingthemodelFirst-fit思想：在h个主机中依次寻找能处理该任务的主机，将任务分配给第一个找到的能处理该任务的主机，若h个主机都找完了也没有找到适应的主机，则等待其他任务执行完毕进行下一次寻找5SolvingthemodelBest-fit思想：给定当前任务，首先预测每个任务所需的最大与最小资源，然后以所需最小资源数降序排列。从序列中按序取出任务，对所有主机进行遍历，寻求使得收益最大的主机作为处理机。5ExperimentsData:Li-BCNworkload2010,acollectionoftracesfromdifferentrealhostedweb-sitesofferingservicesfromfilehostingtoforumservice.(webservice[Apachev2+php+MySQLV5])ExperimentEnvironments-Realhostingmachine:AIntelXeon4coresrunningat3GHZ,16GbRAM[UbuntuLinux10.10ServerEdition+VirtualBoxv3.1.8]-Simulateddatacenter:EEFSIMwith20machines,eachcontaining4processors-Parameters:Jobprice:0.17/our,Powerpricing:0.09euro/KWh,RT0=[0.004~0.008s],a=15ExperimentsHeuristicsvsILPSolver-Timetosolve:4secondsvs+4hoursofcompletesearch-Desc.orderedbest-fitisreallyclosetocompletesearch!(onPowerconsumption,SLAlevel,Profit)5ExperimentsDifferenttests:changejobrevenues,powercosts,SLAPowerCostMaxRT5ExperimentsAddinganewpolicy(migration)where-migrationprocessescanbeexpensiveintimeorproducesmalldowntimesinthemigratedservice-Modifiedmodel5ExperimentsAddinganewpolicy(migration)-Migration-awareversionvs.previousversionNoneMigrationMigration-aware5ExperimentsAddinganewpolicy(migration)-Sensitivitytopowercost6ConclusionsPresentedamathematicalmodelforthe“job×host”allocationproblemModeledjobsandsystembehaviors-InanautomaticmannerthroughMachineLearning-ImprovingdecisionmakerswithaprioriestimationfunctionsTestedthemodelagainstcompletesolversandfindingapproximatealgorithmsFuturework:-Extendtomultidimensionalresource(CPU,memory,disk,database,bandwidth....)-Extendtocloud(=manydatacenters)-……7DiscussionsProvideanewwaytoestimatetheinterveneusingbyHuangke?IsmachineLearningstrategyfeasibleforallkindsofjobs?-Webservicejobisusedinthispaper,-computingservices?withsameworkloadbutdifferentresponsetimeHowtomodeltherelationsofattributes?Thanksforyourlistening!