Optimal Stopping and Free Boundary Characterizatio

OptimalStoppingandFreeBoundaryCharacterizationsforsomeBrownianControlProblems.¤AmarjitBudhirajaandKevinRossUniversityofNorthCarolinaatChapelHillandStanfordUniversityMarch28,2007AbstractAsingularstochasticcontrolproblemwithstateconstraintsintwo-dimensionsisstudied.WeshowthatthevaluefunctionisC1anditsdirectionalderivativesarethevaluefunctionsofcertainoptimalstoppingproblems.Guidedbytheoptimalstoppingproblemwethenintroducetheassociatedno-actionregionandthefreeboundaryandshowthat,underappropriateconditions,anoptimallycontrolledprocessisaBrownianmotionintheno-actionregionwithre°ectionatthefreeboundary.ThisprovesaconjectureofMartins,ShreveandSoner[21]ontheformofanoptimalcontrolforthisclassofsingularcontrolproblems.AnimportantissueinouranalysisisthattherunningcostisLipschitzbutnotC1.Thislackofsmoothnessisoneofthekeyobstaclesinestablishingregularityofthefreeboundary.WeshowthatthefreeboundaryisLipschitzandiftheLipschitzconstantissu±cientlysmall,acertainobliquederivativeproblemontheno-actionregionadmitsauniqueviscositysolution.Thisuniquenessresultiskeyincharacterizinganoptimallycontrolledprocessasare°ecteddi®usionintheno-actionregion.1Introduction.Weconsiderasingularstochasticcontrolproblemwithstateconstraintsintwo-dimensions.Roughlyspeaking,bysingularcontrolonemeansthatthecontroltermsinthedynamicsofthestateprocessneednotbeabsolutelycontinuouswithrespecttoLebesguemeasure,andareonlyrequiredtohavepathsofboundedvariation.Stateconstraints,akeyfeatureofourproblem,referstotherequirementthatthecontrolleddi®usionprocesstakevaluesinsomepropersubsetSofIR2.Moreprecisely,inoursettingS=IR2+andthestateprocessisdescribedbytheequationX=x+B+Y,wherex2S,BisatwodimensionalBrownianmotionwithdriftµandcovariancematrix§andthecontrolYisanon-decreasing,rightcontinuouswithleftlimits(RCLL),adaptedprocess.Yis¤ResearchsupportedinpartbyAROgrantW911NF-04-1-0230andNSFgrantEMSW21-VIGRE0502385.AMS2000subjectclassi¯cations.93E20,60K25,60G40,49J30,49L25,35J60.Keywordsandphrases.SingularControl,StateConstraints,BrownianControlProblems,OptimalStopping,FreeBoundary,Obstacleproblems,ViscositySolutions,Hamilton-Jacobi-BellmanEquations,StochasticNetworks.1saidtobeanadmissiblecontrolifX(t)2Sforallt¸0.Associatedwitheachinitialstateandcontrolpolicyisanin¯nitehorizondiscountedcostJ(x;Y):=IEZ10e¡°t`(X(t))dt;(1)where°2(0;1)isthediscountfactorand`:S!IR+isaconvexfunctionofthefollowingform:Forz=(z1;z2)02S,`(z):=½®¢z;z2¸cz1;¯¢z;z2·cz1;(2)where®;¯2IR2,andc2(0;1).ThevaluefunctionV(x)isthein¯mumofJ(x;Y)overalladmissiblecontrols.Suchacontrolproblemanditsconnectionswithqueuingnetworksinheavytra±chasbeenstudiedbymanyauthors[12,21,5,8,23,18].Inageneralmultidimensionalsettingandwithamuchmoregeneralcostfunction,suchcontrolproblemswerestudiedin[1]and[6].In[1]thevaluefunctionwascharacterizedastheuniqueviscositysolutionofanappropriateHamilton-Jacobi-Bellman(HJB)equationwhereas[6]establishedtheexistenceofanoptimalcontrolbygeneralcompactnessarguments.Ourmaincontributioninthisworkistoprovideanexplicitrepresentationforanoptimalcontrolunderappropriateconditionson`.Explicitlysolvablesingularcontrolproblemsarequiterare.Inthefewexampleswhereexplicitsolutionsareavailableone¯ndsthatanoptimalcontroltakesthefollowingform.ThereisanopensetOinthestatespacesuchthatstartingfromwithin¹Onocontrolisapplieduntilthestatetrajectoryreachestheboundary@O,atwhichpointaminimalamountofpushisappliedalonganappropriatecontroldirectiontoconstrainthestateprocesswithin¹O.Furthermore,iftheinitialconditionisoutside¹O,aninstantaneousjumpoccursattime0thatbringstheprocessto@Oandsubsequently,controlisappliedasdescribedaboveconstrainingtheprocesswithin¹O.Inotherwords,anoptimallycontrolledprocessisare°ecteddi®usionon¹Owithanappropriate(possiblyoblique)re°ection¯eld.IntermsoftheassociatedHJBequation,inOthevaluefunctionsatis¯esalinearellipticPDEandinOcanonlinear¯rstorderPDEissatis¯ed;theboundary@O,separatingthesetworegions,isreferredtoasthefreeboundaryforthesystemofPDEs.Suchcharacterizationsforoptimalcontrolsofsingularcontrolproblemsintermsofadi®usionre°ectedatthefreeboundaryaresomeofthemostusefulandelegantresultsinthe¯eld.Foronedimensionalsettingstherehavebeenseveralworks(seeforexample,[3,14,11])thathaveusedthesocalledprincipleofsmooth¯ttoestablishtheC2propertyofvaluefunctionsofcertainsingularcontrolproblemsandthencharacterizethefreeboundaryandanoptimallycontrolledprocess.InmorethanonedimensiontheonlysuchresultsareduetoShreveandSoner[27,28].Asonemayexpect,suchresultsareintimatelytiedtoregularity(i.e.smoothness)propertiesofthefreeboundarywhichinturnhingesonsimilarpropertiesofthevaluefunctionofthecontrolproblem.Forexample,in[27]theauthorsconsideratwodimensionalsingularcontrolprobleminIR2(inparticulartherearenostateconstraints)withdynamicsgovernedbytheequationX=x+B+Y,whereBisatwodimensionalBrownianmotionandYisaRCLLcontrolwithpathsofboundedvariation.ThegoalistheminimizationofthecostIER[0;1)e¡t(`(Xt)dt+djYjt).Understrictconvexityof`andsuitablegrowthconditionsonits¯rsttwoderivatives,theauthors¯rstestablishusingideasofEvans[10]and