9无约束优化解析法

第二章无约束优化解析法SchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPU无约束极小化方法直接搜索法（不需要求函数的导数）解析搜索法最速下降法共轭梯度法牛顿法阻尼牛顿法变尺度法……坐标轮换法模式搜索法（步长加速法）方向加速法（Powell法）单纯形搜索法……2.1引言SchoolofEconomic&Management,NCEPU2.2最速下降法(梯度法)最速下降法是在迭代中选择负梯度方向作为搜索方向而形成的一种下降迭代算法。最速下降法的理论依据：定理：设函数在某点可微，则负梯度方向是函数在该点的最速（快）下降方向。XfXfSchoolofEconomic&Management,NCEPU最速下降法的基本步骤（解析搜索法）初始点0X精度检查Y停止迭代最速下降法基本原理:从初始点出发，沿着函数负梯度方向进行搜索，得到一个新的近似极小点，使目标函数值减小。重复上述步骤，直到近似点满足精度要求。kkkkkkXfxXXfp1kkTkkTkkXfXHXfXfXfN搜索规则:目标函数的负梯度方向为搜索方向,按最优步长确定kXfkXf)1(1kXSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPU牛顿法的基本结构（解析搜索法）牛顿法基本原理:运用目标函数的一阶、二阶导数，构造在近似点的二阶Taylor展开式，逼近目标函数。运用逼近函数在某点的梯度为零，可以得到新的近似点。重复上述步骤，直到近似点满足精度要求。)()(111牛顿方向kkkkkkkXfXHDXfXHXXSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPU解析搜索法的总结kXS无论是最速梯度法还是牛顿法，在搜索的过程中要用到目标函数的导数，相邻近似点的迭代公式为:对于搜索方向的不同确定得出不同的搜索方法。（1），为最速下降法；（2），为牛顿法；（3），为变尺度法（DFP法）……kkkkXShXX1kkXfXSkkkXfXHXS)(1kkkXfXHXS)(SchoolofEconomic&Management,NCEPUSchoolofEconomic&Management,NCEPU