离散信源的信息率失真函数

§4.2离散信源的信息率失真函数回顾上节所学信息率失真函数R(D)条件：1.已知信源概率分布函数2.已知失真函数3.满足保真度准则结论：)(xip),(yxjidDD)()(ln)()(min);(min)(11yxyxyxjijijninjippppYXIDR§4.2.1离散信源信息率失真函数的参量表示计算方法1.计算条件2.计算方法3.计算4.计算信息率失真函数5.参量S的物理意义)p()(uviji和、jvp1.计算条件（1）已知信源的概率分布（2）失真函数为平均互信息表达式（3）计算约束条件)(............)()(2211uuuuuusspppPUsjridvuji,,1,,,1),,()()(log)()();(11vuvuvujijijrisjippppVUIrisjjiijisjijijDdppripsjripvuuvuuvuv111),()()(),,2,1(1)(),,2,1)(,2,1(0)(2.计算方法应用拉格朗日乘子法，引入乘子S和，可将求条件极值问题化为无条件极值问题。即求解三个偏导的结果i0])();([)(1sjijiijuvuvpSDVUIp)()(log)()();(vuvuuvjijiijppppVUI),()(][)(vuuuvjiiijdSpSDpisjijiijuvuvpp])([)(1将三个偏导结果代入得为了方便，令3.计算利用关系})(exp{)},(exp{)()(uvuvuviijijijpSdpp})(exp{uiiip)},(exp{)()(vuvuvjijiijSdpp)()(uvvijijpp和、)()()(1)(11uvuvuvijriijsjijpppp和sjjijivuvSdp1)},(exp{)(1此式是关于的有s个方程的方程组，解之可得到S为参量的s个的值4.计算信息率失真函数平均失真度)},(exp{)()()()(11vuvuuvujijiriiijriiSdpppp1)},(exp{)()},(exp{)(11risijijjiivuvvuuSdpSdp)(vjp)(vjp亦可得到和)()(uvvijjpp),()()(11vuuvujiijrisjidppD)},(exp{),()()()(11vuvuvujijijirisjiSddppSD以下是以S为参量的信息率失真函数的表达式5.参量S的物理意义根据上式的结论，R(S)是D(S),S和的隐函数，因此利用全微分公式计算导数如下)(log)()()(log)()()},(exp{)(log)},(exp{)(11)()(111SSDpSSDppvjpvjuiSdvjpivjuiSdvjpirisjuipSRiriisjijiriiuuvuidDdSdSdpSDSdDdpdDdSSDSdDdSRdDdSSSRSDSRdDDdRiriiiiriiiiriiuu])()([)()()()()()()(111将式子对S求导将上式乘，并对j求和最后得1)},(exp{)(1vuujiiriiSdp0)},(exp{),()()},(exp{)(11vuSdvudupvuSddSdupjijiiriijiirii)(vjp0)(1)(1SDdSdpiiriiuSdDDdR)(R(D)及S与D的关系曲线0minDD1.当，2.当，参量S达到最大值，由于S取值为负，3.当※在处，参量S将从一个很小的负值跳跃到零，S在这一点不连续，而在内，S是失真度D的连续函数。SminDDmaxSmax0maxS0)(,0)(,maxdDDdRDRDDDDmax),0(maxD例1.r元对称信源，输入符号集为信源符号等概分布,,输出符号集为，失真函数为，，求信息率失真函数解:(1)R(D)的定义域(2)计算依题意，失真矩阵为},,,,{21uuuUrrupi1)(),,2,1(ri},,,{21vvvvr)(0)(1),(jijivudji),,2,1,(rji0),(min)(1minvudupDjijriirvudupDjiriij1)},()({min1maxi),,2,1(,ri0110),(),(),(),(1111vudvudvudvuddrrrr列方程组解得(3)计算解得rSSrSSrSSrrr)exp()exp()exp()exp()exp()exp(212121)exp()1(1Srri),,2,1(ri),,2,1()(rjvpjrSrvpSvpSvprSrSvpvpSvprSrSvpSvpvprrr)exp()1(1)()exp()()exp()()exp()1(1)exp()()()exp()()exp()1(1)exp()()exp()()(212121),,2,1(1)(rjrvpj(4)求D(S)从上式可以解得参量S(5)求R(D))1log()1(log)1log(log)1log()1log(log11loglog)1)(1(log})exp()1(1log{log)()(1DDDDrDrDDrDDDDrDrDDSrrSDupDRirii)exp()1(1)exp()1()()exp(11)exp()1(1),(exp),()()()(211SrSrrrSrrSrrvuSdvudvpupSDjijijirirji)1)(1(logDrDS§4.2.2离散信源信息率失真函数的迭代计算方法1.迭代计算条件2.迭代计算公式3.迭代计算步骤1.迭代计算条件设离散信源输入序列为输出序列为失真函数信息率失真函数R(D)是平均互信息I(U;V)的极小值满足的约束条件)()()(2121upupupuuuPUrr)()()(2121vpvpvpvvvPVsssjrivudji,,2,1,,,2,1),,(计算双重极小化2.迭代计算公式(1)计算为使I(U;V)达到极小值的试验信道，计算时，假设固定不变，与无关。),,2,1(),,2,1(),,2,1(),()()(1)(0)(111risjriDvuduvpupuvpuvprisjjiijisjijij)()(log)()(minmin);(minmin)(11)()()()(vpuvpuvpupuvvVUIuvvDRjijrisjijippppijjijj),,2,1;,,2,1)((*sjriuvpij)(*uvpij)(*uvpij),,2,1)((sjvpj)(uvpij引入参量S和，作辅助函数这样，就将条件极值化为无条件极值。设F1对的偏导为零：i0),()(1)()/(log)(ijiijijivuduSpvpuvpup0])/()()/(log)/()([)/(1111sjijriijijijrisjiijuvpSDvpuvpuvpupuvPsjijriijijijrisjisjijriiuvpSDvpuvpuvpupuvpSDVUIF111111)/()()/(log)/()()/();(1)/(uvpij利用对j求和整理得(2)计算为使I(U;V)达到极小值的信宿的概率分布)})(1(),()(exp{)()(*upvuduSpvpuivjpiijiij1)(1sjijuvp),,2,1(ri1)})(1(),()(exp{)(1upvuduSpvpiijiisjjsjjiijiivuduSpvpup1),()(exp)(1)})(1(exp{sjjijjijvuSdvpvuSdvpuivjp1*)],(exp[)()],(exp[)()()(*vpj),,2,1(sj)(*vpj计算时，把看成独立变量，即假设固定不变，与无关。为计算I(U;V)的条件极值，同样引入参量S和，作辅助函数设F2对的偏导为零)(*vpj)(*vpj),,2,1,,,2,1)((sjriuvpij)(vpjsjjjijijrisjisjjvpSDvpuvpuvpupvpSDVUIF11112)()()(log)()()();()(vpj0})()()(log)()({)(111sjjjijijrisjijvpSDvpuvpuvpupvp0)()()(1*riijivjpuvpup解出利用，将上式对j求和所以3.迭代计算步骤(1)假定为一相当大的负值，选定起始传递概率,可取,将选定的代入上式，得到(2)计算出(3)在通过得到(4)重复前面的步骤，计算出第k次和第k+1次的)()(1)(1*uvpupvjpijrii1)(1*vpjsj1)()(11uvpupijrisji)()()(1*uvpupvjpijriiS1),,2,1;,,2,1)(()1(sjriuvpijSuvpij1)()1()()1(uvpij)()2(uvpij)()2(uvpij)()2(uvpij)()2(vpj)()()1()(DRDRkk和(5)当与的差别小于预先给定的值,取或作为的近似值。(6)选略大一些的负数，重复上述过程，得到对应于的(7)重复上述过程，直到逼近于零(8)根据R(D)函数的S参量表述理论，进而得到R(D)曲线，完成整个迭代过程。)()(DRk)()1(DRk)()(DRk)()1(DRk)(1SRS2S2)(2SR)(maxSR