电子科技大学：《现代数字信号处理理论与算法 Modern theory and algorithm of digital signal processing》课程教学资源（课件讲稿）06 LS Method & RLS Algorithm

法国数学家勒让德于1806年首次发表最小二乘理论。 德国的高斯于1794年已经应用这一理论推算了谷神星的轨 道，但迟至1809年才正式发表。 CH6 LS Method RLS Algorithm RLS procedure is derived from least squares estimation theory. The LMS algorithm may then be viewed as a pruned version of the RLS algorithm. In contrast to the usual approach in which the LMS algorithm is derived from Wiener filter theory,based on steepest-descent optimization and instantaneous estimates of the process statistics

CH6 LS Method & RLS Algorithm RLS procedure is derived from least squares estimation theory. The LMS algorithm may then be viewed as a pruned version of the RLS algorithm. In contrast to the usual approach in which the LMS algorithm is derived from Wiener filter theory, based on steepest-descent optimization and instantaneous estimates of the process statistics. 法国数学家勒让德于1806年首次发表最小二乘理论。 德国的高斯于1794年已经应用这一理论推算了谷神星的轨 道，但迟至1809年才正式发表

Contents o S1.Introduction o S2.LS Method o S3.RLS algorithm o S4.Examples 2020-01-18 2

2020-01-18 2 Contents  S1. Introduction  S2. LS Method  S3. RLS algorithm  S4. Examples

S1.Introduction o 'given statistics'case Wiener filter theory:Probabilistic cost function statistical information on the stochastic processes involved is available. The Wiener-Hopf equations may be solved if the correlation matrix and cross-correlation vector are given. o‘given data'case In most applications,however,only data sequences are given,so that the process statistics have to be estimated from these data. Data based cost function. 2020-01-18 3

2020-01-18 3 S1. Introduction  ‘given statistics’ case  Wiener filter theory：Probabilistic cost function  statistical information on the stochastic processes involved is available.  The Wiener-Hopf equations may be solved if the correlation matrix and cross-correlation vector are given.  ‘given data’ case  In most applications, however, only data sequences are given, so that the process statistics have to be estimated from these data.  Data based cost function

Self-designing filter o The filter is supplemented with an adaptation algorithm, ● monitor the environment (process statistics) vary the filter transfer function accordingly. 2020-01-18 4

2020-01-18 4 Self-designing filter  The filter is supplemented with an adaptation algorithm,  monitor the environment (process statistics)  vary the filter transfer function accordingly

Detour:probabilistic machinery 0 In the 'given data'case,time averaging represents a practical means for the estimation of the process statistics (after invoking stationarity and ergodicity). o RLS algorithm may be derived starting from Wiener filter theory and employing time averaged estimates of the statistical parameters. o Viewing signals as realizations of stochastic processes,and then trying to estimate the corresponding process statistics is somewhat of a detour,which to some extent can be avoided. 2020-01-18 5

2020-01-18 5 Detour: probabilistic machinery  In the ‘given data’ case, time averaging represents a practical means for the estimation of the process statistics (after invoking stationarity and ergodicity).  RLS algorithm may be derived starting from Wiener filter theory and employing time averaged estimates of the statistical parameters.  Viewing signals as realizations of stochastic processes, and then trying to estimate the corresponding process statistics is somewhat of a detour, which to some extent can be avoided

Direct approach o Data based cost function. A valid alternative to the probabilistic detour leads to comparable results (e.g.the RLS procedure) without having to rely on all the probabilistic machinery (stationarity, ergodicity,etc.) 2020-01-18 6

2020-01-18 6 Direct approach  Data based cost function.  A valid alternative to the probabilistic detour  leads to comparable results (e.g. the RLS procedure)  without having to rely on all the probabilistic machinery (stationarity, ergodicity, etc.)

True adaptive filtering O F Batch-mode processing ● a complete batch of data is available to design the optimal filter (i.e.the least squares parameter estimation problem). o True adaptive filtering:RLS The algorithm starts from a set of initial conditions,which may correspond to complete ignorance about the environment, ● And then adapts itself while doing the filtering operation. 2020-01-18 7

2020-01-18 7 True adaptive filtering  Batch-mode processing  a complete batch of data is available to design the optimal filter (i.e. the least squares parameter estimation problem).  True adaptive filtering: RLS  The algorithm starts from a set of initial conditions, which may correspond to complete ignorance about the environment,  And then adapts itself while doing the filtering operation

S2.LS Method o The basic set-up of the filter o Data based cost function o LS estimation versus WF design o LS versus Orthogonal Principle o The sufficient-order problem o SVD solution o WLS,TLS,IRWLS... 2020-01-18 8

2020-01-18 8 S2. LS Method  The basic set-up of the filter  Data based cost function  LS estimation versus WF design  LS versus Orthogonal Principle  The sufficient-order problem  SVD solution  WLS, TLS, IRWLS…

(1)basic set-up of the filter o Similar to WF o But the filter is fed by true data sequences instead of stochastic processes o The filter is fed by an input sequence either the FIR case or the linear combiner case o The error signal is e欧=d-w吸=康-ufW 1≤k≤L. 2020-01-18 9

2020-01-18 9 (1) basic set-up of the filter  Similar to WF  But the filter is fed by true data sequences instead of stochastic processes  The filter is fed by an input sequence  either the FIR case or the linear combiner case  The error signal is

Matrix form ey d 吲 e d 吗 M 三 : : WN-1 eL dL 吲 W d U 2020-01-18 10

2020-01-18 10 Matrix form