香港中文大学：《Probability and Statistics for Engineers》课程教学资源（课件讲稿）Limit Theorems

ENGG40Proatatisticsfoner Chapter 5:Limit Theorems Instructor:Shengyu Zhang

Instructor: Shengyu Zhang

Content Markov and Chebyshev Inequalities The Weak Law of Large Numbers ■ Convergence in Probability The Central Limit Theorem The Strong Law of Large Numbers

Content  Markov and Chebyshev Inequalities  The Weak Law of Large Numbers  Convergence in Probability  The Central Limit Theorem  The Strong Law of Large Numbers

Background -We will discuss fundamental issues related to the asymptotic behavior of sequences of random variables. Our principal context involves a sequence X1,X2,...of independent identically distributed (i.i.d.)random variables with mean w and variance o2

Background  We will discuss fundamental issues related to the asymptotic behavior of sequences of random variables.  Our principal context involves a sequence 𝑋1, 𝑋2, … of independent identically distributed (i.i.d.) random variables with mean 𝜇 and variance 𝜎 2

Background Let Sn=X1+…+Xn be the sum of the first n of them. Limit theorems are mostly concerned with the properties of S and related random variables as n becomes very large

Background  Let 𝑆𝑛 = 𝑋1 + ⋯ + 𝑋𝑛 be the sum of the first 𝑛 of them.  Limit theorems are mostly concerned with the properties of 𝑆𝑛 and related random variables as 𝑛 becomes very large

Background Because of independence,we have var(Sn)=var(X1)+..+var(Xn)=no2 The distribution of s spreads out as n increases Thus S cannot have a meaningful limit. But the situation is different if we consider the sample mean Mn= X1+..+Xn Sn n n

Background  Because of independence, we have var 𝑆𝑛 = var 𝑋1 + ⋯ + var(𝑋𝑛) = 𝑛𝜎 2  The distribution of 𝑆𝑛 spreads out as 𝑛 increases  Thus 𝑆𝑛 cannot have a meaningful limit.  But the situation is different if we consider the sample mean 𝑀𝑛 = 𝑋1+⋯+𝑋𝑛 𝑛 = 𝑆𝑛 𝑛

Background A quick calculation yields, E[Mn]=u,var(Mn)= 02 The variance of M decreases to zero as n increases. Thus the bulk of the distribution of M must be very close to the mean u. This phenomenon is the subject of certain laws of large numbers

Background  A quick calculation yields, 𝐄[𝑀𝑛] = 𝜇, var(𝑀𝑛) = 𝜎 2 𝑛 .  The variance of 𝑀𝑛 decreases to zero as 𝑛 increases.  Thus the bulk of the distribution of 𝑀𝑛 must be very close to the mean 𝜇.  This phenomenon is the subject of certain laws of large numbers

Background The laws generally assert that the sample mean Mn converges to the true mean u. These laws provide a mathematical basis for the loose interpretation of an expectation E X=u... .. as the average of a large number of independent samples drawn from the distribution of X

Background  The laws generally assert that the sample mean 𝑀𝑛 converges to the true mean 𝜇.  These laws provide a mathematical basis for the loose interpretation of an expectation 𝐄[𝑋] = 𝜇 …  … as the average of a large number of independent samples drawn from the distribution of 𝑋

Background We will also consider a quantity which is intermediate between Sn and Mn. Zm is defined as follows. 1. subtract nu from S,to obtain the zero-mean random variable Sm-nu 2.then divide by ovn,to form the random variable Sn -nu In= 0√m

Background  We will also consider a quantity which is intermediate between 𝑆𝑛 and 𝑀𝑛.  𝑍𝑛 is defined as follows. 1. subtract 𝑛𝜇 from 𝑆𝑛, to obtain the zero-mean random variable 𝑆𝑛 − 𝑛𝜇 2. then divide by 𝜎 𝑛, to form the random variable 𝑍𝑛 = 𝑆𝑛 − 𝑛𝜇 𝜎 𝑛

Background -It can be seen that E[Zn]=0, var[Zn]=1 ■ Since the mean/variance of Zn remain unchanged as n increases,its distribution neither spreads,nor shrinks to a point The central limit theorem is concerned with 口 the asymptotic shape of the distribution of Zm and asserts that Z,becomes the standard normal distribution

Background  It can be seen that 𝐄 𝑍𝑛 = 0, var 𝑍𝑛 = 1  Since the mean/variance of 𝑍𝑛 remain unchanged as 𝑛 increases, its distribution neither spreads, nor shrinks to a point.  The central limit theorem is concerned with  the asymptotic shape of the distribution of 𝑍𝑛  and asserts that 𝑍𝑛 becomes the standard normal distribution

Application Limit theorems are useful for several reasons: ■ (a)Conceptually.They provided an interpretation of expectations/probabilities in terms of a long sequence of identical independent experiments

Application  Limit theorems are useful for several reasons:  (a) Conceptually. They provided an interpretation of expectations/probabilities in terms of a long sequence of identical independent experiments