香港中文大学：《Probability and Statistics for Engineers》课程教学资源（课件讲稿）Limit Theorems

ENGG430Protaistics onnrs Chapter 5:Limit Theorems Instructor: Shengyu Zhang

Instructor: Shengyu Zhang

Content Markov and Chebyshev Inequalities The Weak Law of Large Numbers ■（ Convergence in Probability The Central Limit Theorem The Strong Law of Large Numbers

Content ◼ Markov and Chebyshev Inequalities ◼ The Weak Law of Large Numbers ◼ Convergence in Probability ◼ The Central Limit Theorem ◼ The Strong Law of Large Numbers

Background We will discuss fundamental issues related to the asymptotic behavior of sequences of random variables. Our principal context involves a sequence X1,X2,...of independent identically distributed (i.i.d.)random variables with mean w and variance o2

Background ◼ We will discuss fundamental issues related to the asymptotic behavior of sequences of random variables. ◼ Our principal context involves a sequence 𝑋1,𝑋2, … of independent identically distributed (i.i.d.) random variables with mean 𝜇 and variance 𝜎 2

Background Let Sn=X1+…+Xn be the sum of the first n of them. Limit theorems are mostly concerned with the properties of S and related random variables as n becomes very large

Background ◼ Let 𝑆𝑛 = 𝑋1 + ⋯ + 𝑋𝑛 be the sum of the first 𝑛 of them. ◼ Limit theorems are mostly concerned with the properties of 𝑆𝑛 and related random variables as 𝑛 becomes very large

Background Because of independence,we have var(Sn)=var(X1)+.+var(Xn)=no2 The distribution of S spreads out as n increases Thus S cannot have a meaningful limit. But the situation is different if we consider the sample mean Mn= X1++Xn Sn n n

Background ◼ Because of independence, we have var 𝑆𝑛 = var 𝑋1 + ⋯ + var(𝑋𝑛) = 𝑛𝜎 2 ◼ The distribution of 𝑆𝑛 spreads out as 𝑛 increases ◼ Thus 𝑆𝑛 cannot have a meaningful limit. ◼ But the situation is different if we consider the sample mean 𝑀𝑛 = 𝑋1+⋯+𝑋𝑛 𝑛 = 𝑆𝑛 𝑛

Background A quick calculation yields, E[Mn]u,var(Mn)= n The variance of M decreases to zero as n increases. Thus the bulk of the distribution of Mm must be very close to the mean u. This phenomenon is the subject of certain laws of large numbers

Background ◼ A quick calculation yields, 𝐄[𝑀𝑛] = 𝜇, var(𝑀𝑛) = 𝜎 2 𝑛 . ◼ The variance of 𝑀𝑛 decreases to zero as 𝑛 increases. ◼ Thus the bulk of the distribution of 𝑀𝑛 must be very close to the mean 𝜇. ◼ This phenomenon is the subject of certain laws of large numbers

Background The laws generally assert that the sample mean Mn converges to the true mean u. These laws provide a mathematical basis for the loose interpretation of an expectation EX]=八.… .. as the average of a large number of independent samples drawn from the distribution of X

Background ◼ The laws generally assert that the sample mean 𝑀𝑛 converges to the true mean 𝜇. ◼ These laws provide a mathematical basis for the loose interpretation of an expectation 𝐄[𝑋] = 𝜇 … ◼ … as the average of a large number of independent samples drawn from the distribution of 𝑋

Background We will also consider a quantity which is intermediate between Sn and Mn. Zm is defined as follows. 1. subtract nu from S,to obtain the zero-mean random variable Sm-nu 2.then divide by ovn,to form the random variable In= Sn -nu σVm

Background ◼ We will also consider a quantity which is intermediate between 𝑆𝑛 and 𝑀𝑛. ◼ 𝑍𝑛 is defined as follows. 1. subtract 𝑛𝜇 from 𝑆𝑛, to obtain the zero-mean random variable 𝑆𝑛 − 𝑛𝜇 2. then divide by 𝜎 𝑛, to form the random variable 𝑍𝑛 = 𝑆𝑛 − 𝑛𝜇 𝜎 𝑛

Background It can be seen that E[Zn]=0, var[Zn]1 Since the mean/variance of Zm remain unchanged as n increases,its distribution neither spreads,nor shrinks to a point. The central limit theorem is concerned with 0 the asymptotic shape of the distribution of Zn and asserts that Z,becomes the standard normal distribution

Background ◼ It can be seen that 𝐄 𝑍𝑛 = 0, var 𝑍𝑛 = 1 ◼ Since the mean/variance of 𝑍𝑛 remain unchanged as 𝑛 increases, its distribution neither spreads, nor shrinks to a point. ◼ The central limit theorem is concerned with ❑ the asymptotic shape of the distribution of 𝑍𝑛 ❑ and asserts that 𝑍𝑛 becomes the standard normal distribution

Application Limit theorems are useful for several reasons: (a)Conceptually.They provided an interpretation of expectations/probabilities in terms of a long sequence of identical independent experiments

Application ◼ Limit theorems are useful for several reasons: ◼ (a) Conceptually. They provided an interpretation of expectations/probabilities in terms of a long sequence of identical independent experiments