香港中文大学：《Probability and Statistics for Engineers》课程教学资源（课件讲稿）Classical Statistical Inference

ENGG40Proatatisticsfoer Chapter 9:Classical Statistical Inference Instructor:Shengyu Zhang

Instructor: Shengyu Zhang

Preceding chapter:Bayesian inference Preceding chapter:Bayesian approach to inference. Unknown parameters are modeled as random variables. Work within a single,fully-specified probabilistic model. Compute posterior distribution by judicious application of Bayes'rule

Preceding chapter: Bayesian inference  Preceding chapter: Bayesian approach to inference.  Unknown parameters are modeled as random variables.  Work within a single, fully-specified probabilistic model.  Compute posterior distribution by judicious application of Bayes' rule

This chapter:classical inference We view the unknown parameter 0 as a deterministic (not random!)but unknown quantity. The observation X is random and its distribution px(x;0)if X is discrete fx(x;0)if X is continuous depends on the value of 0

This chapter: classical inference  We view the unknown parameter 𝜃 as a deterministic (not random!) but unknown quantity.  The observation 𝑋 is random and its distribution  𝑝𝑋 𝑥; 𝜃 if 𝑋 is discrete  𝑓𝑋 𝑥; 𝜃 if 𝑋 is continuous depends on the value of 𝜃

Classical inference Deal simultaneously with multiple candidate models,one model for each possible value of 0. A "good"hypothesis testing or estimation procedure will be one that possesses certain desirable properties under every candidate model. i.e.for every possible value of 0

Classical inference  Deal simultaneously with multiple candidate models, one model for each possible value of 𝜃.  A ''good" hypothesis testing or estimation procedure will be one that possesses certain desirable properties under every candidate model.  i.e. for every possible value of 𝜃

Bayesian: Prior pe Observation Posterior Pelx(X=)Point Estimatesi Process Calculation Error Analysis Conditional etc. Pxje Classical: px(;9) Point Estimates Observation x Hypothesis selection Process Confidence intervals 8 etc

 Bayesian:  Classical:

Notation Our notation will generally indicate the dependence of probabilities and expected values on 0. For example,we will denote by Ee[h()]the expected value of a random variable h()as a function of 0. Similarly,we will use the notation Pe()to denote the probability of an event A

Notation  Our notation will generally indicate the dependence of probabilities and expected values on 𝜃.  For example, we will denote by 𝐸𝜃 ℎ 𝑋 the expected value of a random variable ℎ 𝑋 as a function of 𝜃.  Similarly, we will use the notation 𝑃𝜃 𝐴 to denote the probability of an event 𝐴

Content Classical Parameter Estimation Linear Regression Binary Hypothesis Testing Significance Testing

Content  Classical Parameter Estimation  Linear Regression  Binary Hypothesis Testing  Significance Testing

■ Given observations (X1,...,Xn),an estimator is a random variable of the form 6=g(x),for some function g. Note that since the distribution of X depends on 0,the same is true for the distribution of 6. We use the term estimate to refer to an actual realized value of⑥

 Given observations 𝑋 = 𝑋1, … , 𝑋𝑛 , an estimator is a random variable of the form Θ ෡ = 𝑔 𝑋 , for some function 𝑔.  Note that since the distribution of 𝑋 depends on 𝜃, the same is true for the distribution of Θ ෡.  We use the term estimate to refer to an actual realized value of Θ ෡

Sometimes,particularly when we are interested in the role of the number of observations n.we use the notation,for an estimator. It is then also appropriate to view as a sequence of estimators. One for each value of n. The mean and variance of are denoted Ea[⑥n and vare[⑥nl,respectively. We sometimes drop this subscript 0 when the context is clear

 Sometimes, particularly when we are interested in the role of the number of observations 𝑛, we use the notation Θ ෡ 𝑛 for an estimator.  It is then also appropriate to view Θ ෡ 𝑛 as a sequence of estimators.  One for each value of 𝑛.  The mean and variance of Θ ෡ 𝑛 are denoted 𝐸𝜃 Θ ෡ 𝑛 and 𝑣𝑎𝑟𝜃 Θ ෡ 𝑛 , respectively.  We sometimes drop this subscript 𝜃 when the context is clear

Terminology regarding estimators Estimator:n,a function of n observations for an (X1,...,X)whose distribution depends on 0. Estimation error:n=n-0. Bias of the estimator:be(n)=EOn-0,is the expected value of the estimation error

Terminology regarding estimators  Estimator: Θ ෡ 𝑛, a function of 𝑛 observations for an 𝑋1, … , 𝑋𝑛 whose distribution depends on 𝜃.  Estimation error: Θ෩𝑛 = Θ ෡ 𝑛 − 𝜃.  Bias of the estimator: 𝑏𝜃 Θ ෡ 𝑛 = 𝐸𝜃 Θ ෡ 𝑛 − 𝜃, is the expected value of the estimation error