香港中文大学：《Probability and Statistics for Engineers》课程教学资源（PPT课件讲稿）Chapter 09 Classical Staistical Inference

ENGG2430A Probability and Statistics for Engineers lassical Instructor: Shengyu Zhang

Instructor: Shengyu Zhang

Preceding chapter: Bayesian inference Preceding chapter: Bayesian approach to inference a Unknown parameters are modeled as random variables a Work within a single, fully-specified probabilistic model a Compute posterior distribution by judicious application of Bayes rule

Preceding chapter: Bayesian inference ◼ Preceding chapter: Bayesian approach to inference. ❑ Unknown parameters are modeled as random variables. ❑ Work within a single, fully-specified probabilistic model. ❑ Compute posterior distribution by judicious application of Bayes' rule

This chapter: classical inference We view the unknown parameter 0 as a deterministic(not random! )but unknown quantit ity a The observation x is random and its distribution Px(x; 0)if X is discrete a x(x; 0)if X is continuous depends on the value of 0

This chapter: classical inference ◼ We view the unknown parameter 𝜃 as a deterministic (not random!) but unknown quantity. ◼ The observation 𝑋 is random and its distribution ❑ 𝑝𝑋 𝑥; 𝜃 if 𝑋 is discrete ❑ 𝑓𝑋 𝑥; 𝜃 if 𝑋 is continuous depends on the value of 𝜃

Classical inference Deal simultaneously with multiple candidate models, one model for each possible value of a"good "hypothesis testing or estimation procedure will be one that possesses certain desirable properties under every candidate model o i.e. for every possible value of 0

Classical inference ◼ Deal simultaneously with multiple candidate models, one model for each possible value of 𝜃. ◼ A ''good" hypothesis testing or estimation procedure will be one that possesses certain desirable properties under every candidate model. ❑ i.e. for every possible value of 𝜃

a Bayesian Prior pe HObservation H Posterior Pe(I X=r); Point Estimates i Process Calculation E rror Analysis Conditional et Pxe Classical Px(; I Point estimates Observation Hypothesis selection P rocess Confidence intervals i etc

◼ Bayesian: ◼ Classical:

Notation Our notation will generally indicate the dependence of probabilities and expected values on e For example, we will denote by egl(x)i the expected value of a random variable h(X)as a function of e Similarly, we will use the notation Pe(A)to denote the probability of an event A

Notation ◼ Our notation will generally indicate the dependence of probabilities and expected values on 𝜃. ◼ For example, we will denote by 𝐸𝜃 ℎ 𝑋 the expected value of a random variable ℎ 𝑋 as a function of 𝜃. ◼ Similarly, we will use the notation 𝑃𝜃 𝐴 to denote the probability of an event 𝐴

Content a Classical parameter estimation Linear regression Binary Hypothesis Testing Significance Testing

Content ◼ Classical Parameter Estimation ◼ Linear Regression ◼ Binary Hypothesis Testing ◼ Significance Testing

Given observationsⅩ=(1,…,!n),an estimator is a random variable of the form 0=g(X), for some function g Note that since the distribution of X depends on 0 the same is true for the distribution of o We use the term estimate to refer to an actual realized value of e

◼ Given observations 𝑋 = 𝑋1,… ,𝑋𝑛 , an estimator is a random variable of the form Θ ෡ = 𝑔 𝑋 , for some function 𝑔. ◼ Note that since the distribution of 𝑋 depends on 𝜃, the same is true for the distribution of Θ ෡. ◼ We use the term estimate to refer to an actual realized value of Θ ෡

Sometimes, particularly when we are interested in the role of the number of observations n, we use the notation o for an estimator a It is then also appropriate to view On as a sequence of estimators o One for each value of n a The mean and variance of e are denoted Een and vare on respectively a We sometimes drop this subscript 0 when the context is clear

◼ Sometimes, particularly when we are interested in the role of the number of observations 𝑛, we use the notation Θ෡𝑛 for an estimator. ◼ It is then also appropriate to view Θ෡𝑛 as a sequence of estimators. ❑ One for each value of 𝑛. ◼ The mean and variance of Θ ෡ 𝑛 are denoted 𝐸𝜃 Θ෡𝑛 and 𝑣𝑎𝑟𝜃 Θ෡𝑛 , respectively. ❑ We sometimes drop this subscript 𝜃 when the context is clear

Terminology regarding estimators Estimator:o a function of n observations for an(X1,., Xn)whose distribution depends on日. Estimation error: 0n=0n -8 Bias of the estimator: be(0n=Ee0n-0,is the expected value of the estimation error

Terminology regarding estimators ◼ Estimator: Θ ෡ 𝑛, a function of 𝑛 observations for an 𝑋1,… , 𝑋𝑛 whose distribution depends on 𝜃. ◼ Estimation error: Θ෩𝑛 = Θ ෡ 𝑛 − 𝜃. ◼ Bias of the estimator: 𝑏𝜃 Θ ෡ 𝑛 = 𝐸𝜃 Θ ෡ 𝑛 − 𝜃, is the expected value of the estimation error