电子科技大学：《统计学习理论及应用 Statistical Learning Theory and Applications》课程教学资源（课件讲稿，英文版）Lecture 02 Review of Linear Algebra and Probability Theory

Statistical Learning Theory and Applications Lecture 2 Review of Linear Algebra and Probability Theory Instructor:Quan Wen SCSE@UESTC Fall 2021

Statistical Learning Theory and Applications Lecture 2 Review of Linear Algebra and Probability Theory Instructor: Quan Wen SCSE@UESTC Fall 2021

Outline (Level 1) ①Linear Algebra ②Probability Theory 1/130

Outline (Level 1) 1 Linear Algebra 2 Probability Theory 1 / 130

Topics: Basic concepts,properties and calculations of vectors o Basic concepts,properties and calculations of matrices o Basic concepts,properties and calculations of probability Basic concepts,properties and calculations of random variable and it's distributions Common probability distributions Key points and difficulties: o Key points:Main concepts and properties of linear algebra and probability theory,and common probability distributions involved in statistical learning Difficulties:Concept and understanding of random variables 2/130

Topics: Basic concepts, properties and calculations of vectors Basic concepts, properties and calculations of matrices Basic concepts, properties and calculations of probability Basic concepts, properties and calculations of random variable and it’s distributions Common probability distributions Key points and difficulties: Key points: Main concepts and properties of linear algebra and probability theory, and common probability distributions involved in statistical learning. Difficulties: Concept and understanding of random variables 2 / 130

Outline (Level 1) ○Linear Algebra Probability Theory 3/130

Outline (Level 1) 1 Linear Algebra 2 Probability Theory 3 / 130

1.Linear Algebra Linear algebra provides a way of compactly representing and operating on sets of linear equations.For example,consider the following system of equations: 4x1-5x2=-13 -2x1+3x2=9 In matrix notation. Ax=b where 4=()=()6=() 4/130

1. Linear Algebra Linear algebra provides a way of compactly representing and operating on sets of linear equations. For example, consider the following system of equations: 4x1 − 5x2 = −13 −2x1 + 3x2 = 9 In matrix notation, Ax = b where A =  4 −5 −2 3  , x =  x1 x2  , b =  −13 9  4 / 130

Outline (Level 1-2) ○Linear Algebra o Notations and Fundamental Concepts o Matrix Multiplication o Operational Properties o Linear Space o Quadratic Form and Positive Definite Matrix o Matrix Calculus 5/130

Outline (Level 1-2) 1 Linear Algebra Notations and Fundamental Concepts Matrix Multiplication Operational Properties Linear Space Quadratic Form and Positive Definite Matrix Matrix Calculus 5 / 130

Outline (Level 2-3) Notations and Fundamental Concepts o Matrix and Vector 6/130

Outline (Level 2-3) Notations and Fundamental Concepts Matrix and Vector 6 / 130

1.1.Notations and Fundamental Concepts 1.1.1.Matrix and Vector Matrix and Transpose:A E Rx",m rows,n columns d11 a12 d13 ain d21 a22 a23 a2n A= aml am2 am3 amn d11 a21 a31 aml AT= d12 a22 a32 am2 ain a2n a3n amn 7/130

1.1. Notations and Fundamental Concepts 1.1.1. Matrix and Vector Matrix and Transpose: A ∈ R m×n , m rows, n columns A =   a11 a12 a13 · · · a1n a21 a22 a23 · · · a2n . . . . . . . . . . . . . . . am1 am2 am3 · · · amn   A T =   a11 a21 a31 · · · am1 a12 a22 a32 · · · am2 . . . . . . . . . . . . . . . a1n a2n a3n · · · amn   7 / 130

Vector and Matrix:d-dimensional column vector x and its transpose x'or x: X1 X2 X= ,x=X=(1,x2,…,xa Xd Denote the j-th column of A by aj,4..or A.,j,a is column vector. A=[a1,a2,..,an. Denote the i-th row of A by af,4i.:or 4i.,af is row vector.Can be understand as the row version of the i-th column of 4 A= a】 8/130

Vector and Matrix: d-dimensional column vector x and its transpose x ′ or x T : x =   x1 x2 . . . xd   , x T = x ′ = (x1, x2, · · · , xd) ▶ Denote the j-th column of A by aj , A:, j or A∗, j , aj is column vector. A = [a1, a2, · · · , an] . ▶ Denote the i-th row of A by a T i , Ai,: or Ai,∗, a T i is row vector. Can be understand as the row version of the i-th column of A T A =      a T 1 a T 2 . . . a T m      8 / 130

Outline (Level 1-2) ○Linear Algebra o Notations and Fundamental Concepts Matrix Multiplication o Operational Properties o Linear Space o Quadratic Form and Positive Definite Matrix o Matrix Calculus 9/130

Outline (Level 1-2) 1 Linear Algebra Notations and Fundamental Concepts Matrix Multiplication Operational Properties Linear Space Quadratic Form and Positive Definite Matrix Matrix Calculus 9 / 130