同济大学：支持向量机（PPT讲稿）Support Vector Machine（SVM）

Support vector Machine (svm) ING SHEN SSE TONGJIUNIVERSITY SEP.2016

Support Vector Machine (SVM) Y I NG SH EN SSE, TO NG JI UNI VERSITY SEP. 2 0 1 6

What is a vector SVM= Support VECTOR Machine If we define a point a(3, 4 )in r2 we can plot it like this There exists a vector between the origin and a 5 1/27/2021 PATTERN RECOGNITION

What is a vector? SVM = Support VECTOR Machine If we define a point A(3,4) in ℝ 2 we can plot it like this. There exists a vector between the origin and A. 1/27/2021 PATTERN RECOGNITION 2

The magnitude of a vector The magnitude or length of a vector x is written alland is called its norm 1/27/2021 PATTERN RECOGNITION

The magnitude of a vector The magnitude or length of a vector x is written 𝒙 and is called its norm 1/27/2021 PATTERN RECOGNITION 3

The direction of a vector The direction of a vector u=(u,u2)is the vector w=( u 2 u=4 0.8 a 20=1 05 5 COs()=2_3 0.6 lull 5 050.6 cos(a)=2 4 =0.8 1/27/2021 PATTERN RECOGNITION

The direction of a vector The direction of a vector u=(u1 ,u2 ) is the vector w=( 𝑢1 𝑢 , 𝑢2 𝑢 ) 1/27/2021 PATTERN RECOGNITION 4 cos 𝜃 = 𝑢1 𝑢 = 3 5 = 0.6 cos 𝛼 = 𝑢2 𝑢 = 4 5 = 0.8

The dot product if we have two vectors x and y and there is an angle b between them, their dot product is x·y=‖ xlllyllcos(0) Talking about the dot product x y is the same as talking about the inner product (x y(in linear algebra scalar product 1/27/2021 PATTERN RECOGNITION

The dot product if we have two vectors x and y and there is an angle θ between them, their dot product is : 𝒙 ∙ 𝒚 = 𝒙 𝒚 𝑐𝑜𝑠(𝜃) Talking about the dot product x∙y is the same as talking about ◦ the inner product ⟨x,y⟩ (in linear algebra) ◦ scalar product 1/27/2021 PATTERN RECOGNITION 5

The orthogonal projection of a vector Given two vectors x and y, we would like to find the orthogonal projection ot x onto y To do this we project the vector x onto y This give us the vector z 6 1/27/2021 PATTERN RECOGNITION

The orthogonal projection of a vector Given two vectors x and y, we would like to find the orthogonal projection of x onto y. To do this we project the vector x onto y This give us the vector z 1/27/2021 PATTERN RECOGNITION 6

The orthogonal projection of a vector y detinItion COs(6)=-→|2‖=| xcos(6 We have x·y coS(0) llllll x·y元·y z x Ixlllyll lyll If we define the vector u as the direction of y then y x·y 小=∥m=a·x 1/27/2021 PATTERN RECOGNITION

The orthogonal projection of a vector By definition We have If we define the vector u as the direction of y then 1/27/2021 PATTERN RECOGNITION 7 cos 𝜃 = 𝒛 𝒙 ⇒ 𝒛 = ‖𝒙‖cos(𝜃) cos 𝜃 = 𝒙 ∙ 𝒚 𝒙 𝒚 𝒛 = 𝒙 𝒙 ∙ 𝒚 𝒙 𝒚 = 𝒙 ∙ 𝒚 𝒚 𝒖 = 𝒚 𝒚 𝒛 = 𝒙 ∙ 𝒚 𝒚 = 𝒖 ∙ 𝒙

The orthogonal projection of a vector Since this vector is in the same direction as y, it has the direction u z →z=|l=(a·x)u It allows us to compute the distance between x and the line which goes through y X-Z 1/27/2021 PATTERN RECOGNITION

The orthogonal projection of a vector Since this vector is in the same direction as y, it has the direction u It allows us to compute the distance between x and the line which goes through y 1/27/2021 PATTERN RECOGNITION 8 𝒖 = 𝒛 𝒛 ⇒ 𝒛 = 𝒛 𝒖 = (𝒖 ∙ 𝒙)𝒖

The equation of the hyperplane You have learnt that an equation of a line is: y=ax b However when reading about hyperplane, you will often find that the equation of an hyperplane is defined by w'x=0 Inner product/dot product How does these two forms relate Note that y + is the same thing as x-b=0 Given two vectors w= a andx=x 1/27/2021 PATTERN RECOGNITION 9

The equation of the hyperplane You have learnt that an equation of a line is : y = ax + b However when reading about hyperplane, you will often find that the equation of an hyperplane is defined by 𝒘𝑇𝒙 = 0 1/27/2021 PATTERN RECOGNITION 9 How does these two forms relate ? Note that y = ax + b is the same thing as y – ax – b = 0 Given two vectors 𝒘 = −𝑏 −𝑎 1 and 𝒙 = 1 𝑥 𝑦 Inner product/dot product

The equation of the hyperplane You have learnt that an equation of a line is: y=ax b However when reading about hyperplane, you will often find that the equation of an hyperplane is defined by w'x=o Inner product/dot product How does these two forms relate Wx=-b*1+(-a)*x+1*y y-ax-b The two equations are just different ways of expressing the same thing 1/27/2021 PATTERN RECOGNITION

The equation of the hyperplane You have learnt that an equation of a line is : y = ax + b However when reading about hyperplane, you will often find that the equation of an hyperplane is defined by 𝒘𝑇𝒙 = 0 1/27/2021 PATTERN RECOGNITION 10 How does these two forms relate ? 𝒘𝑇𝒙 = −𝑏 ∗ 1 + −𝑎 ∗ 𝑥 + 1 ∗ 𝑦 = 𝑦 − 𝑎𝑥 − 𝑏 The two equations are just different ways of expressing the same thing. Inner product/dot product