《Advanced Artificial Intelligence》课程PPT教学课件（高级人工智能）Lecture 5 Neural Networks

Advanced Artificial Intelligence Lecture 5: Neural networks

Advanced Artificial Intelligence Lecture 5: Neural Networks

Outline Perceptron Introduction Deep Neural Network Structure Backpropagation

2 Outline ▪ Perceptron Introduction ▪ Deep Neural Network Structure ▪ Backpropagation

Perceptron Introduction impulses carried toward cell body branches dendrites of axon axon axon nucleus terminals impulses carried away from cell body domestication domestication axon from a neuron se synapse cell body w1 ∑mx+b output axo activation u22 domestication Perceptron is Inspired by Neuron a diagram showing a perceptron updating It is a classifier its linear boundary as more training examples are added

3 Perceptron Introduction Perceptron is Inspired by Neuron. It is a classifier. A diagram showing a perceptron updating its linear boundary as more training examples are added

Single layer one-input Perceptron Input 0, Linear Combiner Limiter Output ∑ Y Threshold 0,0 OR a=Wx, b 8(r) 1,x≥0 1.5,b=-0.5 Y=6(a) 0,x<0 1

4 Single layer one-input Perceptron 𝑎 = 𝑤1𝑥1 + 𝑏 𝑌 = 𝜃(𝑎ሻ 𝜃(𝑥ሻ = ቊ 1, 𝑥 ≥ 0 0, 𝑥 < 0 𝑤1 = −1.5, 𝑏 = −0.5 𝑏

Single layer multi-input Perceptron Inputs L Hard Combiner Limiter Output Positive region 2·」 Threshold 1,w2=1b W1X1+W2x2+ b +a·E·x Y=(a) b=b+a·E Single layer perceptron is a linear E=Y-Y classifier A(x 0,x<0

5 Single layer multi-input Perceptron 𝑎 = 𝑤1𝑥1 + 𝑤2𝑥2 + 𝑏 𝑌 = 𝜃(𝑎ሻ 𝑤𝑖 = 𝑤𝑖 + 𝛼 ⋅ 𝐸 ⋅ 𝑥𝑖 𝑏 = 𝑏 + 𝛼 ⋅ 𝐸 𝐸 = 𝑌෠ − 𝑌 𝜃(𝑥ሻ = ቊ 1, 𝑥 ≥ 0 0, 𝑥 < 0 Single layer perceptron is a linear classifier 𝑏

Single Hidden layer multi-input Perceptron b A Multiple inputs, single hidden node perceptron Still a linear classifier, with a hyper-classify plane

6 Single Hidden layer multi-input Perceptron Multiple inputs, single hidden node perceptron. Still a linear classifier, with a hyper-classify plane

Non-linear activation Perceptron Inputs Linear sigmoid Combiner 2) Threshold o( is a non-linear activation function, sigmoid was the most popular one aEWx,tw,xat b o(y)= 1+e o(a

7 Non-linear activation Perceptron 𝑎 = 𝑤1𝑥1 + 𝑤2𝑥2 + 𝑏 𝑏 𝑠𝑖𝑔𝑚𝑜𝑖𝑑 𝑌 = 𝜎(𝑎ሻ

Non-linear activation Perceptron YA b A b With sigmoid activation function 8

8 Non-linear activation Perceptron With sigmoid activation function

Outline Perceptron Introduction Deep Neural Network Structure Backpropagation

9 Outline ▪ Perceptron Introduction ▪ Deep Neural Network Structure ▪ Backpropagation

Deep Neural Network One neuron(perceptron): Linear separation Multi-hidden layers non-linear One hidden layer: Realization of convex regions activation All the complex shapes Two hidden layers: Realization of non-convex regions 10

10 Deep Neural Network One neuron (perceptron): Linear separation One hidden layer: Realization of convex regions Two hidden layers: Realization of non-convex regions Multi-hidden layers non-linear activation: All the complex shapes