西安电子科技大学：人工神经网络（PPT讲稿）Artificial Neural Networks（Introduction）

Artificial neural networks 人工神经网络 Introduction

Artificial Neural Networks 人工神经网络 Introduction

Table of contents · Introduction to ANNs taxonomy eatures Learning Applications · Supervised anns Unsupervised anNs Examples Examples Applications Applications Further topics Further topi II 02/02/2021 Artificial Neural Networks

02/02/2021 Artificial Neural Networks - I 2 Table of Contents • Introduction to ANNs – Taxonomy – Features – Learning – Applications I • Supervised ANNs – Examples – Applications – Further topics II • Unsupervised ANNs – Examples – Applications – Further topics III

Contents -I Introduction to anns Processing elements (neurons) Architecture Functional taxonomy of anns Structural Taxonomy of ANNs Features Learning Paradigms Applications 02/02/2021 Artificial Neural Networks

02/02/2021 Artificial Neural Networks - I 3 Contents - I • Introduction to ANNs – Processing elements (neurons) – Architecture • Functional Taxonomy of ANNs • Structural Taxonomy of ANNs • Features • Learning Paradigms • Applications

The Biological Neuron The synapse Soma Axon Terminal button Dendrite Nucleus Synap Ne euro gap transmitters Terminal buttons Dentate Schematic of biological neuron 10 billion neurons in human brain 10 billion synapses in human brain Summation of input stimuli Chemical transmission and Spatial (signals) modulation of signals emporal (pulses) · Inhibitory synapses Threshold over composed inputs · Excitatory synapses Constant firing strength 02/02/2021 Artificial Neural Networks

02/02/2021 Artificial Neural Networks - I 4 The Biological Neuron • 10 billion neurons in human brain • Summation of input stimuli – Spatial (signals) – Temporal (pulses) • Threshold over composed inputs • Constant firing strength • billion synapses in human brain • Chemical transmission and modulation of signals • Inhibitory synapses • Excitatory synapses 6 10

Biological Neural Networks CEREBRAL CORTEX 10,000 synapses per neuron Computational power Ib connectivity · Plasticity new connections strength of connections modified Fig. 204 -CaL TYPE IN LATERS IV-VI Oy VISUAL SENSORY CORTE Infant, Golgi Explanation in text. (Combined from fgures by Cajal.) 02/02/2021 Artificial Neural Networks

02/02/2021 Artificial Neural Networks - I 5 Biological Neural Networks • 10,000 synapses per neuron • Computational power = connectivity • Plasticity – new connections (?) – strength of connections modified

Neural dynamics mV 0 Action potential w 100mV -40 Activation threshold s 20-30mV Rest potential x-651 Spike time≈1-2ms Refractory time s 10-20ms 0102030405060708090100 02/02/2021 Artificial Neural Networks 6

02/02/2021 Artificial Neural Networks - I 6 Neural Dynamics -120 -100 -80 -60 -40 -20 0 20 40 0 10 20 30 40 50 60 70 80 90 100 ms mV membrane rest activation Refractory time Action potential Action potential ≈ 100mV Activation threshold ≈ 20-30mV Rest potential ≈ -65mV Spike time ≈ 1-2ms Refractory time ≈ 10-20ms

神经网络的复杂性 神经网路的复杂多样,不仅在于神经元和突触 的数量大、组合方式复杂和联系广泛,还在于 突触传递的机制复杂。现在已经发现和阐明的 突触传递机制有:突触后兴奋,突触后抑制, 突触前抑制,突触前兴奋,以及“远程”抑制 等等。在突触传递机制中,释放神经递质是实 现突触传递机能的中心环节,而不同的神经递 质有着不同的作用性质和特点 02/02/2021 Artificial Neural Networks

02/02/2021 Artificial Neural Networks - I 7 神经网络的复杂性 • 神经网路的复杂多样，不仅在于神经元和突触 的数量大、组合方式复杂和联系广泛，还在于 突触传递的机制复杂。现在已经发现和阐明的 突触传递机制有：突触后兴奋，突触后抑制， 突触前抑制，突触前兴奋，以及“远程”抑制 等等。在突触传递机制中，释放神经递质是实 现突触传递机能的中心环节，而不同的神经递 质有着不同的作用性质和特点

神经网络的研究 神经系统活动,不论是感觉、运动,还是脑的 高级功能(如学习、记忆、情绪等)都有整体 上的表现,面对这种表现的神经基础和机理的 分析不可避免地会涉及各种层次。这些不同层 次的研究互相启示,互相推动。在低层次(细 胞、分子水平)上的工作为较高层次的观察提 供分析的基础,而较高层次的观察又有助于引 导低层次工作的方向和体现其功能意义。既有 物理的、化学的、生理的、心理的分门别类研 究,又有综合研究 02/02/2021 Artificial Neural Networks

02/02/2021 Artificial Neural Networks - I 8 神经网络的研究 • 神经系统活动，不论是感觉、运动，还是脑的 高级功能（如学习、记忆、情绪等）都有整体 上的表现，面对这种表现的神经基础和机理的 分析不可避免地会涉及各种层次。这些不同层 次的研究互相启示，互相推动。在低层次（细 胞、分子水平）上的工作为较高层次的观察提 供分析的基础，而较高层次的观察又有助于引 导低层次工作的方向和体现其功能意义。既有 物理的、化学的、生理的、心理的分门别类研 究，又有综合研究

The Artificial neuron Stimulus alt x3(t a之w(n)x=fa( yi(t) Response ()=(uln+1() xat) Neuron i urest= resting potential it)=output of neuron j at time t Wi=connection strength between neuron i and neuron j u(t= total stimulus at time t 02/02/2021 Artificial Neural Networks

02/02/2021 Artificial Neural Networks - I 9 The Artificial Neuron ( ) =  ( ) j i ij j u t w x t Stimulus urest = resting potential xj (t) = output of neuron j at time t wij = connection strength between neuron i and neuron j u(t) = total stimulus at time t yi (t) x1(t) x2(t) x5(t) x3(t) x4(t) wi1 wi3 wi2 wi4 wi5 f( (t)) i i j wij  x j (t) y = u Neuron i y (t) f (u u (t)) i = rest + i Response

Artificial Neural models McCulloch Pitts-type Neurons(static) Digital neurons: activation state interpretation (Snapshot of the system each time a unit fires Analog neurons: firing rate interpretation (activation of units equal to firing rate Activation of neurons encodes information Spiking Neurons(dynamic) Firing pattern interpretation(spike trains of units Timing of spike trains encodes information (time to first spike, phase of signal, correlation and synchronic 02/02/2021 Artificial Neural Networks

02/02/2021 Artificial Neural Networks - I 10 Artificial Neural Models • McCulloch Pitts-type Neurons (static) – Digital neurons: activation state interpretation (snapshot of the system each time a unit fires) – Analog neurons: firing rate interpretation (activation of units equal to firing rate) – Activation of neurons encodes information • Spiking Neurons (dynamic) – Firing pattern interpretation (spike trains of units) – Timing of spike trains encodes information (time to first spike, phase of signal, correlation and synchronicity