西安电子科技大学：《神经网络与模糊系统 Neural Networks and Fuzzy Systems》课程PPT课件讲稿（2003）02. Neuronal Dynamics——Activation Models（2/2）

Review 1.Neuronal Dynamical Systems We describe the neuronal dynamical systems by first- order differential or difference equations that govern the time evolution of the neuronal activations or membrane potentials

1.Neuronal Dynamical Systems We describe the neuronal dynamical systems by first￾order differential or difference equations that govern the time evolution of the neuronal activations or membrane potentials. ( , , ) ( , , )     X Y X Y y h F F x g F F = = Review

Review 4.Additive activation models 衣，=-4x,+∑S,(y)ni+1, 立，=-A,y,+∑S,(x)m,+J Hopfield circuit: i=1 1.Additive autoassociative model; 2.Strictly increasing bounded signal function (S>0); 3.Synaptic connection matrix is symmetric(M=M). Cx=R+s,m,+

4.Additive activation models   = = = − + + = − + + n i j j j i i ij j p j i i i j j ji i y A y S x m J x A x S y n I 1 1 ( ) ( )   Hopfield circuit: 1. Additive autoassociative model; 2. Strictly increasing bounded signal function ; 3. Synaptic connection matrix is symmetric . (S  0) ( ) T M = M = − + + j j j ji i i i i i S x m I R x C x ( ) Review

Review 5.Additive bivalent models x+1=∑S,Oy5)m:+1 y=∑S,(x)m,+1 Lyapunov Functions Cannot find a lyapunov function,nothing follows; Can find a lyapunov function,stability holds

5.Additive bivalent models   = + = + + + n i ij j k i i k j p j ji i k j j k i y S x m I x S y m I ( ) ( ) 1 1 Lyapunov Functions Cannot find a lyapunov function,nothing follows; Can find a lyapunov function,stability holds. Review

Review A dynamics system is stable,ifL≤O asymptotically stable,if <O Monotonicity of a lyapunov function is a sufficient not necessary condition for stability and asymptotic stability

A dynamics system is stable , if ; asymptotically stable, if . L  0  L  0  Monotonicity of a lyapunov function is a sufficient not necessary condition for stability and asymptotic stability. Review

Review Bivalent BAM theorem. Every matrix is bidirectionally stable for synchronous or asynchronous state changes. Synchronous:update an entire field of neurons at a time. ● Simple asynchronous:only one neuron makes a state- change decision. Subset asynchronous:one subset of neurons per field makes state-change decisions at a time

Bivalent BAM theorem. Every matrix is bidirectionally stable for synchronous or asynchronous state changes. • Synchronous:update an entire field of neurons at a time. • Simple asynchronous:only one neuron makes a state￾change decision. • Subset asynchronous:one subset of neurons per field makes state-change decisions at a time. Review

Chapter 3.Neural Dynamics II:Activation Models The most popular method for constructing M:the bipolar Hebbian or outer-product learning method binary vector associations:(4,,B i=1,2,…m bipolar vector associations:(XY 4=K,+ X,=2A-1 2002.10.8

2002.10.8 Chapter 3. Neural Dynamics II:Activation Models The most popular method for constructing M:the bipolar Hebbian or outer-product learning method binary vector associations: bipolar vector associations: ( , ) Ai Bi ( , ) Xi Yi i = 1,2, m [ 1] 2 1 Ai = Xi + Xi = 2Ai −1

Chapter 3.Neural Dynamics II:Activation Models The binary outer-product law: M=∑AB The bipolar outer-product law: M=∑XY k The Boolean outer-product law: M=田ABE m,=max a'b1,…,anbh) 2002.10.8

2002.10.8 Chapter 3. Neural Dynamics II:Activation Models The bipolar outer-product law: =  m k k T M X k Y The binary outer-product law: =  m k k T M Ak B The Boolean outer-product law: k T k m k M =  A B max( , , ) 1 1 j m i m i j mij = a b  a b

Chapter 3.Neural Dynamics II:Activation Models The weighted outer-product law: m M=∑wXiY Where∑w&=1 holds. In matrix notation: M=XWY Where XT=[X.Xm] Yr=[YI…lYm] W=Diagonal[w1,…,wm] 2002.10.8

2002.10.8 Chapter 3. Neural Dynamics II:Activation Models The weighted outer-product law: In matrix notation: Where holds. =  m k k T M wk X k Y  = m k wk 1 M X WY T = Where [ | | ] 1 T m T T X = X  X [ , , ] W = Diagonal w1  wm [ | | ] 1 T m T T Y = Y  Y

Chapter 3.Neural Dynamics II:Activation Models X3.6.1 Optimal Linear Associative Memory Matrices Optimal linear associative memory matrices: M=XY The pseudo-inverse matrix of: XYY-X X'XX=X X'Y-(XX XX=(XX') 2002.10.8

2002.10.8 Chapter 3. Neural Dynamics II:Activation Models Optimal linear associative memory matrices: M X Y * = XX X = X * * * * X XX = X T X X (X X) * * = T XX (XX ) * * = The pseudo-inverse matrix of X : * X ※3.6.1 Optimal Linear Associative Memory Matrices

Chapter 3.Neural Dynamics II:Activation Models X3.6.1 Optimal Linear Associative Memory Matrices Optimal linear associative memory matrices: The pseudo-inverse matrix of: If x is a nonzero scalar:x=1/x If x is a nonzero vector: X If x is a zero scalar or zero vector x*=0 For a rectangular matrix X,if ()exists: X"=X(XX) 2002.10.8

2002.10.8 Chapter 3. Neural Dynamics II:Activation Models ※3.6.1 Optimal Linear Associative Memory Matrices Optimal linear associative memory matrices: The pseudo-inverse matrix of X : * X If x is a nonzero scalar: x 1/ x * = If x is a zero scalar or zero vector : For a rectangular matrix , if exists: 0 * x = If x is a nonzero vector: T T xx x x = * 1 ( ) T − XX * 1 ( ) − = T T X X XX X