西安电子科技大学：《神经网络与模糊系统 Neural Networks and Fuzzy Systems》课程PPT课件讲稿（2004）Chapter 02 ACTIVATIONS AND SIGNALS

ACTIVATIONS AND SIGNALS ◆ NEURONS AS FUNCTIONS SIGNAL MONOTONICITY ■ BIOLOGICAL ACTIVATIONS AND SIGNALS NEURON FIELDS NEURONAL DYNAMICAL SYSTEMS COMMON SIGNAL FUNCTION PULSE-CODED SIGNAL FUNCTION

ACTIVATIONS AND SIGNALS ◼ NEURONS AS FUNCTIONS ◼ SIGNAL MONOTONICITY ◼ BIOLOGICAL ACTIVATIONS AND SIGNALS ◼ NEURON FIELDS ◼ NEURONAL DYNAMICAL SYSTEMS ◼ COMMON SIGNAL FUNCTION ◼ PULSE-CODED SIGNAL FUNCTION

NEURONS AS FUNCTIONS Neurons behave as functions. Neurons transduce an unbounded input activation x(t) at time t into a bounded output signal S(x(t))

NEURONS AS FUNCTIONS Neurons behave as functions. Neurons transduce an unbounded input activation x(t) at time t into a bounded output signal S(x(t))

NEURONS AS FUNCTIONS S(x) X -00 0 + +00 Fig.1 s(x)is a bounded monotone-nondecreasing function of x If c+,we get threshold signal function (dash line) Which is piecewise differentiable

NEURONS AS FUNCTIONS S(x) x -∞ - 0 + +∞ Fig.1 s(x) is a bounded monotone-nondecreasing function of x If c→+∞，we get threshold signal function (dash line), Which is piecewise differentiable

NEURONS AS FUNCTIONS The transduction description:a sigmoidal or S-shaped curve the logistic signal function: S(x)= l+e-cx S'= dS =cS(1-S)>0 (c>0) dx The logistic signal function is sigmoidal and strictly increases for positive scaling constant c>0

NEURONS AS FUNCTIONS The transduction description: a sigmoidal or S-shaped curve the logistic signal function: cx e S x − + = 1 1 ( ) ' = = cS(1− S)  0 (c  0) dx dS S The logistic signal function is sigmoidal and strictly increases for positive scaling constant c >0

NEURONS AS FUNCTIONS S(x) X -00 0 + +00 Fig.1 s(x)is a bounded monotone-nondecreasing function of x If c+,we get threshold signal function (dash line) Which is piecewise differentiable

NEURONS AS FUNCTIONS S(x) x -∞ - 0 + +∞ Fig.1 s(x) is a bounded monotone-nondecreasing function of x If c→+∞，we get threshold signal function (dash line), Which is piecewise differentiable

NEURONS AS FUNCTIONS Swould transduce the four-neuron vector of activations (-6 350 49 -689)to the four- dimensional bit vector of signal(0 1 1 0) Zero activations to unity,zero,or the previous signal

NEURONS AS FUNCTIONS S would transduce the four-neuron vector of activations (-6 350 49 –689) to the four￾dimensional bit vector of signal (0 1 1 0) Zero activations to unity,zero,or the previous signal

SIGNAL MONOTONICITY In general,signal functions are monotone nondecreasing S'>=0. S(x) X -00 0 +00 This means signal functions have an upper bound or saturation value

SIGNAL MONOTONICITY In general, signal functions are monotone nondecreasing S’>=0. This means signal functions have an upper bound or saturation value. S(x) x -∞ - 0 + +∞

SIGNAL MONOTONICITY An important exception:bell-shaped signal function or Gaussian signal functions S(x)=e-cx2 c>0 S=-2cxe Soc-x The sign of the signal-activation derivation s'is opposite the sign of the activation x.We shall assume signal functions are monotone nondecreasing unless stated otherwise

SIGNAL MONOTONICITY An important exception: bell-shaped signal function or Gaussian signal functions 0 2 =  − S x e c cx ( ) S cxe S x cx = −  − − ' 2 , ' 2 The sign of the signal-activation derivation s’ is opposite the sign of the activation x. We shall assume signal functions are monotone nondecreasing unless stated otherwise

SIGNAL MONOTONICITY Generalized Gaussian signal function define potential or radial basis function S,(x): SU-ml2a2,门 input activation vector::x=(xi,…,xn)∈R" variance: mean vector: 4,=(4,,4) we shall consider only scalar-input signal functions:S,(x)

SIGNAL MONOTONICITY Generalized Gaussian signal function define potential or radial basis function : ( ) ] 2 1 ( ) exp[ 2 = − 2  − n j i j j i i S x x   n x = (x1 ,  , xn )R ( , , ) i n i i = 1   input activation vector: variance: mean vector: 2  i S (x) i we shall consider only scalar-input signal functions: ( ) i i S x

SIGNAL MONOTONICITY A property of signal monotonicity:semi-linearity Comparation: a.Linear signal functions: computation and analysis is comparatively easy; do not suppress noise. b.Nonlinear signal functions: increases a network's computational richness and facilitates noise suppression; risks computational and analytical intractability;

SIGNAL MONOTONICITY A property of signal monotonicity: semi-linearity Comparation: a. Linear signal functions: computation and analysis is comparatively easy; do not suppress noise. b. Nonlinear signal functions: increases a network’s computational richness and facilitates noise suppression; risks computational and analytical intractability;