Introduction to Convolution Neural Networks（CNN）and systems

ch,9: Introduction to convolution Neural Networks (CNN) and systems KH Wong

Ch. 9: Introduction to Convolution Neural Networks (CNN) and systems KH Wong ch9. CNN. V9b3 1

Overview Part 1 Al. theory of cnN A2 Feed forward details A2 Back propagation details Part B: CNN Systems Part c: cnn tools ch9. CNN. V9b3

Overview • Part 1 – A1. Theory of CNN – A2. Feed forward details – A2. Back propagation details • Part B: CNN Systems • Part C: CNN Tools ch9. CNN. V9b3 2

Introduction Very popular Toolboxes: tensorflow, cuda-convnet and caffe( user friendlier) a high performance Classifier(multi-class Successful in object recognition, handwritten optical character oCr recognition, image noise removal etc Easy to implementation Slow in learning Fast in classification ch9. CNN. V9b3

Introduction • Very Popular: – Toolboxes: tensorflow, cuda-convnet and caffe (user friendlier) • A high performance Classifier (multi-class) • Successful in object recognition, handwritten optical character OCR recognition, image noise removal etc. • Easy to implementation – Slow in learning – Fast in classification ch9. CNN. V9b3 3

Overview of this note Prerequisite: knowledge of fully connected Back Propagation Neural Networks (BPNN), in http://www.cse.cuhk.edu.hk//khwong/www2/cm sc5707 5707 08 neural net. pptx Convolution neural networks(Cnn) -Part a2 feed forward of cnn Part a3: feed backward of cnn ch9. CNN. V9b3

Overview of this note • Prerequisite: knowledge of Fully connected Back Propagation Neural Networks (BPNN), in – http://www.cse.cuhk.edu.hk/~khwong/www2/cm sc5707/5707_08_neural_net.pptx • Convolution neural networks (CNN) – Part A2: feed forward of CNN – Part A3: feed backward of CNN ch9. CNN. V9b3 4

Part A1 Theory of cnn Convolution Neural Networks

Part A.1 Theory of CNN Convolution Neural Networks ch9. CNN. V9b3 5

An example optical character recognition(OCR) Example test example CNn. m in http://www.mathworks.com/matlabcentral fileexchange /38310-deep-learning-toolbox Based on a data base(mnist uint8, from http://yann.lecun.com/exdb/mnist/) 60,000 training examples(e. g 28x28 pixels eac .0,000 testing samples a different dataset After training, given an unknown image, it willl tell whether it iso, or 1...9 etc https://towardsdatascience.com a-simple-2d-cnn-for-mnist-digit- recognition-a998dbcle79a ch9. CNN. V9b3

An example optical character recognition (OCR) • Example test_example_CNN.m in http://www.mathworks.com/matlabcentral /fileexchange/38310-deep-learning-toolbox • Based on a data base (mnist_uint8, from http://yann.lecun.com/exdb/mnist/) • 60,000 training examples (e.g. 28x28 pixels each) • 10,000 testing samples (a different dataset) – After training , given an unknown image, it will tell whether it is 0, or 1 ,..,9 etc. ch9. CNN. V9b3 6 https://towardsdatascience.com/ a-simple-2d-cnn-for-mnist-digit- recognition-a998dbc1e79a

The basic idea of convolution neural networks cnn Same idea as back-propagation- neural networks ( bpnn but different implementation After vectorized (vec), the 2d arranged inputs become 1D https://adeshpande3.github.io/adeshpande3.github.io/a-begInner%27s Guide- To-Understanding-Convolutional-Neural-Networks/ vectors then the network is just like a BPNN ch9. CNN. V9b3 (Back propagation neural networks

The basic idea of Convolution Neural Networks CNN Same idea as Back-propagation-neural networks (BPNN) but different implementation • ch9. CNN. V9b3 7 https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner%27s- Guide-To-Understanding-Convolutional-Neural-Networks/ After vectorized (vec), the 2D arranged inputs become 1D vectors. Then the network is just like a BPNN (Back propagation neural networks )

Basic structure of cnn The convolution layer: see how to use convolution for feature identifier ch9. CNN. V9b3

Basic structure of CNN The convolution layer: see how to use convolution for feature identifier ch9. CNN. V9b3 8

The basic structure Input conv subs. conV subs fully fully output Alternating Convolution(conv) and subsampling layer(subs Subsampling allows the features to be flexibly positioned ch9. CNN. V9b3

The basic structure • ch9. CNN. V9b3 9 Input conv. subs. conv subs fully fully output • Alternating Convolution (conv) and subsampling layer (subs) • Subsampling allows the features to be flexibly positioned

Convolution(conv) layer Example: From the input layer to the first hidden layer · The first hidden layer represents the filter outputs of a certain input neurons first hidden layer feature ·So, what is a feature? · Answer is in the next slide isualization of 5 x 5 filter convolving around an input volume and producing an activation map ch9. CNN. V9b3

Convolution (conv) layer: Example: From the input layer to the first hidden layer • The first hidden layer represents the filter outputs of a certain feature • So, what is a feature? • Answer is in the next slide ch9. CNN. V9b3 10