《深度自然语言处理》课程教学课件（Natural language processing with deep learning）17 Deep leanring Programing framework

西安交通大学Natural Language ProcessingwithDeepLearningXIANHAOTONGUNIVERSITYDeep leanringPrograming framework交通大学Chen Li2023

Chen Li 2023 Deep leanring Programing framework Natural Language Processing with Deep Learning

IntroductionDeep leanringPrograming frameworkCaffePyTorchmxnetMARYLINGTensorFlowKKerassPaddlePaddleChainerCNTKDL4JtheanoMatConvnetDEEPLEARNING

Introduction Deep leanring Programing framework

IntroductionDeep leanring Programing frameworkIn general, the deep learning framework provides a series of deeplearning components (for general algorithms,there will beimplementation). When a new algorithm needs to be used, it needsto be defined by the user, and then the function interface of the deeplearning framework is called to use the new algorithm defined by theuser.you can useexisting models,you can train the parameters of themodels yourselfyou can add your own layer to the existing models.you can select the classifier and optimization algorithm you need atthetop (youdon'tneedpropramthematscratch!)No framework is perfect, just as there may not be a set of buildingblocks that you need, so different frameworks may not fit in exactlythe sameareas

Introduction Deep leanring Programing framework In general, the deep learning framework provides a series of deep learning components (for general algorithms, there will be implementation). When a new algorithm needs to be used, it needs to be defined by the user, and then the function interface of the deep learning framework is called to use the new algorithm defined by the user. you can use existing models, you can train the parameters of the models yourself you can add your own layer to the existing models. you can select the classifier and optimization algorithm you need at the top ( you don't need propram them at scratch ! ) No framework is perfect, just as there may not be a set of building blocks that you need, so different frameworks may not fit in exactly the same areas

IntroductionNameReleaseOrganization Low-level Language Support LanguageGit starCaffeBVLCC++2013/927000+C++/Python/MatlabTensorflow2015/9GoogleC++/Python/Java等124000+C++/Python2017/1PytorchFacebookC/C++/PythonPython26000+C++Mxnet2015/5DMLCC++/Python/Julia/R等16000+Keras2015/3Google39600+PythonPython2016/8Baidu8300+PaddlepaddleC++/PythonC++/PythonCntk2014/7C++Microsoft15900+C++/Python/C#/.NET/Java2014/2VLFeatC/MatlabMatlab1100+Matconvnet2013/9EclipseC/C++/CudaJava/Scalar等10000+Deeplearning4jChainer2015/44600+Preferred networksPython/CythonPython2014/9Lasagne/theanoC/PythonPython3600+LasagneCcDarknet2013/9JosephRedmon

Introduction Name Release Organization Low-level Language Support Language Git star

IntroductionEach frame has different merits and demerits, choose the best one for you!NameReleaseOrganizationLow-level Language SupportLanguageGitstarCaffe2013/9BVLCC++27000+C++/Python/Matlab2015/9GoogleC++/Python/Java等124000+TensorflowC++/PythonPytorch2017/126000+FacebookC/C++/PythonPython2015/5DMLCC++MxnetC++/Python/Julia/R等16000+Keras2015/3GooglePythonPython39600+2016/8BaiduC++/Python8300+PaddlepaddleC++/PythonCntk2014/7C++MicrosoftC++/Python/C#/.NET/ava15900+2014/2VLFeatMatlab1100+MatconvnetC/Matlab2013/9Java/Scalar等Deeplearning4jEclipseC/C++/Cuda10000+4600+Chainer2015/4PythonPreferrednetworksPython/CythonLasagne/theano2014/9C/Python3600+LasagnePythoncc2013/9DarknetJosephRedmon

Introduction Name Release Organization Low-level Language Support Language Git star Each frame has different merits and demerits, choose the best one for you !

IntroductionEach frame has different merits and demerits, choose the best one for you!NameReleaseOrganization Low-level Language SupportLanguage GitstarCaffe2013/9BVLCC++27000+C++/Python/MatlabTensorflow2015/9GoogleC++/PythonC++/Python/Java等124000+Pytorch2017/126000+FacebookC/C++/PythonPython2015/5DMLCC++MxnetC++/Python/Julia/R等16000+Keras2015/3GooglePython39600+Python2016/8BaiduC++/Python8300+PaddlepaddleC++/PythonCntk2014/7C++MicrosoftC++/Python/C#/.NET/ava15900+2014/2VLFeatMatlab1100+MatconvnetC/Matlab2013/9Java/Scalar等Deeplearning4jEclipseC/C++/Cuda10000+4600+Chainer2015/4Preferred networksPythonPython/CythonLasagne/theano2014/9C/Python3600+LasagnePythoncc2013/9DarknetJosephRedmon

Introduction Name Release Organization Low-level Language Support Language Git star Each frame has different merits and demerits, choose the best one for you !

IntroductionEach frame has different merits and demerits, choose the best one for you!NameReleaseOrganization Low-level Language SupportLanguageGitstarCaffe2013/9BVLCC++C++/Python/Matlab27000+2015/9124000+TensorflowGoogleC++/PythonC++/Python/Java等Pytorch2017/126000+FacebookC/C++/PythonPythonC++Mxnet2015/5DMLC16000+C++/Python/Julia/R等Keras2015/3GooglePythonPython39600+2016/8Baidu8300+PaddlepaddleC++/PythonC++/PythonCntk2014/7C++MicrosoftC++/Python/C#/.NET/ava15900+2014/2VLFeatMatlab1100+MatconvnetC/Matlab2013/9Java/Scalar等Deeplearning4jEclipseC/C++/Cuda10000+4600+Chainer2015/4PythonPreferred networksPython/CythonLasagne/theano2014/9C/Python3600+LasagnePythoncc2013/9DarknetJosephRedmon

Introduction Name Release Organization Low-level Language Support Language Git star Each frame has different merits and demerits, choose the best one for you !

IntroductionEach frame has different merits and demerits, choose the best one for you!NameReleaseOrganization Low-level Language SupportLanguageGitstarCaffe2013/9BVLCC++C++/Python/Matlab27000+2015/9124000+TensorflowGoogleC++/PythonC++/Python/Java等Pytorch2017/126000+FacebookC/C++/PythonPython2015/5C++MxnetDMLCC++/Python/Julia/R等16000+Keras2015/3GooglePython39600+PythonPaddlepaddle2016/8Baidu8300+C++/PythonC++/PythonCntk2014/7C++MicrosoftC++/Python/C#/.NET/Java15900+2014/2VLFeatMatlab1100+MatconvnetC/Matlab2013/9Java/Scalar等Deeplearning4jEclipseC/C++/Cuda10000+4600+Chainer2015/4PythonPreferred networksPython/CythonLasagne/theano2014/9C/Python3600+LasagnePythoncc2013/9DarknetJosephRedmon

Introduction Name Release Organization Low-level Language Support Language Git star Each frame has different merits and demerits, choose the best one for you !

CaffeCaffeis oneoftheoldestframesCaffeMARYLINGIt's very robust, very fast.Caffeis5timeslessthankeras (withtheanobackend)insomeexperiments.The drawback of Caffe is that it's not flexible enough.> If you want to change it a little bit, you need to use C++ andCUDA Programming.Caffe'sdocumentationisverypoor.> You need to spend a lot of time checking the code tounderstand it?Cafeishardtoinstall> Itneedsto solvealot of dependencypackagesCaffeCaffe2.0Now, it is parts of Pytorch

Caffe l Caffe is one of the oldest frames. l It's very robust, very fast. l Caffe is 5 times less than keras (with theano backend) in some experiments. l The drawback of Caffe is that it's not flexible enough. Ø If you want to change it a little bit, you need to use C++ and CUDA Programming. l Caffe's documentation is very poor. Ø You need to spend a lot of time checking the code to understand it? l Cafe is hard to install. Ø It needs to solve a lot of dependency packages Caffe Caffe2.0 Now, it is parts of Pytorch

TheanotheanoTheano is another of theoldest and most stable librariesIn the beginning of deep learning, libraries you should use is either cafeortheano.> It supports automatic function gradient calculation, has PythoninterfaceandintegratesnumpyTheano is a lower level library.> more suitable for numerical optimization rather than deep learningToday, theano still works well> but it's beginning to be forgotten because it doesn't support multipleGPUsandhorizontalexpansion.Difficult debugging method> You have to wait until the function is compiled, which takes morethan an hour sometimes

Theano l Theano is another of the oldest and most stable libraries. l In the beginning of deep learning, libraries you should use is either cafe or theano. Ø It supports automatic function gradient calculation, has Python interface and integrates numpy l Theano is a lower level library. Ø more suitable for numerical optimization rather than deep learning l Today, theano still works well, Ø but it's beginning to be forgotten because it doesn't support multiple GPUs and horizontal expansion. l Difficult debugging method Ø You have to wait until the function is compiled, which takes more than an hour sometimes