电子科技大学：《先进计算机网络技术》课程教学资源（课件讲稿）Unit 11 AI Enabled Wireless Access Control and Handoff

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC Unit 11 AI Enabled Wireless Access Control and Handoff Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.1

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC 11: AI Enabled Wireless Access Control and Handoff Page.1 Unit 11 AI Enabled Wireless Access Control and Handoff

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC Roadmap What is AI Why AI in Wireless Networks Reinforcement Learning at Mobile Network Edge 。 Case Study SMART Handoff Policy in mmWave HetNets Multi-Agent RL-Based Content Local Caching Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.2

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC 11: AI Enabled Wireless Access Control and Handoff Page.2 Roadmap • What is AI • Why AI in Wireless Networks • Reinforcement Learning at Mobile Network Edge • Case Study - SMART Handoff Policy in mmWave HetNets - Multi-Agent RL-Based Content Local Caching

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC What is AI AI:Artificial intelligence AI is the intelligence exhibited by machines or software. It is also the name of the academic field of study which studies how to create computers and computer software that are capable of intelligent behavior. ----from wikipedia Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.3

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC 11: AI Enabled Wireless Access Control and Handoff Page.3 What is AI AI: Artificial intelligence • AI is the intelligence exhibited by machines or software. • It is also the name of the academic field of study which studies how to create computers and computer software that are capable of intelligent behavior. ----from wikipedia

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC The begins of modern AI John McCarthy coined the term "artificial intelligence"as the topic of the Dartmouth Conference (1956),the first conference devoted to the subject. Pioneers of AI Shannon:Information Theory John McCarthy:coined the term "artificial intelligence",Turing Award Hyman Minsky:Optimization and Game Theory,Turing Award Herbert A.Simon:Decision Theory,Turing Award and The Nobel Prize Oliver Selfridge:Pattern recognition, Allen Newell:Game Theory and Cognitive Theory,Turing Award Holland:Genetic Algorithm Solomonoff:Algorithmic information theory Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.4

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC 11: AI Enabled Wireless Access Control and Handoff Page.4 The begins of modern AI • John McCarthy coined the term "artificial intelligence" as the topic of the Dartmouth Conference (1956), the first conference devoted to the subject. • Pioneers of AI - Shannon: Information Theory - John McCarthy: coined the term "artificial intelligence“, Turing Award - Hyman Minsky: Optimization and Game Theory, Turing Award - Herbert A. Simon: Decision Theory, Turing Award and The Nobel Prize - Oliver Selfridge: Pattern recognition, - Allen Newell: Game Theory and Cognitive Theory, Turing Award - Holland: Genetic Algorithm - Solomonoff: Algorithmic information theory

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC What is AI ·Tools used in AI - versions of search and mathematical optimization,neural networks and methods based on statistics,probability and economics ·AI fields: - computer science,mathematics,psychology,linguistics,philosophy, neuroscience,artificial psychology and many others AI evolves to multi-disciplinary techniques: machine learning optimization theory game theory control theory meta-heuristics Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.5

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC 11: AI Enabled Wireless Access Control and Handoff Page.5 What is AI • Tools used in AI - versions of search and mathematical optimization, neural networks and methods based on statistics, probability and economics • AI fields: - computer science, mathematics, psychology, linguistics, philosophy, neuroscience, artificial psychology and many others • AI evolves to multi-disciplinary techniques: - machine learning - optimization theory - game theory - control theory - meta-heuristics

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC Machine Intelligence Definition: Study of intelligent agents:any device that perceives its environment and takes actions that maximize its chance of success at some goal 。 Machine Learning belongs to one of the most important subfields in AI. Classification of ML algorithms Supervised Learning Supervised Unsupervised Unsupervised Learning Learning Learning Machine Reinforcement Learning Learning Reinforcement Learning Fig.classification of machine learning Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.6

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC 11: AI Enabled Wireless Access Control and Handoff Page.6 Machine Intelligence • Definition: • Study of intelligent agents: any device that perceives its environment and takes actions that maximize its chance of success at some goal • Machine Learning belongs to one of the most important subfields in AI. • Classification of ML algorithms - Supervised Learning - Unsupervised Learning - Reinforcement Learning Fig. classification of machine learning

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC Supervised Learning A supervised learning agent will be fed with "labeled"inputs and their desired outputs,and aims to determine a general rule that nicely maps inputs to outputs. e.g.,SVM algorithm,Wireless Channel Estimation 0 0 0 8 0 0 89 : 0 Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.7

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC Supervised Learning • A supervised learning agent will be fed with “labeled” inputs and their desired outputs, and aims to determine a general rule that nicely maps inputs to outputs. • e.g., SVM algorithm, Wireless Channel Estimation 11: AI Enabled Wireless Access Control and Handoff Page.7

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC Unsupervised Learning Unsupervised learning is the machine learning task of inferring a function to describe hidden structure from "unlabeled"inputs. e.g.,Clustering algorithm Cluster 0 Cluster 2 x-axs Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.8

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC Unsupervised Learning • Unsupervised learning is the machine learning task of inferring a function to describe hidden structure from "unlabeled" inputs. • e.g., Clustering algorithm 11: AI Enabled Wireless Access Control and Handoff Page.8

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC Reinforcement learning Inspired by control theory and behaviorist psychology,concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward in a Markov Decision Process(MDP). e.g.,Reinforcement learning algorithm Agent Action at Environment ⊙⊙ Reward rt State St+1 Reinforcement Learning Setup Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.9

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC Reinforcement learning • Inspired by control theory and behaviorist psychology, concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward in a Markov Decision Process (MDP). • e.g., Reinforcement learning algorithm 11: AI Enabled Wireless Access Control and Handoff Page.9

Feng Gang.Qin Shuang National Laboratory of Communication,UESTC Why AI in Wireless Networks Application Scenarios and Requirements:5G mobile networks Three use case families Extreme Mobile Broad Band (eMBB):extreme high throughputs and low-latency. - Massive Machine-Type Communications(mMTC):provides 100,000 wireless connections per AP. Ultra-reliable Low-Latency Communications (uRLLC):provides ultra- reliable low-latency and/or resilient communication links for network services. Advanced Topics in Computer Networks 11:AI Enabled Wireless Access Control and Handoff Page.10

Advanced Topics in Computer Networks Feng Gang, Qin Shuang National Laboratory of Communication, UESTC Why AI in Wireless Networks • Three use case families - Extreme Mobile Broad Band (eMBB)：extreme high throughputs and low-latency. - Massive Machine-Type Communications (mMTC): provides 100,000 wireless connections per AP. - Ultra-reliable Low-Latency Communications (uRLLC): provides ultra￾reliable low-latency and/or resilient communication links for network services. 11: AI Enabled Wireless Access Control and Handoff Page.10 Application Scenarios and Requirements: 5G mobile networks