电子科技大学：《大数据分析与挖掘 Big Data Analysis and Mining》课程教学资源（课件讲稿）Lecture 5 Data Stream Mining

Lecture 5 Data Stream Mining

Lecture 5 Data Stream Mining

Outline ▣What is data stream? What is Concept Drift? Data stream classification Data stream clustering

 What is data stream?  What is Concept Drift?  Data stream classification  Data stream clustering Outline

Internet Surveillance SPAM SPAM FILTER Spam Filtering DATA Network Intrusion Industry STREAM Mobile Smart Phone Sensor *Note:some pictures derived from internet

DATA STREAM Internet Industry Surveillance *Note: some pictures derived from internet Sensor Network Intrusion Smart Phone Spam Filtering Mobile

Potential Applications Telecommunication calling records Business:credit card transaction flows Network monitoring and traffic engineering Financial market:stock exchange Engineering industrial processes:power supply manufacturing Sensor,monitoring surveillance:video streams,RFIDs ·Security monitoring Web logs and Web page click streams

Potential Applications • Telecommunication calling records • Business: credit card transaction flows • Network monitoring and traffic engineering • Financial market: stock exchange • Engineering & industrial processes: power supply & manufacturing • Sensor, monitoring & surveillance: video streams, RFIDs • Security monitoring • Web logs and Web page click streams

What is data stream? A data stream is a massive sequence of data objects which have some unique features: >One by One >Potentially Unbounded >Concept Drift data4 data3 data2 datal Data mining system Data stream

What is data stream? A data stream is a massive sequence of data objects which have some unique features:  One by One  Potentially Unbounded  Concept Drift data1 Data stream data4 data3 data2 Data mining system

Challenges Data Stream:(a)Infinite Length (b)Evolving Nature ◆Single Pass Handling ◆Memory Limitation ◆Low Time Complexity ◆Concept Drift

Challenges Data Stream: (a) Infinite Length (b) Evolving Nature  Single Pass Handling  Memory Limitation  Low Time Complexity  Concept Drift

What is concept drift? In predictive analytics and machine learning,the concept drift means that the statistical properties of the target variable, which the model is trying to predict,change over time in unforeseen ways. In a word,the probability distribution changes. ·Change in P(c) ·Change in P(X) ·Change in P(ClX)

What is concept drift？ In predictive analytics and machine learning, the concept drift means that the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways. In a word, the probability distribution changes. • Change in P(C) • Change in P(X) • Change in P(C|X)

Real concept drift vs.Virtual concept drift Original data Real concept drift Virtual drift ● p(yX)changes p(X)changes,but not p(ylX) P(C,IX)=P(C)P(XIC,) P(X)

Real concept drift vs. Virtual concept drift P(C ) P(X | C ) (C | X) P(X) i i P i 

Example:Concept-Drift Current hyperplane 0 O 0 0 0 O 0 6 0 00 0 00 8 000 8 000 0 O Previous hyperplane A data chunk Negative instance● Instances victim of concept-drift Positive instance o

Example: Concept-Drift Negative instance Positive instance A data chunk Current hyperplane Previous hyperplane Instances victim of concept-drift

1,Concept Drift Detection

1、 Concept Drift Detection