Stream Processing
Objectives
Learn the fundamentals, languages and systems for building application that process streams of data, ranging from general purpose distributed realtime stream processing systems to structured data models for dealing with streams.
General characterization
Code
11562
Credits
6.0
Responsible teacher
Nuno Manuel Ribeiro Preguiça, Sérgio Marco Duarte
Hours
Weekly - 4
Total - 50
Teaching language
Português
Prerequisites
Available soon
Bibliography
Opher Etzion and Peter Niblett. Event Processing in Action. Manning Publications, 2010.
Lukasz Golab and Tamer Özsu. Data Stream Management. Morgan and Claypool, 2010.
Bifet et al., (2018) Machine Learning for Data Streams, MIT Press
Several papers will be provided for further reading.
Teaching method
.
Evaluation method
?
Subject matter
Distributed Stream Processing Systems.
System models for stream processing: streams as sequences of mini-batches (e.g. Spark streaming); continuous processing (e.g. Apache Flink, Storm).
Programming models. System aspects: distribution, scalability and fault-tolerance.
Distributed time-series databases. Systems for IoT stream processing.
Data Stream Management Systems (DSMS).
Structured Data Models for Streams. Algebraic operators on stream and relations.
Continuous query languages (extensions to SQL and database management systems to deal with data streams).
Machine Learning for Streams.
Introduction to learning from data Dimensionality reduction for streams.
Learning under concept drift. Incremental learning.
Learning under imbalance and learning from graphs.
Programs
Programs where the course is taught: