A Survey of Stream Processing Frameworks for Big Data

Abstract

— In recent years due to the acceleration in IoT (Internet-of-Things) and M2M (Machine-to-Machine)
communications streams are everywhere. Twitter streams, log streams, TCP streams click streams and event streams are
some good examples. Big data streaming applications need to process and analyze information in real-time. The
Map/Reduce model and its open source implementation Hadoop designed as a high fault-tolerant system for batch
processing and high throughput jobs. However, the Map/Reduce framework is not suitable real-time streaming
applications that require very low latency of response. Owing to the high demand for processing non -batch jobs such as
real-time and streaming jobs several big data frameworks have been developed or under developing. This paper presents
a survey of open source frameworks that support big data stream processing.