Fast Analysis of Sensor Data over MapReduce using Spark
| Author(s) | : | Mansi Shah, Vatika Tayal |
| Institution | : | M. Tech. Scholar, Computer Science and Engineering Department, N.S.I.T, Jetalpur, Gujarat |
| Published In | : | Vol. 2, Issue 5 — May 2015 |
| Page No. | : | 1071-1075 |
| Domain | : | Engineering |
| Type | : | Research Paper |
| ISSN (Online) | : | 2348-4470 |
| ISSN (Print) | : | 2348-6406 |
Big data analysis is emerging rapidly due to the tremendous volume of data, velocity at which the data isflowing in the organizations and the variety of data. In recent years due to the spurt in Internet of Things (IoT), datagenerated by the sensors is growing exponentially thus transforming into big data. Thus data collection, processing andextracting useful information from such increasing high velocity and high volume of sensor data poses a challenge forthe researchers. Apache Spark is an open source, a general purpose engine for rapid large -scale data processing. Toovercome the data replication and disk I/O overhead of sharing data between parallel operations in Hadoop, Spark usesthe primitive called Resilient Distributed Datasets (RDD’s) which provides the programmers a fault tolerant and in -memory data storage across cluster nodes without replication that increases the processing speed of the applications toseveral magnitudes. We propose a method to analyze the sensor data using the Spark.
Mansi Shah, Vatika Tayal, “Fast Analysis of Sensor Data over MapReduce using Spark”, International Journal of Advance Engineering and Research Development (IJAERD), Vol. 2, Issue 5, pp. 1071-1075, May 2015.
