The Hadoop philosophy for Fog computing

Fog computing draws its analogy from the success of Hadoop and MapReduce, and to better understand the importance of Fog Computing, it is worth taking some time to think about how Hadoop works. MapReduce is a method of mapping and Hadoop is an open source framework based on the MapReduce algorithm.  

MapReduce has three steps: map, shuffle, and reduce. In the map phase, computing functions are applied to local data. The shuffle step redistributes the data as needed. This is a critical step as the system attempts to collocate all dependent data to one node. The final step is the reduce phase, where the processing across all the nodes occurs in parallel. 

The general takeaway here is that MapReduce attempts to bring processing to where the data is and not to move the data to where the processors are. This scheme effectively removes communication overhead and a natural bottleneck in systems that have extremely large structured or unstructured datasets. This paradigm applies to IoT as well. In the IoT space, data (possibly a very large amount of data) is produced in real time as a stream of data. This is the big data in IoT's case. It's not static data like a database or a Google storage cluster, but an endless live stream of data from every corner of the world. A fog-based design is the natural way to resolve this new big data problem.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset