How streaming works

There are a few ways to achieve this process, and each comes with its own advantages and disadvantages. The first approach is called native stream processing. In the native streaming process, the events are processed as they come into the existence, one after another. This results in a very low latency. If an event is successively large, it blocks other events from being processed. In addition to that, this technique is computationally very expensive. 

Another approach is called Micro-batch processing. This technique makes the opposite trade-off. It breaks the incoming events up into several batches either by arrival time or until a batch has reached a threshold size. Splitting reduces the computational cost of processing. However, the process introduces higher latency. Whether it is native-stream processing or micro-batch processing, they involve discrete steps of data ingestion, data processing, and data analytics as listed as follows:

  • Data ingestion or data harvesting
  • Data processing
  • Data analytics or deriving value
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset