The Fast Data Difference
Fast Data is different from Big Data and has different requirements. It's based on a different technology stack that has the ability to analyze, decide, act on and extract value -- recommendations, decisions, and actions -- as fast as data arrives, typically in milliseconds.
Fast streaming data is generated by thousands of unique data sources (people, smartphones, sensors), contributing data at high velocity, in high volume. It contains valuable potential insights and can be augmented in real time with contextual information, but these insights are perishable and the opportunity to act on them is lost when the moment passes.
Today's applications need a fast data stack that isn’t built just to capture and pipe streaming data, but also to enrich, add context, personalize, and act on it before it becomes data at rest. These high-velocity applications require the ability to analyze and transact on streaming data.
Batch vs. Continuous Processing: Which is Best?
Batch has been the prevailing approach for processing big data for years. It’s an efficient way of processing large volumes of data: you collect, process, and then report. But while batch has gotten faster it’s not real-time, and falls short of what’s needed for fast data applications. If you want to grab real-time data and output recommendations, decisions, and analyses in milliseconds you need a different approach.
Fast data applications continuously ingest, analyze and make decisions/take action on each event as it flows through the system. The benefit of this approach is that incoming data is processed in real time on a per-person or per-event basis, and applications can deliver richer, more individualized interactions. The data is eventually exported to a long-term data store.
Batch processing has its place, but for real-time analytics and action “in the moment,” continuous processing is often a superior approach.
Advances with in-memory operational databases and high-speed data ingestion/export technologies make it easier and more practical to build fast data applications than ever before.
|Type of Data
||Big Data (at rest)
||Fast Data (in motion)
||Large historical data sets
||Incremental analysis of new events
||Minutes or more
|Type of Applications
||Reporting, Business Intelligence
||Operational, Mission Critical, Real-Time