Apache Kafka & Amazon Kinesis are terrific tools for building real-time data streaming pipelines, but performing transactional processing and analytics on stand-alone message bus technology can be challenging. With a combination of real-time actionable data processing capabilities such as: contextual state, ACID transactions, and embedded machine learning along with fast ingestion and export; you can drastically reduce: complexity, latency, and hardware costs. Enabling use cases such as smart meter billing, predictive maintenance in Industrial IoT, fraud prevention, personalized offers, and more.
Traditional databases are fast, but lack streaming capabilities. While traditional streaming architectures like Kappa can do fast streaming, but offer only very rudimentary data management. VoltDB’s Smart Streaming architecture significantly simplifies the Kappa Architecture. In the Kappa architecture; Apache Kafka is used as a real-time glue for ingestion because of the need for durability and speed mismatches downstream. VoltDB is fast enough to ingest and offers durability built-in, eliminating the need for an Apache Kafka ingestion layer. In VoltDB, the data is stored in a relational format therefore a separate serving database isn’t required either. Finally, VoltDB has complex event processing capabilities like aggregations, filtering, sampling, correlations, along with stored procedures and user defined functions. Machine Learning models can be embedded within Java based stored procedures to provide real-time decisions on fast streaming data. Smart Streaming brings analysis to the data as opposed to taking the data to the analysis, resulting in: