In-memory databases deliver the fastest possible data access available today. They are gaining popularity among organizations struggling to keep up with fast transactions and other forms of high velocity streaming data.
When SQL databases were initially designed, memory was expensive, so rotating magnetic disks in the form of mechanical disk drives became the primary storage device for databases. While disk-based storage is still used and relied upon, the steadily decreasing price of RAM has the potential to make mechanical disks obsolete as the primary data storage layer for operational systems.
However, not all in-memory databases are the same. Some in-memory RDBMS technologies provide much higher performance than others when compared to traditional disk-based products. How can that be? It really depends on how the in-memory product was designed — was it created from the ground up to run in-memory, or was it a simple migration of a disk-based product to in-memory?
In 2007, the research team behind the Aurora Complex Event Processing system (commercialized as Streambase) and the C-Store Analytics (OLAP) system (commercialized as Vertica) set to building an operational database aligned with the needs of the 21st century enterprise.
They took an open source RDBMS that followed traditional RDBMS architecture, ran it on a memory-based filesystem, and measured where it spent its time. Over 80% of the database software’s time was spent on page buffer management, index management, and concurrency management. Only 12% of its time was spent actually doing the real work the database was supposed to do.
The research team also discovered that the concurrency problem could be exacerbated by trying to increase the speed of individual components within the database software system architecture — the whole was much less than the sum of its parts.
The original research behind VoltDB was led by Dr. Michael Stonebraker and a team of senior computer scientists from MIT, Yale University, and Brown University. The original research paper (H-Store) is here.
Stonebraker’s team decided an in-memory RDBMS built for today’s requirements needed a radical approach to solving concurrency issues. Modern in-memory databases that offer significant advantages must provide:
VoltDB is faster, smarter and simpler than traditional databases. Its in-memory scale-out architecture enables 100x faster performance than traditional alternatives. This enables our customers to convert live data into business value, analyze and act on streaming data, and use real-time intelligence.
With VoltDB, data is always consistent, correct, and never lost. This spells seamless ecosystem integration, simpler apps, easier testing, and better maintenance. VoltDB customers report they have experienced: