Our latency goals are determined by the use cases we target.
Let’s look at an example: In mobile call authorization, a decision about whether to authorize a call must be made in about 50ms, or our customers lose money. This changes latency from the naive question:
“What is the average latency for a decision made with VoltDB?”
“What fraction of decisions take longer than 50ms to respond?”
When we started to engage with these kinds of users, early versions of VoltDB could maintain a 1-2ms average latency, but like many systems, latency at the 99.9 percentile and beyond was less favorable. We spend a tremendous amount of engineering and testing effort to meet latency SLAs for these customers, often requiring 99.999% of transactions to complete within about 50ms.
This meant re-targeting our long-running tests to measure for long tail latency issues. We needed to ask the same kinds of questions our users were asking. This lead to identifying and mitigating sources of latency in priority order, until we could run telco workloads with “five nines” under the latency SLA.
- We moved many blocking tasks to asynchronous tasks. For example, initiating a snapshot used to run disk operations to scan directories and create files in the blocking path. That was moved into a pre-check.
- We carefully examined Java garbage collections. We refined how we use memory so that most objects are short lived or very long lived, and we kept the heap size small. This all-but eliminated longer pauses for garbage collection.
- Some blocking tasks were user-initiated, so we added more visibility for administrators and the ability to auto-cancel some tasks that take longer than a defined threshold.
In-The-Moment: Other Latency-Sensitive Use Cases
Many VoltDB use cases require similar power, in-the-moment.
Lots of fraud detection applications happen after the fact. Given a set of transactions, run some sophisticated analysis and determine which of them were fraudulent. While great strides have been made in accuracy, if you really want to save people and institutions money, it’s important to detect fraud before the transaction is even authorized.
Mobile game companies use sophisticated big data and machine learning systems to model user engagement. The goal is to keep players playing. They know what kinds of things keep players coming, but they need a system that can react in real time to leverage that.
If a player walks into a virtual store, the application need to decide what should be for sale and for what price. VoltDB can combine what’s known about the player, the recent history of player action, and rich models to decide to show a particular sword for 30 coins, then record whether the user was interested or not. All this needs to happen live, without the user waiting.
Ad-Tech & Micro-Personalization
Similar to the gaming use case, what advertisement or offer to show might depend on real time data. Combine recent history with rules and models to decide which ad to show while the page is being drawn. Record which ad was shown, and record what action the user took, combining offer and response event streams.
Need a system for keeping stats and counters while driving an AB-test-fest? Try strongly consistent and transactional VoltDB.