VoltDB hired Kyle Kingsbury, creator of the Jepsen Tests, to build a new, stronger Jepsen test especially for VoltDB. We promise strong serializability in a distributed database, a stronger promise than almost any other system, and we’ve been working with Kingsbury to validate that promise.
What is Jepsen Testing? These tests are veritable stress tests created in an effort to improve the safety of distributed databases. Read more here: http://jepsen.io
- VoltDB 6.4 has passed official Jepsen testing performed by Kyle Kingsbury, Jepsen’s creator.
- VoltDB 6.4 has passed more stringent testing than any other system Jepsen has tested.
- Jepsen found several issues in VoltDB 6.3 and we fixed every one. Our tolerance for these bugs is zero.
- We have integrated Jepsen testing into our automated testing for each VoltDB build. We have and will continue to make our Jepsen and non-Jepsen tests stronger and better.
- We stake our reputation on correctness, consistency, and safety.
You can read Kingsbury's detailed post on his experiences here:https://aphyr.com/posts/331-jepsen-voltdb-6-3
The most important quote:
VoltDB’s pre-6.4 development builds have now passed all the original Jepsen tests, as well as more aggressive elaborations on their themes. Version 6.4 appears to provide strong serializability: the strongest safety invariant of any system we’ve tested thus far.
Jepsen has proven its value as a tool in any distributed system tester’s arsenal, and multiple people have asked us about Jepsen and VoltDB specifically. Jepsen is both famous and notorious in the database industry for finding undiscovered problems with distributed systems. There are few tests of mettle as recognizable as Jepsen in our community.
While we had been planning to do the testing ourselves, we understood that nothing we did would have the same credibility as a test run by Kyle Kingsbury himself, creator of Jepsen and embarraser of databases. When Kingsbury started his Jepsen-For-Hire business last fall, we immediately got in line, and over the past two months, we’ve been working closely with him as he tested VoltDB.
We’ve written a blog post that goes into more detail about VoltDB and our Jepsen testing. It starts with what’s on this page, but also covers:
- The Most Stringent Jepsen Tests So Far
- Are the Issues Found Serious?
- Reproducible and Open
- Data Safety and Correct Answers Are Our Highest Priority
- Going Forward