Written from the ground up to run in-memory, VoltDB is the solution for fast data applications that need to transact in real-time. By occupying this unique place in the database landscape, VoltDB solves a problem that’s pervasive across many industry verticals where organizations are motivated to put their business insights to operational use against a deluge of events constantly streaming from a variety of sources.
Whether it is data warehousing or big data analytics that businesses derive their insights from, Teradata is a leading provider in this space. VoltDB recently partnered with Teradata to integrate with Teradata Listener to add a real-time decisioning layer to Teradata’s Unified Data Architecture. The UDA paints the high-level picture of how organizations’ systems can be architected to handle the data challenges they are facing today. Let’s look at the mechanics of the integration.
Teradata’s recently released Listener is simple to use. With a couple of clicks, users can create a REST endpoint to ingest multiple streams of data and hook up multiple targets to sink all that data into. The target can be a Teradata data warehouse or a Hadoop data lake. Listener is a scalable, low latency, fault-tolerant product that can run in the cloud or in your datacenter.
At VoltDB, it’s hard not to see the benefit of having a single endpoint to export processed data out for longer-term analytical processing without worrying about the protocols supported by the other systems or their scalability, performance and ability to keep up with VoltDB.
VoltDB’s exporter framework makes it easy to build integrations with other technologies. For Teradata Listener, I created a connector class which is fed events from VoltDB streams in a serialized manner. The VoltDB streams are then encoded for writing out to the target using the appropriate protocol. Here, the serialized data is converted into the JSON format Listener expects.
On the other side, Listener’s UI lets us define a source to serve as the endpoint for data ingestion, as well as targets where the data ends up eventually. Users also can update access permissions, check the ingest performance and view the activity of their streams using the well laid out dashboard.
All the details to download, install and use the connector are on Github at https://github.com/VoltDB/voltdb-teradata-connector.
VoltDB also provides connectors for Kafka, RabbitMQ, Kinesis, JDBC and ODBC, but if you’d like to build your own connector, you can find the instructions here.
To use this connector and perform the actual connection in my example application, I configured VoltDB’s export connector with the HTTP request parameters that Listener expects. This can be done by editing VoltDB’s deployment.xml file directly or by using the VoltDB Management Console.
If you are using Teradata Listener or planning on it, do check out VoltDB’s connector for Teradata Listener today, and let us know what you think.