Redis scales vector data, improves data integration capabilities

Redis is updating its suite of data platforms with new capabilities designed to help accelerate performance and enable easier scaling.

Redis got its start as an open-source data caching technology and has expanded to become a set of enterprise and cloud real time database and data serving capabilities. In the past Redis has had a somewhat staggered release cadence with different product updates coming out at different times.

With the new Redis 7.2 update announced today, Redis is introducing what it refers to as a ‘unified release’ across its product suite, in a bid to help unify the company’s product launches and make it easier for users to adopt.

New focus on improving user experience

Among the big updates in Redis 7.2 are expanded capabilities for the vector database feature that can help to accelerate the performance of AI applications. Real time workflow also gets a boost with the Redis Data Integration (RDI) feature that enables better change data capture functionality than what Redis previously had available.

The unified Redis 7.2 release is the first under the direction of the company’s new CEO Rowan Trollope, who is bringing new focus on improving overall user experience.

“This platform is really built to be a remote data structure server, not a database,” Trollope told VentureBeat. “It provides an ability to take what I’m already doing on my local software, outsource it and put it into a distributed system that can do it way faster and make my application that much more clean and simple.”

Making vector search faster for AI

In the modern era of generative AI, vector databases are becoming increasingly critical. A vector database will commonly store vector embeddings in a data structure that enables rapid retrieval and search.

There are purpose built vector databases — like Pinecone and Milvus — and there are also a growing number of existing database platforms, like PostgreSQL and MongoDB that are being expanded to enable vector capabilities. Vector similarity search with Redis belongs to the latter category, as a set of capabilities that can be used to extend the Redis Enterprise platform.

Redis has had vector search capabilities in its platform before, but with the growth of AI in 2023, there has been a spike in interest across the company’s user base.

“Just about every one of our customers is coming to us and saying, ‘We want to implement this or that project, using Gen AI — can you help us?” said Trollope . “So we’re at the right place at the right time, I would say with our vector database.”

New vector search use cases

As part of the Redis 7.2 updates, vector search is getting a big boost. Trollope said that Redis developers have been working on implementing multi-threading capabilities to provide significantly more scale. He noted that in some use cases for vector search there is a need to potentially be able to query billions of vectors in real time.

He explained that in one scenario for deployment, organizations are taking their own content, vectorizing the content using Redis tools, storing, then enabling searches against that data with AI tools.

“We’ve seen many customers starting to implement chat products using OpenAI and using our vector database capability to do retrieval augmented generation,” said Trollope.

Vectors aren’t just for large language models (LLMs) and generative AI. Trollope said that a government agency (that he did not specify) has deployed Redis to help with real time face detection in airports, which has to be done in milliseconds.

“We’re finding all these really interesting use cases emerge for vector search.” he said. “It has become one of the big areas of investment for the company. “

Redis data integration and auto tiering boost real time 

Data is often generated and collected by different systems and databases, which potentially creates a challenge with data silos that aren’t connected.

With Redis Data Integration (RDI) in the Redis 7.2 update, the company is providing an integrated approach to help get data from other data sources including Oracle Database, PostgreSQL, MySQL and MongoDB.

“Redis Data Integration is essentially a change data capture  platform that streams changes from your source database into Redis,” Trollope explained “Then we can filter and transform it right in that process and map it into the supported data types of Redis.”

Optimizing data placement

Capturing large volumes of data and storing it all can become a cumbersome process over time. Depending on access demands, different types of data can be stored in different kinds of storage, including in-memory DRAM (dynamic random access memory) approaches as well as Solid State Drives (SSDs).

In-memory often has the fastest performance, but it also tends to have a higher cost than SSDs. Trollope noted that Redis has had a solution known as Redis on Flash to help organizations optimize the placement of data. That feature is now being rebuilt and rebranded as Auto-Tiering to help organizations automatically place data in the most effective deployment based on usage.

“We’ve also doubled the throughput and (halved) the latency so that it becomes a much more viable option than previous generations,” said Trollope. “That’s really important because a lot of customers and developers are finding the utility of Redis to draw in more kinds of data, but not necessarily in all cases do you want to be paying for DRAM.”