- Home
- HONGKE BUSINESS
Enterprise IT Solutions
Test Measurement
Industrial Internet of Things
- Latest Articles
- About Us
- Talent Recruitment
- Contact Us
EN
Presentation by Lei Shi, Senior Redis Architect
Search, Explore, Seek, Redis Vector Database.
Search, Explore, Search, Redis Vector Database
Shi Lei, Redis Senior Architect
Interview: How Redis, used by ChatGPT and OpenAI, was upgraded to a vector database?
Interview Summary: Companies across industries are increasingly recognizing that making data-driven decisions is a necessity to compete now, in the next 5 years, in the next 20 years and beyond. Data growth, especially unstructured data growth, is at an unprecedented level, and the era of data sprawl and artificial intelligence is upon us.
Implicit in this reality is the fact that AI can meaningfully categorize and process massive amounts of data - not only for tech giants like Alphabet, Meta, and Microsoft, with their massive R&D operations and custom AI tools, but also for ordinary businesses and even small and medium-sized enterprises.
Well-designed AI-based applications can sift through extremely large data sets extremely quickly to generate new insights and ultimately drive new revenue streams that create real value for businesses. But no data growth can be truly actionable and democratized without something new and emerging - vector databases.
With the explosion of big speech modeling, vector databases have become a hot topic. With just a few lines of simple Python code, vector databases can act as a cheap but efficient "external brain" for big language models. But do we really need a specialized vector database? Are vector databases a hype or a necessity? Let's listen to Redis Senior Architect Lei Shi, the official ecosystem partner of Hongke Cloud Technology, talk about Redis vector database technology practices.
Use AI to reimagine search
Using AI to Reconstruct the Search for Unstructured Data
Users expect search functionality in every application and website they encounter. However, more than 80% of business data is unstructured and stored in text, image, audio, video or other formats.
Organizations need to reimagine the way they make every kind of data discoverable - not least because users want it. Powerful search capabilities will power the next generation of applications.
What is Vector Database?
MISSION-DRIVEN
Vector Similarity Search (VSS)
--Key Features of Vector Database
A vector database is a database that stores data in the form of mathematical representations of vectors or data points Artificial Intelligence and Machine Learning have enabled unstructured data to be transformed into numerical representations (vectors) that capture meaning and context, thanks to advances in natural language processing and computer vision.
Vector Similarity Search (VSS) is a key feature of vector databases. It is the process of finding data points that are similar to a given query vector in a vector database. Common uses of VSS include recommender systems, image and video search, natural language processing, and anomaly detection. For example, if a recommendation system is constructed, VSS can be used to find (and suggest) products that are similar to those in which the user has previously shown interest.
Why is VSS a key component of vector databases?
Traditional keyword matching and filtering can only take you so far. Common search algorithms are useful for text and document use cases, but they are limited when the search results do not contain meaning or context. The proliferation of unstructured data has created a huge gap in the effectiveness of traditional keyword matching and filtering. Every organization (and almost everyone) that stores non-textual data could benefit from improved search capabilities for unstructured data. But until recently, only a handful of large cloud-native technology companies had this capability.
Redis Vector Database
Vector Database Solutions for Every Organization
1
Real-time search performance
The search and recommendation system must run very fast. the VSS feature in Redis Enterprise Edition ensures low search latency, whether the data collection is tens of thousands or hundreds of millions of objects spread across multiple database nodes.
2
Built-in fault tolerance and flexibility
Redis Enterprise Edition uses an unshared cluster architecture. It is fault tolerant at all levels, with automatic failover at the process level, at a single node, and across infrastructure availability zones.Redis Enterprise Edition includes tunable persistence and disaster recovery mechanisms.
3
Reduces architectural and application complexity
Most likely, your organization has benefited from Redis Enterprise for its caching needs. Developers can store vectors as easily as any other field in a Redis hash or JSON object.
4
Flexibility Across Clouds and Geographic Locations
Choose where to run the database.Redis Enterprise Edition can be deployed in any location, on any cloud platform, locally or in a multi-cloud or hybrid cloud architecture.
Redis Vector Database Use Cases
1
Retrieval Augmentation (RAG)
Redis Enterprise provides powerful hybrid semantic capabilities, injecting relevant contextual data into hints before sending it to the LLM, and storing external domain-specific knowledge to improve the quality of results.
2
Semantic Buffering
Redis Enterprise Edition identifies and retrieves cache responses that are semantically very similar to the input query, significantly reducing response times and the number of requests sent to the LLM.
3
Recommended Systems
Redis Enterprise Edition helps the recommendation engine deliver fresh and relevant suggestions to users with low latency. Helps you find products similar to the ones shoppers like.
4
Document Search
Redis Enterprise uses natural language and semantic search to make it easier to discover and retrieve information from large document corpora.
Client Cases
With Redis VSS, we can offer vector search to our customers and be confident that it is reliable and fast. We've seen an 80% improvement in latency compared to our original Lucene-based implementation, and we're excited to work with a trusted brand and team that makes it easier to work with organizations.
Our real-time recommender infrastructure takes milliseconds to search and update vector embeddings, providing an ultra-fast experience for our social and marketing customers. We benchmarked every product on the market and Redis came out on top. Plus, it's easy to use with a one-click Redis Enterprise Cloud setup.
Our retrieval, conversion, and extension platform relies heavily on vector databases to seamlessly integrate internal data with our powerful language model, while prioritizing data security. After an exhaustive evaluation of several vector database providers, Mantium chose Redis for its superior performance and extensive developer support, and the speed and cost advantages offered by Redis were simply unmatched.
Docugami's generative AI for business documents requires incredible speed and efficiency to realize our LLM and ML operations. With Redis Enterprise, we've seen significant performance improvements in document XML knowledge mapping and a significant reduction in COGS, and the integration of the Redis Vector Database has enabled us to process embeds more efficiently, improving consistency and accuracy when converting long-form documents into customer data.
Vector similarity search function
Vector Indexing Algorithm
Redis Enterprise Edition manages vectors with an indexed data structure, enabling intelligent similarity searches that balance search speed and search quality. Depending on your data and use cases, choose from two popular techniques, FLAT (a brute-force approach) and HNSW (a faster, more approximate approach).
Vector Search Distance Indicator
Redis Enterprise uses a distance metric to measure the similarity between two vectors. Choose from three popular metrics (euclidean, inner product, and cosine similarity) for calculating how "close" or "far" two vectors are.
Powerful Mixed Filtration
Take advantage of the full suite of search capabilities available in Redis Enterprise Query and Search. Enhance your workflow by combining the power of vector similarity with more traditional numeric, text, and label filters. Incorporate more business logic into queries and simplify client application code.
Live Updates
Real-time search and recommendation systems generate large amounts of constantly changing data. New images, text, products, or metadata? Seamlessly perform updates, inserts, and deletes to the search index as the data set changes over time.Redis Enterprise reduces the costly impact of data stagnation.
Vector Range Inquiry
While traditional vector searches are performed by finding the "top K" most similar vectors, Redis Enterprise also supports discovery of related content within predefined similarity ranges or thresholds, providing a more flexible search experience.
Related Resources
Retrieving with Redis VSS
Enterprise Vector Database
![[Enclosed Version] Hongke Redis White Paper - Vector Similarity Search_00.png](https://aiportek.com/wp-content/uploads/2024/03/【加封版】虹科Redis白皮书-矢量相似性搜索_00.png "[Enclosed Version] Hongke Redis White Paper - Vector Similarity Search_00.png")
Redis Enterprise Vector Similarity Search
Redis Enterprise Cloud
Redis Enterprise Software
Partners
