Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases. As the heart of the Elastic Stack, it centrally stores data for fast search, fine‑tuned relevancy, and analytics that scale.
Elasticsearch now features generative AI search capabilities. Elasticsearch Relevance Engine™ (ESRE) powers generative AI solutions for private data sets with a vector database and machine learning models for semantic search that bring increased relevance to more search application developers.
ESRE combines AI with Elastic’s text search to give developers a full suite of sophisticated retrieval algorithms and the ability to integrate with large language models (LLMs). It is accessed through a single, unified API.
The Elasticsearch Relevance Engine’s configurable capabilities can be used to help improve relevance by:
- Applying advanced relevance ranking features including BM25f, a critical component of hybrid search
- Creating, storing, and searching dense embeddings using Elastic’s vector database
- Processing text using a wide range of natural language processing (NLP) tasks and models
- Letting developers manage and use their own transformer models in Elastic for business specific context
- Integrating with third-party transformer models such as OpenAI’s GPT-3 and 4 via API to retrieve intuitive summarization of content based on the customer’s data stores consolidated within Elasticsearch deployments
- Enabling ML-powered search without training or maintaining a model using Elastic’s out-of-the-box Learned Sparse Encoder model to deliver highly relevant, semantic search across a variety of domains
- Combining sparse and dense retrieval using Reciprocal Rank Fusion (RRF), a hybrid ranking method that gives developers control to optimize their AI search engine to their unique mix of natural language and keyword query types
- Integrating with third-party tooling such as LangChain to help build sophisticated data pipelines and generative AI applications
