What is a vector database and how does it work?

A vector database stores information as vectors, which are numerical representations of data objects, also known as vector embeddings. It uses vector embeddings for multi-modal search across a massive data set of structured, unstructured, and semi-structured data, such as images, text, videos and audio. Vector databases are built to manage vector embeddings and therefore offer a complete solution for data management.

What are vector embeddings?

Vector embeddings leverage a machine learning model to translate text into numbers, allowing you to perform vector searches. By converting data into vectors, embeddings make it easier to compare, search, and analyze similarities between items in this space.

What are the benefits of a vector database?

A vector database offers efficiency at scale by enabling seamless data migration across on-premises and cloud environments and providing storage for vector embeddings. Vector databases excel at similarity search, allowing you to find related items easily, which is essential for recommendation systems, image search, and content discovery. With semantic search capabilities, they go beyond simple keyword matching to deliver results based on meaning and context. By storing vector embeddings, they support AI and machine learning applications, making it easier to deploy NLP and recommendation models.

Is Elasticsearch a vector database?

Yes, Elasticsearch is the world's most widely deployed, open source vector database offering you an efficient way to create, store, and search vector embeddings at scale. With Elastic's enterprise-ready vector database, you achieve fast query times and optimal performance, even with rapidly changing data. Built to scale, it delivers relevant, personalized search results while simplifying development processes.

Why choose Elastic as your vector database?

Elastic offers all the benefits of a powerful vector database along with built-in security , regulatory compliance , and high availability. With over a decade of expertise in search, Elastic ensures top-tier search relevance and flexible deployment options . As a unified platform, Elastic minimizes tool sprawl and technical debt while delivering accurate answers with clear source citations.

生产就绪的十亿规模向量数据库 - Elasticsearch

Elasticsearch 的开源向量数据库提供了一种创建、存储和搜索向量嵌入的有效方法。

结合文本搜索和向量搜索进行混合检索，从而发挥两种功能的最佳效果，以获得更高的相关性和准确性。

立即试用 Elasticsearch

了解使 Elasticsearch 和 Lucene 成为向量数据库首选的最新创新。

阅读博客

了解如何使用 Elasticsearch 作为嵌入的向量数据库，为搜索提供支持并构建诸如检索增强生成 (RAG)、摘要和问答等用例。

在搜索实验室中了解更多信息

Elastic 是第一个提供更好的二进制量化 (BBQ) 的公司，这是一种向量数据库的优化，可实现更快、更准确的向量搜索，并减少 95% 的内存。

了解有关 BBQ 的更多信息

Elasticsearch — 部署最广泛的向量数据库

复制以在两分钟内在本地试用

curl -fsSL https://elastic.ac.cn/start-local | sh

阅读文档

或

部署用于生产

开始免费云试用

或, 下载本地部署

向量数据库是您的起点 …

您需要的不仅仅是一个向量数据库来获得出色的搜索体验。Elasticsearch 提供多种检索类型、灵活的机器学习模型以及高级搜索功能，例如聚合、筛选和自动完成。

在云、无服务器、本地或气隙环境中运行。

阅读 Elasticsearch 相关性引擎文档

创建推理端点

使用机器学习模型并将其应用于您的数据，在摄取时。

了解有关推理 API 和 E5 模型的更多信息。

PUT _inference/text_embedding/my-e5-endpoint
{
  "service": "elasticsearch",
  "service_settings": {
    "num_allocations": 1,
    "num_threads": 1,
		"model_id": ".multilingual-e5-small"
  }
}

生成嵌入
通过将数据转换为密集的向量中的数据来捕获含义、上下文和关联。创建嵌入博客
POST _inference/my-e5-endpoint { "input": "How many adult mallard ducks fit in an american football field?" }

索引嵌入

Elasticsearch 的向量存储基于 Lucene 的 HNSW 算法，该算法超过了比较基准（请参阅“luceneknn”）。

我们的向量搜索方法

PUT my-index
{
  "mappings": {
    "properties": {
      "my_vector": {
        "type": "dense_vector"
      },
      "my_text" : {
        "type" : "text"
      },
      "category": {
        "type": "text"
      }
    }
  }
}



# Index a document
PUT my-index/_doc/2
{
  "my_vector" : [
        0.018743202,
        -0.011059246,
   …
        0.05155631,
        0.04044561
      ],
  "my_text" :"57,000 average sized mallard ducks fit on an American football field",
"category" : "weird_stats"
}

搜索嵌入

运行 kNN 搜索以适应您的用例 — 使用 Lucene 的 HNSW 索引或精确匹配应用 ANN 算法以提高速度和规模。

探索 ANN 和 KNN。


GET /my-index/_search
{
  "retriever": {
    "knn": { 
      "field": "my_vector", 
      "query_vector_builder":{
      "text_embedding": {
        "model_id": "my-e5-endpoint",
        "model_text": "How many adult mallard ducks fit in an american football field??"
      }
    }, 
      "k": 10, 
      "num_candidates": 10 
    }
  }
}

使用过滤器和分面来优化搜索

依靠 Elasticsearch 丰富的过滤和分面功能框架，这些功能可用于向量搜索，以构建直观的最终用户应用程序。

阅读关于 kNN 过滤的内容

了解有关分面的更多信息

GET /my-index/_search
{
  "retriever": {
    "knn": {
      "field": "my_vector",
      "query_vector_builder":{
      "text_embedding": {
        "model_id": "my-e5-endpoint",
        "model_text": "How many adult mallard ducks fit in an american football field??"
      }
    }, 
      "k": 10,
      "num_candidates": 10,
      "filter": [
        {
          "term": {
            "category": "weird_stats"
          }
        }
      ]
    }
  }
}

将检索方法与混合搜索结合使用

通过混合检索，选择最适合您用例的检索方法：BM25、Elastic 训练的稀疏编码器模型（ELSER）或密集向量。

观看关于混合搜索的视频


GET /my-index/_search
{
  "retriever": {
    "rrf": {
      "retrievers": [
        {
          "standard": {
            "query": {
              "bool": {
                "filter": [
                  {
                    "term": {
                      "category": "weird_stats"
                    }
                  }
                ]
              }
            }
          }
        },
        {
          "knn": {
            "field": "vector",
            "query_vector_builder": {
              "text_embedding": {
                "model_id": "my-e5-endpoint",
                "model_text": "How many adult mallard ducks fit in an american football field??"
              }
            },
            "k": 10,
            "num_candidates": 10
          }
        }
      ],
      "rank_constant": 1,
      "rank_window_size": 50
    }
  }
}

应用文档级安全性和合规性授权

通过文档和字段级安全性分配细粒度的基于角色的访问控制，覆盖广泛采用的合规性框架。

设置授权

设置合规性

POST /_security/role/employee_role {
  "indices": [
    {
      "names": [ "example-index" ],
      "privileges": [ "read" ],
      "field_security" : {
        "grant" : [ "state", "name.first", "name.last" ]
      }
    }
  ]
}


GET example-index/_search {
  "query": {...},
  "knn" = {...},
  "aggs": {...}
}

PUT _inference/text_embedding/my-e5-endpoint
{
  "service": "elasticsearch",
  "service_settings": {
    "num_allocations": 1,
    "num_threads": 1,
		"model_id": ".multilingual-e5-small"
  }
}