ES|QL REST API

编辑

概述

编辑

ES|QL 查询 API 接受 query 参数中的 ES|QL 查询字符串,运行它并返回结果。例如

resp = client.esql.query(
    format="txt",
    query="FROM library | KEEP author, name, page_count, release_date | SORT page_count DESC | LIMIT 5",
)
print(resp)
const response = await client.esql.query({
  format: "txt",
  query:
    "FROM library | KEEP author, name, page_count, release_date | SORT page_count DESC | LIMIT 5",
});
console.log(response);
POST /_query?format=txt
{
  "query": "FROM library | KEEP author, name, page_count, release_date | SORT page_count DESC | LIMIT 5"
}

返回结果为

     author      |        name        |  page_count   | release_date
-----------------+--------------------+---------------+------------------------
Peter F. Hamilton|Pandora's Star      |768            |2004-03-02T00:00:00.000Z
Vernor Vinge     |A Fire Upon the Deep|613            |1992-06-01T00:00:00.000Z
Frank Herbert    |Dune                |604            |1965-06-01T00:00:00.000Z
Alastair Reynolds|Revelation Space    |585            |2000-03-15T00:00:00.000Z
James S.A. Corey |Leviathan Wakes     |561            |2011-06-02T00:00:00.000Z

Kibana 控制台

编辑

如果您正在使用 Kibana 控制台(强烈推荐),请在创建查询时利用三引号 """。这不仅会自动转义查询字符串中的双引号("),还支持多行请求

resp = client.esql.query(
    format="txt",
    query="\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
)
print(resp)
const response = await client.esql.query({
  format: "txt",
  query:
    "\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
});
console.log(response);
POST /_query?format=txt
{
  "query": """
    FROM library
    | KEEP author, name, page_count, release_date
    | SORT page_count DESC
    | LIMIT 5
  """
}

响应格式

编辑

ES|QL 可以返回以下人类可读和二进制格式的数据。您可以通过在 URL 中指定 format 参数或设置 AcceptContent-Type HTTP 标头来设置格式。

URL 参数优先于 HTTP 标头。如果两者都未指定,则响应以与请求相同的格式返回。

格式

HTTP 标头

描述

人类可读

csv

text/csv

逗号分隔值

json

application/json

JSON (JavaScript 对象表示法) 人类可读格式

tsv

text/tab-separated-values

制表符分隔值

txt

text/plain

类似 CLI 的表示

yaml

application/yaml

YAML (YAML 不是标记语言) 人类可读格式

二进制

cbor

application/cbor

简洁二进制对象表示

smile

application/smile

Smile 类似于 CBOR 的二进制数据格式

arrow

application/vnd.apache.arrow.stream

实验性。 Apache Arrow 数据框,IPC 流式格式

csv 格式接受格式化 URL 查询属性 delimiter,该属性指示用于分隔 CSV 值的字符。它默认为逗号 ( , ),并且不能采用以下任何值:双引号 ( " )、回车符 ( \r ) 和换行符 ( \n )。制表符(\t)也不能使用。请改用 tsv 格式。

使用 Elasticsearch Query DSL 进行过滤

编辑

filter 参数中指定 Query DSL 查询,以过滤 ES|QL 查询运行的文档集。

resp = client.esql.query(
    format="txt",
    query="\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
    filter={
        "range": {
            "page_count": {
                "gte": 100,
                "lte": 200
            }
        }
    },
)
print(resp)
const response = await client.esql.query({
  format: "txt",
  query:
    "\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
  filter: {
    range: {
      page_count: {
        gte: 100,
        lte: 200,
      },
    },
  },
});
console.log(response);
POST /_query?format=txt
{
  "query": """
    FROM library
    | KEEP author, name, page_count, release_date
    | SORT page_count DESC
    | LIMIT 5
  """,
  "filter": {
    "range": {
      "page_count": {
        "gte": 100,
        "lte": 200
      }
    }
  }
}

返回结果为

    author     |                name                |  page_count   | release_date
---------------+------------------------------------+---------------+------------------------
Douglas Adams  |The Hitchhiker's Guide to the Galaxy|180            |1979-10-12T00:00:00.000Z

列式结果

编辑

默认情况下,ES|QL 将结果作为行返回。例如,FROM 将每个单独的文档作为一行返回。对于 jsonyamlcborsmile 格式,ES|QL 可以以列式方式返回结果,其中一行表示结果中某一列的所有值。

resp = client.esql.query(
    format="json",
    query="\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
    columnar=True,
)
print(resp)
const response = await client.esql.query({
  format: "json",
  query:
    "\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
  columnar: true,
});
console.log(response);
POST /_query?format=json
{
  "query": """
    FROM library
    | KEEP author, name, page_count, release_date
    | SORT page_count DESC
    | LIMIT 5
  """,
  "columnar": true
}

返回结果为

{
  "took": 28,
  "columns": [
    {"name": "author", "type": "text"},
    {"name": "name", "type": "text"},
    {"name": "page_count", "type": "integer"},
    {"name": "release_date", "type": "date"}
  ],
  "values": [
    ["Peter F. Hamilton", "Vernor Vinge", "Frank Herbert", "Alastair Reynolds", "James S.A. Corey"],
    ["Pandora's Star", "A Fire Upon the Deep", "Dune", "Revelation Space", "Leviathan Wakes"],
    [768, 613, 604, 585, 561],
    ["2004-03-02T00:00:00.000Z", "1992-06-01T00:00:00.000Z", "1965-06-01T00:00:00.000Z", "2000-03-15T00:00:00.000Z", "2011-06-02T00:00:00.000Z"]
  ]
}

返回本地化结果

编辑

在请求正文中使用 locale 参数,以按照区域设置的约定格式返回结果(尤其是日期)。如果未指定 locale,则默认为 en-US(英语)。请参阅 JDK 支持的区域设置

语法:locale 参数接受 xyxy-XY 格式(不区分大小写)的语言标记。

例如,要以法语返回月份名称

resp = client.esql.query(
    locale="fr-FR",
    query="\n          ROW birth_date_string = \"2023-01-15T00:00:00.000Z\"\n          | EVAL birth_date = date_parse(birth_date_string)\n          | EVAL month_of_birth = DATE_FORMAT(\"MMMM\",birth_date)\n          | LIMIT 5\n   ",
)
print(resp)
const response = await client.esql.query({
  locale: "fr-FR",
  query:
    '\n          ROW birth_date_string = "2023-01-15T00:00:00.000Z"\n          | EVAL birth_date = date_parse(birth_date_string)\n          | EVAL month_of_birth = DATE_FORMAT("MMMM",birth_date)\n          | LIMIT 5\n   ',
});
console.log(response);
POST /_query
{
  "locale": "fr-FR",
  "query": """
          ROW birth_date_string = "2023-01-15T00:00:00.000Z"
          | EVAL birth_date = date_parse(birth_date_string)
          | EVAL month_of_birth = DATE_FORMAT("MMMM",birth_date)
          | LIMIT 5
   """
}

将参数传递给查询

编辑

值(例如用于条件的值)可以通过将值集成到查询字符串本身来“内联”传递给查询

resp = client.esql.query(
    query="\n    FROM library\n    | EVAL year = DATE_EXTRACT(\"year\", release_date)\n    | WHERE page_count > 300 AND author == \"Frank Herbert\"\n    | STATS count = COUNT(*) by year\n    | WHERE count > 0\n    | LIMIT 5\n  ",
)
print(resp)
const response = await client.esql.query({
  query:
    '\n    FROM library\n    | EVAL year = DATE_EXTRACT("year", release_date)\n    | WHERE page_count > 300 AND author == "Frank Herbert"\n    | STATS count = COUNT(*) by year\n    | WHERE count > 0\n    | LIMIT 5\n  ',
});
console.log(response);
POST /_query
{
  "query": """
    FROM library
    | EVAL year = DATE_EXTRACT("year", release_date)
    | WHERE page_count > 300 AND author == "Frank Herbert"
    | STATS count = COUNT(*) by year
    | WHERE count > 0
    | LIMIT 5
  """
}

为了避免任何黑客攻击或代码注入尝试,请在单独的参数列表中提取值。在查询字符串中使用问号占位符 ( ? ) 表示每个参数

resp = client.esql.query(
    query="\n    FROM library\n    | EVAL year = DATE_EXTRACT(\"year\", release_date)\n    | WHERE page_count > ? AND author == ?\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?\n    | LIMIT 5\n  ",
    params=[
        300,
        "Frank Herbert",
        0
    ],
)
print(resp)
const response = await client.esql.query({
  query:
    '\n    FROM library\n    | EVAL year = DATE_EXTRACT("year", release_date)\n    | WHERE page_count > ? AND author == ?\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?\n    | LIMIT 5\n  ',
  params: [300, "Frank Herbert", 0],
});
console.log(response);
POST /_query
{
  "query": """
    FROM library
    | EVAL year = DATE_EXTRACT("year", release_date)
    | WHERE page_count > ? AND author == ?
    | STATS count = COUNT(*) by year
    | WHERE count > ?
    | LIMIT 5
  """,
  "params": [300, "Frank Herbert", 0]
}

参数可以是命名参数或位置参数。

命名参数使用问号占位符 ( ? ) 后跟字符串。

resp = client.esql.query(
    query="\n    FROM library\n    | EVAL year = DATE_EXTRACT(\"year\", release_date)\n    | WHERE page_count > ?page_count AND author == ?author\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?count\n    | LIMIT 5\n  ",
    params=[
        {
            "page_count": 300
        },
        {
            "author": "Frank Herbert"
        },
        {
            "count": 0
        }
    ],
)
print(resp)
const response = await client.esql.query({
  query:
    '\n    FROM library\n    | EVAL year = DATE_EXTRACT("year", release_date)\n    | WHERE page_count > ?page_count AND author == ?author\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?count\n    | LIMIT 5\n  ',
  params: [
    {
      page_count: 300,
    },
    {
      author: "Frank Herbert",
    },
    {
      count: 0,
    },
  ],
});
console.log(response);
POST /_query
{
  "query": """
    FROM library
    | EVAL year = DATE_EXTRACT("year", release_date)
    | WHERE page_count > ?page_count AND author == ?author
    | STATS count = COUNT(*) by year
    | WHERE count > ?count
    | LIMIT 5
  """,
  "params": [{"page_count" : 300}, {"author" : "Frank Herbert"}, {"count" : 0}]
}

位置参数使用问号占位符 ( ? ) 后跟整数。

resp = client.esql.query(
    query="\n    FROM library\n    | EVAL year = DATE_EXTRACT(\"year\", release_date)\n    | WHERE page_count > ?1 AND author == ?2\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?3\n    | LIMIT 5\n  ",
    params=[
        300,
        "Frank Herbert",
        0
    ],
)
print(resp)
const response = await client.esql.query({
  query:
    '\n    FROM library\n    | EVAL year = DATE_EXTRACT("year", release_date)\n    | WHERE page_count > ?1 AND author == ?2\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?3\n    | LIMIT 5\n  ',
  params: [300, "Frank Herbert", 0],
});
console.log(response);
POST /_query
{
  "query": """
    FROM library
    | EVAL year = DATE_EXTRACT("year", release_date)
    | WHERE page_count > ?1 AND author == ?2
    | STATS count = COUNT(*) by year
    | WHERE count > ?3
    | LIMIT 5
  """,
  "params": [300, "Frank Herbert", 0]
}

运行异步 ES|QL 查询

编辑

ES|QL 异步查询 API 允许您异步执行查询请求、监视其进度并在结果可用时检索结果。

执行 ES|QL 查询通常非常快,但是跨大型数据集或冻结数据的查询可能需要一些时间。为了避免长时间等待,请运行异步 ES|QL 查询。

由异步查询 API 发起的查询可能会返回结果,也可能不返回结果。wait_for_completion_timeout 属性确定等待结果的时间。如果此时结果不可用,则会返回一个 查询 ID,该 ID 稍后可用于检索结果。例如

resp = client.esql.async_query(
    body={
        "query": "\n    FROM library\n    | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n    | STATS MAX(page_count) BY year\n    | SORT year\n    | LIMIT 5\n  ",
        "wait_for_completion_timeout": "2s"
    },
)
print(resp)
const response = await client.esql.asyncQuery({
  body: {
    query:
      "\n    FROM library\n    | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n    | STATS MAX(page_count) BY year\n    | SORT year\n    | LIMIT 5\n  ",
    wait_for_completion_timeout: "2s",
  },
});
console.log(response);
POST /_query/async
{
  "query": """
    FROM library
    | EVAL year = DATE_TRUNC(1 YEARS, release_date)
    | STATS MAX(page_count) BY year
    | SORT year
    | LIMIT 5
  """,
  "wait_for_completion_timeout": "2s"
}

如果在给定的超时时间内(本例中为 2 秒)结果不可用,则不会返回结果,而是返回包含以下内容的响应

  • 查询 ID
  • 值为 trueis_running 值,表示查询正在进行中

查询继续在后台运行,而不会阻塞其他请求。

{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_running": true
}

要检查异步查询的进度,请使用 ES|QL 异步查询获取 API 以及查询 ID。在 wait_for_completion_timeout 参数中指定您希望等待完整结果的时间。

resp = client.esql.async_query_get(
    id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
    wait_for_completion_timeout="30s",
    body=None,
)
print(resp)
response = client.esql.async_query_get(
  id: 'FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=',
  wait_for_completion_timeout: '30s'
)
puts response
const response = await client.esql.asyncQueryGet({
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  wait_for_completion_timeout: "30s",
  body: null,
});
console.log(response);
GET /_query/async/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?wait_for_completion_timeout=30s

如果响应的 is_running 值为 false,则表示查询已完成,并且返回结果以及查询的 took 时间。

{
  "is_running": false,
  "took": 48,
  "columns": ...
}

keep_alive 期间结束之前,使用 ES|QL 异步查询删除 API 删除异步查询。如果查询仍在运行,Elasticsearch 将取消它。

DELETE /_query/async/FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI=