« 相似度模块 存储 »
Elastic 文档 ›Elasticsearch 指南 [8.14] ›索引模块

慢日志

编辑

慢日志编辑

搜索慢日志编辑

分片级慢速搜索日志允许将慢速搜索（查询和获取阶段）记录到专用日志文件中。

可以为执行的查询阶段和获取阶段设置阈值，以下是一个示例

index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 5s
index.search.slowlog.threshold.query.debug: 2s
index.search.slowlog.threshold.query.trace: 500ms

index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 800ms
index.search.slowlog.threshold.fetch.debug: 500ms
index.search.slowlog.threshold.fetch.trace: 200ms

以上所有设置都是*动态的*，可以使用 更新索引设置 API 为每个索引设置。例如

resp = client.indices.put_settings(
    index="my-index-000001",
    body={
        "index.search.slowlog.threshold.query.warn": "10s",
        "index.search.slowlog.threshold.query.info": "5s",
        "index.search.slowlog.threshold.query.debug": "2s",
        "index.search.slowlog.threshold.query.trace": "500ms",
        "index.search.slowlog.threshold.fetch.warn": "1s",
        "index.search.slowlog.threshold.fetch.info": "800ms",
        "index.search.slowlog.threshold.fetch.debug": "500ms",
        "index.search.slowlog.threshold.fetch.trace": "200ms",
    },
)
print(resp)
response = client.indices.put_settings(
  index: 'my-index-000001',
  body: {
    'index.search.slowlog.threshold.query.warn' => '10s',
    'index.search.slowlog.threshold.query.info' => '5s',
    'index.search.slowlog.threshold.query.debug' => '2s',
    'index.search.slowlog.threshold.query.trace' => '500ms',
    'index.search.slowlog.threshold.fetch.warn' => '1s',
    'index.search.slowlog.threshold.fetch.info' => '800ms',
    'index.search.slowlog.threshold.fetch.debug' => '500ms',
    'index.search.slowlog.threshold.fetch.trace' => '200ms'
  }
)
puts response
PUT /my-index-000001/_settings
{
  "index.search.slowlog.threshold.query.warn": "10s",
  "index.search.slowlog.threshold.query.info": "5s",
  "index.search.slowlog.threshold.query.debug": "2s",
  "index.search.slowlog.threshold.query.trace": "500ms",
  "index.search.slowlog.threshold.fetch.warn": "1s",
  "index.search.slowlog.threshold.fetch.info": "800ms",
  "index.search.slowlog.threshold.fetch.debug": "500ms",
  "index.search.slowlog.threshold.fetch.trace": "200ms"
}

默认情况下，阈值被禁用（设置为 -1）。

日志记录是在分片级别范围内完成的，这意味着在特定分片内执行搜索请求。它不包含整个搜索请求，该请求可以广播到多个分片以执行。与请求级别相比，分片级别日志记录的一些好处是与特定机器上的实际执行相关联。

搜索慢日志文件在 log4j2.properties 文件中配置。

识别搜索慢日志来源编辑

识别触发慢速运行查询的原因通常很有用。要包含有关触发慢速搜索的用户信息，请使用 index.search.slowlog.include.user 设置。

resp = client.indices.put_settings(
    index="my-index-000001",
    body={"index.search.slowlog.include.user": True},
)
print(resp)
response = client.indices.put_settings(
  index: 'my-index-000001',
  body: {
    'index.search.slowlog.include.user' => true
  }
)
puts response
PUT /my-index-000001/_settings
{
  "index.search.slowlog.include.user": true
}

这将导致用户信息包含在慢日志中。

{
  "@timestamp": "2024-02-21T12:42:37.255Z",
  "log.level": "WARN",
  "auth.type": "REALM",
  "elasticsearch.slowlog.id": "tomcat-123",
  "elasticsearch.slowlog.message": "[index6][0]",
  "elasticsearch.slowlog.search_type": "QUERY_THEN_FETCH",
  "elasticsearch.slowlog.source": "{\"query\":{\"match_all\":{\"boost\":1.0}}}",
  "elasticsearch.slowlog.stats": "[]",
  "elasticsearch.slowlog.took": "747.3micros",
  "elasticsearch.slowlog.took_millis": 0,
  "elasticsearch.slowlog.total_hits": "1 hits",
  "elasticsearch.slowlog.total_shards": 1,
  "user.name": "elastic",
  "user.realm": "reserved",
  "ecs.version": "1.2.0",
  "service.name": "ES_ECS",
  "event.dataset": "elasticsearch.index_search_slowlog",
  "process.thread.name": "elasticsearch[runTask-0][search][T#5]",
  "log.logger": "index.search.slowlog.query",
  "elasticsearch.cluster.uuid": "Ui23kfF1SHKJwu_hI1iPPQ",
  "elasticsearch.node.id": "JK-jn-XpQ3OsDUsq5ZtfGg",
  "elasticsearch.node.name": "node-0",
  "elasticsearch.cluster.name": "distribution_run"
}

如果使用 X-Opaque-ID 标头启动调用，则该 ID 将包含在搜索慢日志的 elasticsearch.slowlog.id 字段中。有关详细信息和最佳实践，请参阅 X-Opaque-Id HTTP 标头。

索引慢日志编辑

索引慢日志，其功能类似于搜索慢日志。日志文件名以 _index_indexing_slowlog.json 结尾。日志和阈值的配置方式与搜索慢日志相同。索引慢日志示例

index.indexing.slowlog.threshold.index.warn: 10s
index.indexing.slowlog.threshold.index.info: 5s
index.indexing.slowlog.threshold.index.debug: 2s
index.indexing.slowlog.threshold.index.trace: 500ms
index.indexing.slowlog.source: 1000

以上所有设置都是*动态的*，可以使用 更新索引设置 API 为每个索引设置。例如

resp = client.indices.put_settings(
    index="my-index-000001",
    body={
        "index.indexing.slowlog.threshold.index.warn": "10s",
        "index.indexing.slowlog.threshold.index.info": "5s",
        "index.indexing.slowlog.threshold.index.debug": "2s",
        "index.indexing.slowlog.threshold.index.trace": "500ms",
        "index.indexing.slowlog.source": "1000",
    },
)
print(resp)
response = client.indices.put_settings(
  index: 'my-index-000001',
  body: {
    'index.indexing.slowlog.threshold.index.warn' => '10s',
    'index.indexing.slowlog.threshold.index.info' => '5s',
    'index.indexing.slowlog.threshold.index.debug' => '2s',
    'index.indexing.slowlog.threshold.index.trace' => '500ms',
    'index.indexing.slowlog.source' => '1000'
  }
)
puts response
PUT /my-index-000001/_settings
{
  "index.indexing.slowlog.threshold.index.warn": "10s",
  "index.indexing.slowlog.threshold.index.info": "5s",
  "index.indexing.slowlog.threshold.index.debug": "2s",
  "index.indexing.slowlog.threshold.index.trace": "500ms",
  "index.indexing.slowlog.source": "1000"
}

要包含有关触发慢速索引事件的用户信息，请使用 index.indexing.slowlog.include.user 设置。

resp = client.indices.put_settings(
    index="my-index-000001",
    body={"index.indexing.slowlog.include.user": True},
)
print(resp)
response = client.indices.put_settings(
  index: 'my-index-000001',
  body: {
    'index.indexing.slowlog.include.user' => true
  }
)
puts response
PUT /my-index-000001/_settings
{
  "index.indexing.slowlog.include.user": true
}

默认情况下，Elasticsearch 会在慢日志中记录 _source 的前 1000 个字符。您可以使用 index.indexing.slowlog.source 更改它。将其设置为 false 或 0 将完全跳过记录源，而将其设置为 true 将记录整个源，而不管大小。默认情况下，原始 _source 会重新格式化，以确保它适合一行日志。如果保留原始文档格式很重要，您可以通过将 index.indexing.slowlog.reformat 设置为 false 来关闭重新格式化，这将导致源“按原样”记录，并且可能会跨越多行日志。

索引慢日志文件在 log4j2.properties 文件中配置。

慢日志级别编辑

您可以通过设置适当的阈值来模拟搜索或索引慢日志级别，从而关闭“更详细”的记录器。例如，如果我们想模拟 index.indexing.slowlog.level: INFO，那么我们需要做的就是将 index.indexing.slowlog.threshold.index.debug 和 index.indexing.slowlog.threshold.index.trace 设置为 -1。

« 相似度模块 存储 »