« 检索运行时字段 使用运行时字段探索数据 »
Elastic 文档 ›Elasticsearch 指南 [8.17] ›映射 ›运行时字段

为运行时字段建立索引

编辑

为运行时字段建立索引

编辑

运行时字段由它们运行的上下文定义。例如，您可以在搜索查询的上下文中或索引映射的runtime 部分中定义运行时字段。如果您决定为运行时字段建立索引以获得更高的性能，只需将完整的运行时字段定义（包括脚本）移动到索引映射的上下文中。Elasticsearch 会自动使用这些索引字段来驱动查询，从而实现快速响应。此功能意味着您只需编写一次脚本，即可将其应用于任何支持运行时字段的上下文。

目前不支持为 composite 运行时字段建立索引。

然后，您可以使用运行时字段来限制 Elasticsearch 需要计算值的字段数量。将索引字段与运行时字段结合使用，可以灵活地处理您索引的数据以及如何定义其他字段的查询。

为运行时字段建立索引后，您无法更新包含的脚本。如果您需要更改脚本，请创建一个具有更新脚本的新字段。

例如，假设您的公司想要更换一些旧的压力阀。连接的传感器只能报告真实读数的一部分。您决定基于报告的读数来计算值，而不是为压力阀配备新的传感器。根据报告的数据，您在 my-index-000001 的映射中定义了以下字段

resp = client.indices.create(
    index="my-index-000001",
    mappings={
        "properties": {
            "timestamp": {
                "type": "date"
            },
            "temperature": {
                "type": "long"
            },
            "voltage": {
                "type": "double"
            },
            "node": {
                "type": "keyword"
            }
        }
    },
)
print(resp)
response = client.indices.create(
  index: 'my-index-000001',
  body: {
    mappings: {
      properties: {
        timestamp: {
          type: 'date'
        },
        temperature: {
          type: 'long'
        },
        voltage: {
          type: 'double'
        },
        node: {
          type: 'keyword'
        }
      }
    }
  }
)
puts response
const response = await client.indices.create({
  index: "my-index-000001",
  mappings: {
    properties: {
      timestamp: {
        type: "date",
      },
      temperature: {
        type: "long",
      },
      voltage: {
        type: "double",
      },
      node: {
        type: "keyword",
      },
    },
  },
});
console.log(response);
PUT my-index-000001/
{
  "mappings": {
    "properties": {
      "timestamp": {
        "type": "date"
      },
      "temperature": {
        "type": "long"
      },
      "voltage": {
        "type": "double"
      },
      "node": {
        "type": "keyword"
      }
    }
  }
}

然后，您从传感器批量索引一些样本数据。此数据包括每个传感器的 voltage 读数

resp = client.bulk(
    index="my-index-000001",
    refresh=True,
    operations=[
        {
            "index": {}
        },
        {
            "timestamp": 1516729294000,
            "temperature": 200,
            "voltage": 5.2,
            "node": "a"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516642894000,
            "temperature": 201,
            "voltage": 5.8,
            "node": "b"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516556494000,
            "temperature": 202,
            "voltage": 5.1,
            "node": "a"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516470094000,
            "temperature": 198,
            "voltage": 5.6,
            "node": "b"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516383694000,
            "temperature": 200,
            "voltage": 4.2,
            "node": "c"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516297294000,
            "temperature": 202,
            "voltage": 4,
            "node": "c"
        }
    ],
)
print(resp)
response = client.bulk(
  index: 'my-index-000001',
  refresh: true,
  body: [
    {
      index: {}
    },
    {
      timestamp: 1_516_729_294_000,
      temperature: 200,
      voltage: 5.2,
      node: 'a'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_642_894_000,
      temperature: 201,
      voltage: 5.8,
      node: 'b'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_556_494_000,
      temperature: 202,
      voltage: 5.1,
      node: 'a'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_470_094_000,
      temperature: 198,
      voltage: 5.6,
      node: 'b'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_383_694_000,
      temperature: 200,
      voltage: 4.2,
      node: 'c'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_297_294_000,
      temperature: 202,
      voltage: 4,
      node: 'c'
    }
  ]
)
puts response
const response = await client.bulk({
  index: "my-index-000001",
  refresh: "true",
  operations: [
    {
      index: {},
    },
    {
      timestamp: 1516729294000,
      temperature: 200,
      voltage: 5.2,
      node: "a",
    },
    {
      index: {},
    },
    {
      timestamp: 1516642894000,
      temperature: 201,
      voltage: 5.8,
      node: "b",
    },
    {
      index: {},
    },
    {
      timestamp: 1516556494000,
      temperature: 202,
      voltage: 5.1,
      node: "a",
    },
    {
      index: {},
    },
    {
      timestamp: 1516470094000,
      temperature: 198,
      voltage: 5.6,
      node: "b",
    },
    {
      index: {},
    },
    {
      timestamp: 1516383694000,
      temperature: 200,
      voltage: 4.2,
      node: "c",
    },
    {
      index: {},
    },
    {
      timestamp: 1516297294000,
      temperature: 202,
      voltage: 4,
      node: "c",
    },
  ],
});
console.log(response);
POST my-index-000001/_bulk?refresh=true
{"index":{}}
{"timestamp": 1516729294000, "temperature": 200, "voltage": 5.2, "node": "a"}
{"index":{}}
{"timestamp": 1516642894000, "temperature": 201, "voltage": 5.8, "node": "b"}
{"index":{}}
{"timestamp": 1516556494000, "temperature": 202, "voltage": 5.1, "node": "a"}
{"index":{}}
{"timestamp": 1516470094000, "temperature": 198, "voltage": 5.6, "node": "b"}
{"index":{}}
{"timestamp": 1516383694000, "temperature": 200, "voltage": 4.2, "node": "c"}
{"index":{}}
{"timestamp": 1516297294000, "temperature": 202, "voltage": 4.0, "node": "c"}

与一些现场工程师交谈后，您意识到传感器应该至少报告当前值的两倍，并且可能更高。您创建一个名为 voltage_corrected 的运行时字段，该字段检索当前电压并将其乘以 2

resp = client.indices.put_mapping(
    index="my-index-000001",
    runtime={
        "voltage_corrected": {
            "type": "double",
            "script": {
                "source": "\n        emit(doc['voltage'].value * params['multiplier'])\n        ",
                "params": {
                    "multiplier": 2
                }
            }
        }
    },
)
print(resp)
response = client.indices.put_mapping(
  index: 'my-index-000001',
  body: {
    runtime: {
      voltage_corrected: {
        type: 'double',
        script: {
          source: "\n        emit(doc['voltage'].value * params['multiplier'])\n        ",
          params: {
            multiplier: 2
          }
        }
      }
    }
  }
)
puts response
const response = await client.indices.putMapping({
  index: "my-index-000001",
  runtime: {
    voltage_corrected: {
      type: "double",
      script: {
        source:
          "\n        emit(doc['voltage'].value * params['multiplier'])\n        ",
        params: {
          multiplier: 2,
        },
      },
    },
  },
});
console.log(response);
PUT my-index-000001/_mapping
{
  "runtime": {
    "voltage_corrected": {
      "type": "double",
      "script": {
        "source": """
        emit(doc['voltage'].value * params['multiplier'])
        """,
        "params": {
          "multiplier": 2
        }
      }
    }
  }
}

您使用 _search API 上的 fields 参数检索计算的值

resp = client.search(
    index="my-index-000001",
    fields=[
        "voltage_corrected",
        "node"
    ],
    size=2,
)
print(resp)
response = client.search(
  index: 'my-index-000001',
  body: {
    fields: [
      'voltage_corrected',
      'node'
    ],
    size: 2
  }
)
puts response
const response = await client.search({
  index: "my-index-000001",
  fields: ["voltage_corrected", "node"],
  size: 2,
});
console.log(response);
GET my-index-000001/_search
{
  "fields": [
    "voltage_corrected",
    "node"
  ],
  "size": 2
}

在查看传感器数据并进行一些测试后，您确定报告的传感器数据的乘数应为 4。为了获得更高的性能，您决定使用新的 multiplier 参数为 voltage_corrected 运行时字段建立索引。

在名为 my-index-000001 的新索引中，将 voltage_corrected 运行时字段定义复制到新索引的映射中。就这么简单！您可以添加一个名为 on_script_error 的可选参数，该参数确定如果脚本在索引时引发错误（默认情况下）是否拒绝整个文档。

resp = client.indices.create(
    index="my-index-000001",
    mappings={
        "properties": {
            "timestamp": {
                "type": "date"
            },
            "temperature": {
                "type": "long"
            },
            "voltage": {
                "type": "double"
            },
            "node": {
                "type": "keyword"
            },
            "voltage_corrected": {
                "type": "double",
                "on_script_error": "fail",
                "script": {
                    "source": "\n        emit(doc['voltage'].value * params['multiplier'])\n        ",
                    "params": {
                        "multiplier": 4
                    }
                }
            }
        }
    },
)
print(resp)
response = client.indices.create(
  index: 'my-index-000001',
  body: {
    mappings: {
      properties: {
        timestamp: {
          type: 'date'
        },
        temperature: {
          type: 'long'
        },
        voltage: {
          type: 'double'
        },
        node: {
          type: 'keyword'
        },
        voltage_corrected: {
          type: 'double',
          on_script_error: 'fail',
          script: {
            source: "\n        emit(doc['voltage'].value * params['multiplier'])\n        ",
            params: {
              multiplier: 4
            }
          }
        }
      }
    }
  }
)
puts response
const response = await client.indices.create({
  index: "my-index-000001",
  mappings: {
    properties: {
      timestamp: {
        type: "date",
      },
      temperature: {
        type: "long",
      },
      voltage: {
        type: "double",
      },
      node: {
        type: "keyword",
      },
      voltage_corrected: {
        type: "double",
        on_script_error: "fail",
        script: {
          source:
            "\n        emit(doc['voltage'].value * params['multiplier'])\n        ",
          params: {
            multiplier: 4,
          },
        },
      },
    },
  },
});
console.log(response);
PUT my-index-000001/
{
  "mappings": {
    "properties": {
      "timestamp": {
        "type": "date"
      },
      "temperature": {
        "type": "long"
      },
      "voltage": {
        "type": "double"
      },
      "node": {
        "type": "keyword"
      },
      "voltage_corrected": {
        "type": "double",
        "on_script_error": "fail", 
        "script": {
          "source": """
        emit(doc['voltage'].value * params['multiplier'])
        """,
          "params": {
            "multiplier": 4
          }
        }
      }
    }
  }
}

如果脚本在索引时引发错误，则导致拒绝整个文档。将值设置为 ignore 将在文档的 _ignored 元数据字段中注册该字段并继续索引。

将一些来自传感器的示例数据批量索引到 my-index-000001 索引中

resp = client.bulk(
    index="my-index-000001",
    refresh=True,
    operations=[
        {
            "index": {}
        },
        {
            "timestamp": 1516729294000,
            "temperature": 200,
            "voltage": 5.2,
            "node": "a"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516642894000,
            "temperature": 201,
            "voltage": 5.8,
            "node": "b"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516556494000,
            "temperature": 202,
            "voltage": 5.1,
            "node": "a"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516470094000,
            "temperature": 198,
            "voltage": 5.6,
            "node": "b"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516383694000,
            "temperature": 200,
            "voltage": 4.2,
            "node": "c"
        },
        {
            "index": {}
        },
        {
            "timestamp": 1516297294000,
            "temperature": 202,
            "voltage": 4,
            "node": "c"
        }
    ],
)
print(resp)
response = client.bulk(
  index: 'my-index-000001',
  refresh: true,
  body: [
    {
      index: {}
    },
    {
      timestamp: 1_516_729_294_000,
      temperature: 200,
      voltage: 5.2,
      node: 'a'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_642_894_000,
      temperature: 201,
      voltage: 5.8,
      node: 'b'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_556_494_000,
      temperature: 202,
      voltage: 5.1,
      node: 'a'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_470_094_000,
      temperature: 198,
      voltage: 5.6,
      node: 'b'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_383_694_000,
      temperature: 200,
      voltage: 4.2,
      node: 'c'
    },
    {
      index: {}
    },
    {
      timestamp: 1_516_297_294_000,
      temperature: 202,
      voltage: 4,
      node: 'c'
    }
  ]
)
puts response
const response = await client.bulk({
  index: "my-index-000001",
  refresh: "true",
  operations: [
    {
      index: {},
    },
    {
      timestamp: 1516729294000,
      temperature: 200,
      voltage: 5.2,
      node: "a",
    },
    {
      index: {},
    },
    {
      timestamp: 1516642894000,
      temperature: 201,
      voltage: 5.8,
      node: "b",
    },
    {
      index: {},
    },
    {
      timestamp: 1516556494000,
      temperature: 202,
      voltage: 5.1,
      node: "a",
    },
    {
      index: {},
    },
    {
      timestamp: 1516470094000,
      temperature: 198,
      voltage: 5.6,
      node: "b",
    },
    {
      index: {},
    },
    {
      timestamp: 1516383694000,
      temperature: 200,
      voltage: 4.2,
      node: "c",
    },
    {
      index: {},
    },
    {
      timestamp: 1516297294000,
      temperature: 202,
      voltage: 4,
      node: "c",
    },
  ],
});
console.log(response);
POST my-index-000001/_bulk?refresh=true
{ "index": {}}
{ "timestamp": 1516729294000, "temperature": 200, "voltage": 5.2, "node": "a"}
{ "index": {}}
{ "timestamp": 1516642894000, "temperature": 201, "voltage": 5.8, "node": "b"}
{ "index": {}}
{ "timestamp": 1516556494000, "temperature": 202, "voltage": 5.1, "node": "a"}
{ "index": {}}
{ "timestamp": 1516470094000, "temperature": 198, "voltage": 5.6, "node": "b"}
{ "index": {}}
{ "timestamp": 1516383694000, "temperature": 200, "voltage": 4.2, "node": "c"}
{ "index": {}}
{ "timestamp": 1516297294000, "temperature": 202, "voltage": 4.0, "node": "c"}

您现在可以在搜索查询中检索计算值，并根据精确值查找文档。以下范围查询返回所有计算的 voltage_corrected 大于或等于 16 但小于或等于 20 的文档。同样，使用 _search API 上的 fields 参数来检索您想要的字段

resp = client.search(
    index="my-index-000001",
    query={
        "range": {
            "voltage_corrected": {
                "gte": 16,
                "lte": 20,
                "boost": 1
            }
        }
    },
    fields=[
        "voltage_corrected",
        "node"
    ],
)
print(resp)
const response = await client.search({
  index: "my-index-000001",
  query: {
    range: {
      voltage_corrected: {
        gte: 16,
        lte: 20,
        boost: 1,
      },
    },
  },
  fields: ["voltage_corrected", "node"],
});
console.log(response);
POST my-index-000001/_search
{
  "query": {
    "range": {
      "voltage_corrected": {
        "gte": 16,
        "lte": 20,
        "boost": 1.0
      }
    }
  },
  "fields": ["voltage_corrected", "node"]
}

响应包括匹配范围查询的文档的 voltage_corrected 字段，该字段基于包含脚本的计算值

{
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "my-index-000001",
        "_id" : "yoSLrHgBdg9xpPrUZz_P",
        "_score" : 1.0,
        "_source" : {
          "timestamp" : 1516383694000,
          "temperature" : 200,
          "voltage" : 4.2,
          "node" : "c"
        },
        "fields" : {
          "voltage_corrected" : [
            16.8
          ],
          "node" : [
            "c"
          ]
        }
      },
      {
        "_index" : "my-index-000001",
        "_id" : "y4SLrHgBdg9xpPrUZz_P",
        "_score" : 1.0,
        "_source" : {
          "timestamp" : 1516297294000,
          "temperature" : 202,
          "voltage" : 4.0,
          "node" : "c"
        },
        "fields" : {
          "voltage_corrected" : [
            16.0
          ],
          "node" : [
            "c"
          ]
        }
      }
    ]
  }
}
« 检索运行时字段 使用运行时字段探索数据 »