EQL

Elastic Stack Serverless

事件查询语言 (EQL) 是一种用于基于事件的时间序列数据（例如日志、指标和跟踪）的查询语言。

EQL 的优势

EQL 允许您表达事件之间的关系。
许多查询语言允许您匹配单个事件。 EQL 允许您跨不同的事件类别和时间跨度匹配一系列事件。
EQL 的学习曲线很低。
EQL 语法看起来像其他常见的查询语言，例如 SQL。 EQL 允许您直观地编写和读取查询，从而实现快速、迭代的搜索。
EQL 专为安全用例而设计。
虽然您可以将其用于任何基于事件的数据，但我们创建 EQL 是为了进行威胁搜寻。 EQL 不仅支持入侵指标 (IOC) 搜索，还可以描述超出 IOC 的活动。

必需字段

除示例查询外，EQL 搜索要求搜索的数据流或索引包含时间戳字段。默认情况下，EQL 使用 Elastic Common Schema (ECS) 中的 @timestamp 字段。

EQL 搜索还需要一个事件类别字段，除非您使用 any 关键字来搜索没有事件类别字段的文档。默认情况下，EQL 使用 ECS event.category 字段。

要使用不同的时间戳或事件类别字段，请参阅指定时间戳或事件类别字段。

提示

虽然使用 EQL 不需要任何模式，但我们建议使用 ECS。 EQL 搜索默认设计为与核心 ECS 字段配合使用。

运行 EQL 搜索

使用 EQL 搜索 API 运行基本 EQL 查询。

		 GET /my-data-stream/_eql/search {
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

	

默认情况下，基本 EQL 查询在 hits.events 属性中返回 10 个最新的匹配事件。这些命中按时间戳排序，转换为自 Unix 纪元以来的毫秒数，并按升序排列。

		{
  "is_partial": false,
  "is_running": false,
  "took": 60,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "_source": {
          "@timestamp": "2099-12-07T11:07:09.000Z",
          "event": {
            "category": "process",
            "id": "aR3NWVOs",
            "sequence": 4
          },
          "process": {
            "pid": 2012,
            "name": "regsvr32.exe",
            "command_line": "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll",
            "executable": "C:\\Windows\\System32\\regsvr32.exe"
          }
        }
      },
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "xLkCaj4EujzdNSxfYLbO",
        "_source": {
          "@timestamp": "2099-12-07T11:07:10.000Z",
          "event": {
            "category": "process",
            "id": "GTSmSqgz0U",
            "sequence": 6,
            "type": "termination"
          },
          "process": {
            "pid": 2012,
            "name": "regsvr32.exe",
            "executable": "C:\\Windows\\System32\\regsvr32.exe"
          }
        }
      }
    ]
  }
}

	

使用 size 参数获取更小或更大的命中集

		 GET /my-data-stream/_eql/search {
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "size": 50
}

	

搜索事件序列

使用 EQL 的序列语法搜索一系列有序事件。以升序时间顺序列出事件项目，最近的事件最后列出

		 GET /my-data-stream/_eql/search {
  "query": """
    sequence
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

	

响应的 hits.sequences 属性包含 10 个最新的匹配序列。

		{
  ...
  "hits": {
    "total": ...,
    "sequences": [
      {
        "events": [
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "OQmfCaduce8zoHT93o4H",
            "_source": {
              "@timestamp": "2099-12-07T11:07:09.000Z",
              "event": {
                "category": "process",
                "id": "aR3NWVOs",
                "sequence": 4
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "command_line": "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              }
            }
          },
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "yDwnGIJouOYGBzP0ZE9n",
            "_source": {
              "@timestamp": "2099-12-07T11:07:10.000Z",
              "event": {
                "category": "file",
                "id": "tZ1NWVOs",
                "sequence": 5
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              },
              "file": {
                "path": "C:\\Windows\\System32\\scrobj.dll",
                "name": "scrobj.dll"
              }
            }
          }
        ]
      }
    ]
  }
}

	

使用 with maxspan 将匹配序列限制在时间跨度内

		 GET /my-data-stream/_eql/search {
  "query": """
    sequence with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

	

使用 ! 来匹配缺失事件：在给定的时间跨度内不满足条件的序列中的事件

		 GET /my-data-stream/_eql/search {
  "query": """
    sequence with maxspan=1d
      [ process where process.name == "cmd.exe" ]
      ![ process where stringContains(process.command_line, "ocx") ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

	

缺失事件在响应中表示为 missing": true

		{
  ...
  "hits": {
    "total": ...,
    "sequences": [
      {
        "events": [
          {
            "_index": ".ds-my-data-stream-2023.07.04-000001",
            "_id": "AnpTIYkBrVQ2QEgsWg94",
            "_source": {
              "@timestamp": "2099-12-07T11:06:07.000Z",
              "event": {
                "category": "process",
                "id": "cMyt5SZ2",
                "sequence": 3
              },
              "process": {
                "pid": 2012,
                "name": "cmd.exe",
                "executable": "C:\\Windows\\System32\\cmd.exe"
              }
            }
          },
          {
            "_index": "",
            "_id": "",
            "_source": {},
            "missing": true
          },
          {
            "_index": ".ds-my-data-stream-2023.07.04-000001",
            "_id": "BHpTIYkBrVQ2QEgsWg94",
            "_source": {
              "@timestamp": "2099-12-07T11:07:10.000Z",
              "event": {
                "category": "file",
                "id": "tZ1NWVOs",
                "sequence": 5
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              },
              "file": {
                "path": "C:\\Windows\\System32\\scrobj.dll",
                "name": "scrobj.dll"
              }
            }
          }
        ]
      }
    ]
  }
}

	

使用 by 关键字来匹配共享相同字段值的事件

		 GET /my-data-stream/_eql/search {
  "query": """
    sequence with maxspan=1h
      [ process where process.name == "regsvr32.exe" ] by process.pid
      [ file where stringContains(file.name, "scrobj.dll") ] by process.pid
  """
}

	

如果一个字段值应该在所有事件中共享，请使用 sequence by 关键字。以下查询等同于前一个查询。

		 GET /my-data-stream/_eql/search {
  "query": """
    sequence by process.pid with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

	

hits.sequences.join_keys 属性包含共享字段值。

		{
  ...
  "hits": ...,
    "sequences": [
      {
        "join_keys": [
          2012
        ],
        "events": ...
      }
    ]
  }
}

	

使用 until 关键字为序列指定到期事件。匹配序列必须在此事件之前结束。

		 GET /my-data-stream/_eql/search {
  "query": """
    sequence by process.pid with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
    until [ process where event.type == "termination" ]
  """
}

	

按时间顺序排列的示例事件

使用 EQL 的样本语法搜索与一个或多个连接键和一组过滤器匹配的事件。样本类似于序列，但不按时间顺序返回事件。事实上，样本查询可以在没有时间戳的数据上运行。样本查询可用于查找并非总是以相同序列发生的事件或跨越长时间发生的事件中的相关性。

		 PUT /my-index-000001 {
    "mappings": {
        "properties": {
            "ip": {
                "type":"ip"
            },
            "version": {
                "type": "version"
            },
            "missing_keyword": {
                "type": "keyword"
            },
            "@timestamp": {
              "type": "date"
            },
            "type_test": {
                "type": "keyword"
            },
            "@timestamp_pretty": {
              "type": "date",
              "format": "dd-MM-yyyy"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "host": {
              "type": "keyword"
            },
            "os": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

PUT /my-index-000002
{
    "mappings": {
        "properties": {
            "ip": {
                "type":"ip"
            },
            "@timestamp": {
              "type": "date"
            },
            "@timestamp_pretty": {
              "type": "date",
              "format": "yyyy-MM-dd"
            },
            "type_test": {
                "type": "keyword"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "host": {
              "type": "keyword"
            },
            "op_sys": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

PUT /my-index-000003
{
    "mappings": {
        "properties": {
            "host_ip": {
                "type":"ip"
            },
            "@timestamp": {
              "type": "date"
            },
            "date": {
              "type": "date"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "missing_keyword": {
                "type": "keyword"
            },
            "host": {
              "type": "keyword"
            },
            "os": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

POST /my-index-000001/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1234567891","@timestamp_pretty":"12-12-2022","missing_keyword":"test","type_test":"abc","ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":1234,"os":"win10","version":"1.0.0","id":11}
{"index":{"_id":2}}
{"@timestamp":"1234567892","@timestamp_pretty":"13-12-2022","event_type":"alert","type_test":"abc","host":"CS","uptime":5,"port":1,"os":"win10","version":"1.2.0","id":12}
{"index":{"_id":3}}
{"@timestamp":"1234567893","@timestamp_pretty":"12-12-2022","event_type":"alert","type_test":"abc","host":"farcry","uptime":1,"port":1234,"bool":false,"os":"win10","version":"2.0.0","id":13}
{"index":{"_id":4}}
{"@timestamp":"1234567894","@timestamp_pretty":"13-12-2022","event_type":"alert","type_test":"abc","host":"GTA","uptime":3,"port":12,"os":"slack","version":"10.0.0","id":14}
{"index":{"_id":5}}
{"@timestamp":"1234567895","@timestamp_pretty":"17-12-2022","event_type":"alert","host":"sniper 3d","uptime":6,"port":1234,"os":"fedora","version":"20.1.0","id":15}
{"index":{"_id":6}}
{"@timestamp":"1234568896","@timestamp_pretty":"17-12-2022","event_type":"alert","host":"doom","port":65123,"bool":true,"os":"redhat","version":"20.10.0","id":16}
{"index":{"_id":7}}
{"@timestamp":"1234567897","@timestamp_pretty":"17-12-2022","missing_keyword":"yyy","event_type":"failure","host":"doom","uptime":15,"port":1234,"bool":true,"os":"redhat","version":"20.2.0","id":17}
{"index":{"_id":8}}
{"@timestamp":"1234567898","@timestamp_pretty":"12-12-2022","missing_keyword":"test","event_type":"success","host":"doom","uptime":16,"port":512,"os":"win10","version":"1.2.3","id":18}
{"index":{"_id":9}}
{"@timestamp":"1234567899","@timestamp_pretty":"15-12-2022","missing_keyword":"test","event_type":"success","host":"GTA","port":12,"bool":true,"os":"win10","version":"1.2.3","id":19}
{"index":{"_id":10}}
{"@timestamp":"1234567893","missing_keyword":null,"ip":"10.0.0.5","event_type":"alert","host":"farcry","uptime":1,"port":1234,"bool":true,"os":"win10","version":"1.2.3","id":110}

POST /my-index-000002/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1234567991","type_test":"abc","ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":1234,"op_sys":"win10","id":21}
{"index":{"_id":2}}
{"@timestamp":"1234567992","type_test":"abc","event_type":"alert","host":"CS","uptime":5,"port":1,"op_sys":"win10","id":22}
{"index":{"_id":3}}
{"@timestamp":"1234567993","type_test":"abc","@timestamp_pretty":"2022-12-17","event_type":"alert","host":"farcry","uptime":1,"port":1234,"bool":false,"op_sys":"win10","id":23}
{"index":{"_id":4}}
{"@timestamp":"1234567994","event_type":"alert","host":"GTA","uptime":3,"port":12,"op_sys":"slack","id":24}
{"index":{"_id":5}}
{"@timestamp":"1234567995","event_type":"alert","host":"sniper 3d","uptime":6,"port":1234,"op_sys":"fedora","id":25}
{"index":{"_id":6}}
{"@timestamp":"1234568996","@timestamp_pretty":"2022-12-17","ip":"10.0.0.5","event_type":"alert","host":"doom","port":65123,"bool":true,"op_sys":"redhat","id":26}
{"index":{"_id":7}}
{"@timestamp":"1234567997","@timestamp_pretty":"2022-12-17","event_type":"failure","host":"doom","uptime":15,"port":1234,"bool":true,"op_sys":"redhat","id":27}
{"index":{"_id":8}}
{"@timestamp":"1234567998","ip":"10.0.0.1","event_type":"success","host":"doom","uptime":16,"port":512,"op_sys":"win10","id":28}
{"index":{"_id":9}}
{"@timestamp":"1234567999","ip":"10.0.0.1","event_type":"success","host":"GTA","port":12,"bool":false,"op_sys":"win10","id":29}

POST /my-index-000003/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1334567891","host_ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":12,"os":"win10","id":31}
{"index":{"_id":2}}
{"@timestamp":"1334567892","event_type":"alert","host":"CS","os":"win10","id":32}
{"index":{"_id":3}}
{"@timestamp":"1334567893","event_type":"alert","host":"farcry","bool":true,"os":"win10","id":33}
{"index":{"_id":4}}
{"@timestamp":"1334567894","event_type":"alert","host":"GTA","os":"slack","bool":true,"id":34}
{"index":{"_id":5}}
{"@timestamp":"1234567895","event_type":"alert","host":"sniper 3d","os":"fedora","id":35}
{"index":{"_id":6}}
{"@timestamp":"1234578896","host_ip":"10.0.0.1","event_type":"alert","host":"doom","bool":true,"os":"redhat","id":36}
{"index":{"_id":7}}
{"@timestamp":"1234567897","event_type":"failure","missing_keyword":"test","host":"doom","bool":true,"os":"redhat","id":37}
{"index":{"_id":8}}
{"@timestamp":"1234577898","event_type":"success","host":"doom","os":"win10","id":38,"date":"1671235200000"}
{"index":{"_id":9}}
{"@timestamp":"1234577899","host_ip":"10.0.0.5","event_type":"success","host":"GTA","bool":true,"os":"win10","id":39}

	

样本查询使用 by 关键字指定至少一个连接键，最多指定五个过滤器

		 GET /my-index*/_eql/search {
  "query": """
    sample by host
      [any where uptime > 0]
      [any where port > 100]
      [any where bool == true]
  """
}

	

默认情况下，响应的 hits.sequences 属性最多包含 10 个样本。每个样本都有一组 join_keys 和一个数组，其中包含每个过滤器的匹配事件。事件按照它们匹配的过滤器的顺序返回

		{
  ...
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          "doom"
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "7",
            "_source": {
              "@timestamp": "1234567897",
              "@timestamp_pretty": "17-12-2022",
              "missing_keyword": "yyy",
              "event_type": "failure",
              "host": "doom",
              "uptime": 15,
              "port": 1234,
              "bool": true,
              "os": "redhat",
              "version": "20.2.0",
              "id": 17
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "1",
            "_source": {
              "@timestamp": "1234567891",
              "@timestamp_pretty": "12-12-2022",
              "missing_keyword": "test",
              "type_test": "abc",
              "ip": "10.0.0.1",
              "event_type": "alert",
              "host": "doom",
              "uptime": 0,
              "port": 1234,
              "os": "win10",
              "version": "1.0.0",
              "id": 11
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568896",
              "@timestamp_pretty": "17-12-2022",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "os": "redhat",
              "version": "20.10.0",
              "id": 16
            }
          }
        ]
      },
      {
        "join_keys": [
          "farcry"
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567893",
              "@timestamp_pretty": "12-12-2022",
              "event_type": "alert",
              "type_test": "abc",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "os": "win10",
              "version": "2.0.0",
              "id": 13
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "10",
            "_source": {
              "@timestamp": "1234567893",
              "missing_keyword": null,
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": true,
              "os": "win10",
              "version": "1.2.3",
              "id": 110
            }
          },
          {
            "_index": "my-index-000003",
            "_id": "3",
            "_source": {
              "@timestamp": "1334567893",
              "event_type": "alert",
              "host": "farcry",
              "bool": true,
              "os": "win10",
              "id": 33
            }
          }
        ]
      }
    ]
  }
}

	

第一个样本中的事件对于 host 具有值 doom。
此事件与第一个过滤器匹配。
此事件与第二个过滤器匹配。
此事件与第三个过滤器匹配。
第二个样本中的事件对于 host 具有值 farcry。

您可以指定多个连接键

		 GET /my-index*/_eql/search {
  "query": """
    sample by host
      [any where uptime > 0]   by os
      [any where port > 100]   by op_sys
      [any where bool == true] by os
  """
}

	

此查询将返回样本，其中每个事件都为 os 或 op_sys 以及 host 共享相同的值。例如

		{
  ...
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          "doom",
          "redhat"
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "7",
            "_source": {
              "@timestamp": "1234567897",
              "@timestamp_pretty": "17-12-2022",
              "missing_keyword": "yyy",
              "event_type": "failure",
              "host": "doom",
              "uptime": 15,
              "port": 1234,
              "bool": true,
              "os": "redhat",
              "version": "20.2.0",
              "id": 17
            }
          },
          {
            "_index": "my-index-000002",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568996",
              "@timestamp_pretty": "2022-12-17",
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "op_sys": "redhat",
              "id": 26
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568896",
              "@timestamp_pretty": "17-12-2022",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "os": "redhat",
              "version": "20.10.0",
              "id": 16
            }
          }
        ]
      },
      {
        "join_keys": [
          "farcry",
          "win10"
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567893",
              "@timestamp_pretty": "12-12-2022",
              "event_type": "alert",
              "type_test": "abc",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "os": "win10",
              "version": "2.0.0",
              "id": 13
            }
          },
          {
            "_index": "my-index-000002",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567993",
              "type_test": "abc",
              "@timestamp_pretty": "2022-12-17",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "op_sys": "win10",
              "id": 23
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "10",
            "_source": {
              "@timestamp": "1234567893",
              "missing_keyword": null,
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": true,
              "os": "win10",
              "version": "1.2.3",
              "id": 110
            }
          }
        ]
      }
    ]
  }
}

	

此样本中的事件对于 host 具有值 doom，对于 os 或 op_sys 具有值 redhat。

默认情况下，样本查询的响应最多包含 10 个样本，每个唯一连接键集一个样本。使用 size 参数获取更小或更大的样本集。要检索每个连接键集的多个样本，请使用 max_samples_per_key 参数。样本查询不支持管道。

		 GET /my-index*/_eql/search {
  "max_samples_per_key": 2,
  "size": 20,
  "query": """
    sample
      [any where uptime > 0]   by host,os
      [any where port > 100]   by host,op_sys
      [any where bool == true] by host,os
  """
}

	

每个连接键集检索最多 2 个样本。
总共检索最多 20 个样本。

检索选定的字段

默认情况下，搜索响应中的每个命中都包含文档 _source，它是索引文档时提供的整个 JSON 对象。

您可以使用 filter_path 查询参数来过滤 API 响应。例如，以下搜索仅返回每个匹配事件的 _source 中的时间戳和 PID。

		 GET /my-data-stream/_eql/search?filter_path=hits.events._source.@timestamp,hits.events._source.process.pid {
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

	

API 返回以下响应。

		{
  "hits": {
    "events": [
      {
        "_source": {
          "@timestamp": "2099-12-07T11:07:09.000Z",
          "process": {
            "pid": 2012
          }
        }
      },
      {
        "_source": {
          "@timestamp": "2099-12-07T11:07:10.000Z",
          "process": {
            "pid": 2012
          }
        }
      }
    ]
  }
}

	

您还可以使用 fields 参数来检索和格式化响应中的特定字段。此字段与搜索 API 的 fields 参数相同。

因为它会查阅索引映射，所以 fields 参数比直接引用 _source 具有多个优点。具体来说，fields 参数

以标准化方式返回每个值，该方式与其映射类型匹配
接受多字段和字段别名
格式化日期和空间数据类型
检索运行时字段值
返回在索引时通过脚本计算的字段
使用查找运行时字段从相关索引返回字段

以下搜索请求使用 fields 参数来检索 event.type 字段、所有以 process. 开头的字段和 @timestamp 字段的值。该请求还使用 filter_path 查询参数来排除每个命中的 _source。

		 GET /my-data-stream/_eql/search?filter_path=-hits.events._source {
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "fields": [
    "event.type",
    "process.*",
    {
      "field": "@timestamp",
      "format": "epoch_millis"
    }
  ]
}

	

接受完整字段名称和通配符模式。
使用 format 参数为字段的值应用自定义格式。

响应包括每个命中的 fields 部分中的平面列表形式的值。

		{
  ...
  "hits": {
    "total": ...,
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "fields": {
          "process.name": [
            "regsvr32.exe"
          ],
          "process.name.keyword": [
            "regsvr32.exe"
          ],
          "@timestamp": [
            "4100324829000"
          ],
          "process.command_line": [
            "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll"
          ],
          "process.command_line.keyword": [
            "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll"
          ],
          "process.executable.keyword": [
            "C:\\Windows\\System32\\regsvr32.exe"
          ],
          "process.pid": [
            2012
          ],
          "process.executable": [
            "C:\\Windows\\System32\\regsvr32.exe"
          ]
        }
      },
      ....
    ]
  }
}

	

使用运行时字段

使用 runtime_mappings 参数在搜索期间提取和创建运行时字段。使用 fields 参数将运行时字段包含在响应中。

以下搜索从 @timestamp 创建 day_of_week 运行时字段，并在响应中返回它。

		 GET /my-data-stream/_eql/search?filter_path=-hits.events._source {
  "runtime_mappings": {
    "day_of_week": {
      "type": "keyword",
      "script": "emit(doc['@timestamp'].value.dayOfWeekEnum.toString())"
    }
  },
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "fields": [
    "@timestamp",
    "day_of_week"
  ]
}

	

API 返回

		{
  ...
  "hits": {
    "total": ...,
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "fields": {
          "@timestamp": [
            "2099-12-07T11:07:09.000Z"
          ],
          "day_of_week": [
            "MONDAY"
          ]
        }
      },
      ....
    ]
  }
}

	

指定时间戳或事件类别字段

EQL 搜索 API 默认使用 ECS 中的 @timestamp 和 event.category 字段。要指定不同的字段，请使用 timestamp_field 和 event_category_field 参数

		 GET /my-data-stream/_eql/search {
  "timestamp_field": "file.accessed",
  "event_category_field": "file.type",
  "query": """
    file where (file.size > 1 and file.type == "file")
  """
}

	

事件类别字段必须映射为 keyword 系列字段类型。时间戳字段应映射为 date 字段类型。不支持 date_nanos 时间戳字段。您不能使用 nested 字段或 nested 字段的子字段作为时间戳或事件类别字段。

指定排序决胜因素

默认情况下，EQL 搜索 API 按时间戳返回匹配的命中项。如果两个或多个事件共享相同的时间戳，Elasticsearch 会使用一个决胜字段值，以升序对事件进行排序。 Elasticsearch 会将没有决胜值的事件排在具有决胜值的事件之后。

如果您未指定决胜字段，或者事件也共享相同的决胜值，Elasticsearch 会将这些事件视为并发事件，并且可能不会以一致的排序顺序返回它们。

要指定决胜字段，请使用 tiebreaker_field 参数。如果您使用 ECS，我们建议使用 event.sequence 作为决胜字段。

		 GET /my-data-stream/_eql/search {
  "tiebreaker_field": "event.sequence",
  "query": """
    process where process.name == "cmd.exe" and stringContains(process.executable, "System32")
  """
}

	

filter 参数使用 Query DSL 来限制 EQL 查询运行的文档。

		 GET /my-data-stream/_eql/search {
  "filter": {
    "range": {
      "@timestamp": {
        "gte": "now-1d/d",
        "lt": "now/d"
      }
    }
  },
  "query": """
    file where (file.type == "file" and file.name == "cmd.exe")
  """
}

	

运行异步 EQL 搜索

默认情况下，EQL 搜索请求是同步的，并且在返回响应之前等待完整的结果。但是，对于跨大型数据集或冻结数据的搜索，获得完整结果可能需要更长的时间。

为了避免长时间的等待，请运行异步 EQL 搜索。将 wait_for_completion_timeout 设置为您希望等待同步结果的持续时间。

		 GET /my-data-stream/_eql/search {
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

	

如果请求未在超时期限内完成，则搜索将变为异步，并返回包含以下内容的响应：

搜索 ID
is_partial 值为 true，表示搜索结果不完整
is_running 值为 true，表示搜索正在进行中

异步搜索继续在后台运行，而不会阻塞其他请求。

		{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_partial": true,
  "is_running": true,
  "took": 2000,
  "timed_out": false,
  "hits": ...
}

	

要检查异步搜索的进度，请使用 get async EQL search API 以及搜索 ID。在 wait_for_completion_timeout 参数中指定您希望获得完整结果的持续时间。

		 GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?wait_for_completion_timeout=2s 
	

如果响应的 is_running 值为 false，则表示异步搜索已完成。如果 is_partial 值为 false，则表示返回的搜索结果是完整的。

		{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_partial": false,
  "is_running": false,
  "took": 2000,
  "timed_out": false,
  "hits": ...
}

	

另一种更轻量级的方法是使用 get async EQL status API 以及搜索 ID 来检查异步搜索的进度。

		 GET /_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE= 
	

		{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_running": false,
  "is_partial": false,
  "expiration_time_in_millis": 1611690295000,
  "completion_status": 200
}

	

更改搜索保留期限

默认情况下，EQL 搜索 API 会将异步搜索存储五天。在此期限之后，任何搜索及其结果都将被删除。使用 keep_alive 参数来更改此保留期限

		 GET /my-data-stream/_eql/search {
  "keep_alive": "2d",
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

	

您可以使用 get async EQL search API 的 keep_alive 参数来稍后更改保留期限。新的保留期限在 get 请求运行后开始。

		 GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?keep_alive=5d 
	

在 keep_alive 期限结束之前，可以使用 delete async EQL search API 手动删除异步 EQL 搜索。如果搜索仍在进行中，Elasticsearch 会取消搜索请求。

		 DELETE /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE= 
	

存储同步 EQL 搜索

默认情况下，EQL 搜索 API 仅存储异步搜索。要保存同步搜索，请将 keep_on_completion 设置为 true

		 GET /my-data-stream/_eql/search {
  "keep_on_completion": true,
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

	

响应包含一个搜索 ID。 is_partial 和 is_running 均为 false，表示 EQL 搜索是同步的，并返回了完整的结果。

		{
  "id": "FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY=",
  "is_partial": false,
  "is_running": false,
  "took": 52,
  "timed_out": false,
  "hits": ...
}

	

稍后可以使用 get async EQL search API 获取相同的结果

		 GET /_eql/search/FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY= 
	

保存的同步搜索仍然受到 keep_alive 参数的保留期限的约束。当此期限结束时，搜索及其结果将被删除。

您还可以通过使用 get async EQL status API 来仅检查保存的同步搜索的状态，而不获取结果。

您还可以使用 delete async EQL search API 手动删除保存的同步搜索。

跨集群运行 EQL 搜索

警告

此功能目前为技术预览版，并且可能会在未来的版本中更改或删除。Elastic 将努力解决任何问题，但技术预览版中的功能不受官方 GA 功能的支持 SLA 的约束。

EQL 搜索 API 支持跨集群搜索。但是，如果本地和远程集群的版本早于 7.17.7（含）或早于 8.5.1（含），则它们必须使用相同的 Elasticsearch 版本。

以下集群更新设置请求添加了两个远程集群：cluster_one 和 cluster_two。

		 PUT /_cluster/settings {
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_one": {
          "seeds": [
            "127.0.0.1:9300"
          ]
        },
        "cluster_two": {
          "seeds": [
            "127.0.0.1:9301"
          ]
        }
      }
    }
  }
}

	

要在远程集群上定位数据流或索引，请使用 <cluster>:<target> 语法。

		 GET /cluster_one:my-data-stream,cluster_two:my-data-stream/_eql/search {
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

	

EQL 断路器设置

相关的断路器设置可以在断路器页面中找到。