« 正则表达式语法 EQL 语法参考 »
Elastic 文档 ›Elasticsearch 指南 [8.16]

EQL 搜索

编辑

EQL 搜索

编辑

事件查询语言 (EQL) 是一种用于基于事件的时间序列数据的查询语言，例如日志、指标和追踪。

EQL 的优势

编辑
  • EQL 允许您表达事件之间的关系。
    许多查询语言允许您匹配单个事件。EQL 允许您跨不同事件类别和时间跨度匹配一系列事件。
  • EQL 学习曲线平缓。
    EQL 语法 类似于其他常见的查询语言，例如 SQL。EQL 允许您直观地编写和读取查询，从而实现快速、迭代的搜索。
  • EQL 专为安全用例而设计。
    虽然您可以将其用于任何基于事件的数据，但我们创建 EQL 用于威胁狩猎。EQL 不仅支持入侵指标 (IOC) 搜索，还可以描述超出 IOC 的活动。

必需字段

编辑

除示例查询外，EQL 搜索要求被搜索的数据流或索引包含一个 _时间戳_ 字段。默认情况下，EQL 使用来自 Elastic 通用架构 (ECS) 的 @timestamp 字段。

EQL 搜索还需要一个 _事件类别_ 字段，除非您使用 any 关键字 来搜索没有事件类别字段的文档。默认情况下，EQL 使用 ECS event.category 字段。

要使用不同的时间戳或事件类别字段，请参见 指定时间戳或事件类别字段。

虽然使用 EQL 不需要任何模式，但我们建议使用 ECS。EQL 搜索默认情况下设计为与核心 ECS 字段一起使用。

运行 EQL 搜索

编辑

使用 EQL 搜索 API 来运行 基本的 EQL 查询。

resp = client.eql.search(
    index="my-data-stream",
    query="\n    process where process.name == \"regsvr32.exe\"\n  ",
)
print(resp)
response = client.eql.search(
  index: 'my-data-stream',
  body: {
    query: "\n    process where process.name == \"regsvr32.exe\"\n  "
  }
)
puts response
const response = await client.eql.search({
  index: "my-data-stream",
  query: '\n    process where process.name == "regsvr32.exe"\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

默认情况下，基本的 EQL 查询返回 hits.events 属性中最近匹配的 10 个事件。这些命中结果按时间戳排序，转换为自 Unix纪元 以来的毫秒数，按升序排列。

{
  "is_partial": false,
  "is_running": false,
  "took": 60,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "_source": {
          "@timestamp": "2099-12-07T11:07:09.000Z",
          "event": {
            "category": "process",
            "id": "aR3NWVOs",
            "sequence": 4
          },
          "process": {
            "pid": 2012,
            "name": "regsvr32.exe",
            "command_line": "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll",
            "executable": "C:\\Windows\\System32\\regsvr32.exe"
          }
        }
      },
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "xLkCaj4EujzdNSxfYLbO",
        "_source": {
          "@timestamp": "2099-12-07T11:07:10.000Z",
          "event": {
            "category": "process",
            "id": "GTSmSqgz0U",
            "sequence": 6,
            "type": "termination"
          },
          "process": {
            "pid": 2012,
            "name": "regsvr32.exe",
            "executable": "C:\\Windows\\System32\\regsvr32.exe"
          }
        }
      }
    ]
  }
}

使用 size 参数可以获取较小或较大的命中结果集。

resp = client.eql.search(
    index="my-data-stream",
    query="\n    process where process.name == \"regsvr32.exe\"\n  ",
    size=50,
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  query: '\n    process where process.name == "regsvr32.exe"\n  ',
  size: 50,
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "size": 50
}

搜索事件序列

编辑

使用 EQL 的 序列语法 来搜索一系列有序事件。按升序时间顺序列出事件项，最近的事件列在最后。

resp = client.eql.search(
    index="my-data-stream",
    query="\n    sequence\n      [ process where process.name == \"regsvr32.exe\" ]\n      [ file where stringContains(file.name, \"scrobj.dll\") ]\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  query:
    '\n    sequence\n      [ process where process.name == "regsvr32.exe" ]\n      [ file where stringContains(file.name, "scrobj.dll") ]\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "query": """
    sequence
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

响应的 hits.sequences 属性包含最近匹配的 10 个序列。

{
  ...
  "hits": {
    "total": ...,
    "sequences": [
      {
        "events": [
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "OQmfCaduce8zoHT93o4H",
            "_source": {
              "@timestamp": "2099-12-07T11:07:09.000Z",
              "event": {
                "category": "process",
                "id": "aR3NWVOs",
                "sequence": 4
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "command_line": "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              }
            }
          },
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "yDwnGIJouOYGBzP0ZE9n",
            "_source": {
              "@timestamp": "2099-12-07T11:07:10.000Z",
              "event": {
                "category": "file",
                "id": "tZ1NWVOs",
                "sequence": 5
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              },
              "file": {
                "path": "C:\\Windows\\System32\\scrobj.dll",
                "name": "scrobj.dll"
              }
            }
          }
        ]
      }
    ]
  }
}

使用 with maxspan 将匹配的序列限制在一定时间范围内。

resp = client.eql.search(
    index="my-data-stream",
    query="\n    sequence with maxspan=1h\n      [ process where process.name == \"regsvr32.exe\" ]\n      [ file where stringContains(file.name, \"scrobj.dll\") ]\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  query:
    '\n    sequence with maxspan=1h\n      [ process where process.name == "regsvr32.exe" ]\n      [ file where stringContains(file.name, "scrobj.dll") ]\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "query": """
    sequence with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

使用 ! 匹配 缺失事件：在给定时间范围内不满足条件的序列中的事件。

resp = client.eql.search(
    index="my-data-stream",
    query="\n    sequence with maxspan=1d\n      [ process where process.name == \"cmd.exe\" ]\n      ![ process where stringContains(process.command_line, \"ocx\") ]\n      [ file where stringContains(file.name, \"scrobj.dll\") ]\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  query:
    '\n    sequence with maxspan=1d\n      [ process where process.name == "cmd.exe" ]\n      ![ process where stringContains(process.command_line, "ocx") ]\n      [ file where stringContains(file.name, "scrobj.dll") ]\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "query": """
    sequence with maxspan=1d
      [ process where process.name == "cmd.exe" ]
      ![ process where stringContains(process.command_line, "ocx") ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

缺失事件在响应中显示为 missing": true

{
  ...
  "hits": {
    "total": ...,
    "sequences": [
      {
        "events": [
          {
            "_index": ".ds-my-data-stream-2023.07.04-000001",
            "_id": "AnpTIYkBrVQ2QEgsWg94",
            "_source": {
              "@timestamp": "2099-12-07T11:06:07.000Z",
              "event": {
                "category": "process",
                "id": "cMyt5SZ2",
                "sequence": 3
              },
              "process": {
                "pid": 2012,
                "name": "cmd.exe",
                "executable": "C:\\Windows\\System32\\cmd.exe"
              }
            }
          },
          {
            "_index": "",
            "_id": "",
            "_source": {},
            "missing": true
          },
          {
            "_index": ".ds-my-data-stream-2023.07.04-000001",
            "_id": "BHpTIYkBrVQ2QEgsWg94",
            "_source": {
              "@timestamp": "2099-12-07T11:07:10.000Z",
              "event": {
                "category": "file",
                "id": "tZ1NWVOs",
                "sequence": 5
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              },
              "file": {
                "path": "C:\\Windows\\System32\\scrobj.dll",
                "name": "scrobj.dll"
              }
            }
          }
        ]
      }
    ]
  }
}

使用 by 关键字 来匹配共享相同字段值的事件。

resp = client.eql.search(
    index="my-data-stream",
    query="\n    sequence with maxspan=1h\n      [ process where process.name == \"regsvr32.exe\" ] by process.pid\n      [ file where stringContains(file.name, \"scrobj.dll\") ] by process.pid\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  query:
    '\n    sequence with maxspan=1h\n      [ process where process.name == "regsvr32.exe" ] by process.pid\n      [ file where stringContains(file.name, "scrobj.dll") ] by process.pid\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "query": """
    sequence with maxspan=1h
      [ process where process.name == "regsvr32.exe" ] by process.pid
      [ file where stringContains(file.name, "scrobj.dll") ] by process.pid
  """
}

如果字段值应该在所有事件中共享，请使用 sequence by 关键字。以下查询与之前的查询等效。

resp = client.eql.search(
    index="my-data-stream",
    query="\n    sequence by process.pid with maxspan=1h\n      [ process where process.name == \"regsvr32.exe\" ]\n      [ file where stringContains(file.name, \"scrobj.dll\") ]\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  query:
    '\n    sequence by process.pid with maxspan=1h\n      [ process where process.name == "regsvr32.exe" ]\n      [ file where stringContains(file.name, "scrobj.dll") ]\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "query": """
    sequence by process.pid with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

hits.sequences.join_keys 属性包含共享的字段值。

{
  ...
  "hits": ...,
    "sequences": [
      {
        "join_keys": [
          2012
        ],
        "events": ...
      }
    ]
  }
}

使用 until 关键字 为序列指定一个过期事件。匹配的序列必须在此事件之前结束。

resp = client.eql.search(
    index="my-data-stream",
    query="\n    sequence by process.pid with maxspan=1h\n      [ process where process.name == \"regsvr32.exe\" ]\n      [ file where stringContains(file.name, \"scrobj.dll\") ]\n    until [ process where event.type == \"termination\" ]\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  query:
    '\n    sequence by process.pid with maxspan=1h\n      [ process where process.name == "regsvr32.exe" ]\n      [ file where stringContains(file.name, "scrobj.dll") ]\n    until [ process where event.type == "termination" ]\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "query": """
    sequence by process.pid with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
    until [ process where event.type == "termination" ]
  """
}

示例按时间顺序无序的事件

编辑

使用 EQL 的 样本语法 来搜索匹配一个或多个连接键和一组过滤器的事件。样本类似于序列，但不按时间顺序返回事件。实际上，样本查询可以在没有时间戳的数据上运行。样本查询可用于查找并不总是按相同顺序发生或跨越长时间跨度发生的事件中的相关性。

单击以显示以下示例中使用的样本数据 
resp = client.indices.create(
    index="my-index-000001",
    mappings={
        "properties": {
            "ip": {
                "type": "ip"
            },
            "version": {
                "type": "version"
            },
            "missing_keyword": {
                "type": "keyword"
            },
            "@timestamp": {
                "type": "date"
            },
            "type_test": {
                "type": "keyword"
            },
            "@timestamp_pretty": {
                "type": "date",
                "format": "dd-MM-yyyy"
            },
            "event_type": {
                "type": "keyword"
            },
            "event": {
                "properties": {
                    "category": {
                        "type": "alias",
                        "path": "event_type"
                    }
                }
            },
            "host": {
                "type": "keyword"
            },
            "os": {
                "type": "keyword"
            },
            "bool": {
                "type": "boolean"
            },
            "uptime": {
                "type": "long"
            },
            "port": {
                "type": "long"
            }
        }
    },
)
print(resp)

resp1 = client.indices.create(
    index="my-index-000002",
    mappings={
        "properties": {
            "ip": {
                "type": "ip"
            },
            "@timestamp": {
                "type": "date"
            },
            "@timestamp_pretty": {
                "type": "date",
                "format": "yyyy-MM-dd"
            },
            "type_test": {
                "type": "keyword"
            },
            "event_type": {
                "type": "keyword"
            },
            "event": {
                "properties": {
                    "category": {
                        "type": "alias",
                        "path": "event_type"
                    }
                }
            },
            "host": {
                "type": "keyword"
            },
            "op_sys": {
                "type": "keyword"
            },
            "bool": {
                "type": "boolean"
            },
            "uptime": {
                "type": "long"
            },
            "port": {
                "type": "long"
            }
        }
    },
)
print(resp1)

resp2 = client.indices.create(
    index="my-index-000003",
    mappings={
        "properties": {
            "host_ip": {
                "type": "ip"
            },
            "@timestamp": {
                "type": "date"
            },
            "date": {
                "type": "date"
            },
            "event_type": {
                "type": "keyword"
            },
            "event": {
                "properties": {
                    "category": {
                        "type": "alias",
                        "path": "event_type"
                    }
                }
            },
            "missing_keyword": {
                "type": "keyword"
            },
            "host": {
                "type": "keyword"
            },
            "os": {
                "type": "keyword"
            },
            "bool": {
                "type": "boolean"
            },
            "uptime": {
                "type": "long"
            },
            "port": {
                "type": "long"
            }
        }
    },
)
print(resp2)

resp3 = client.bulk(
    index="my-index-000001",
    refresh=True,
    operations=[
        {
            "index": {
                "_id": 1
            }
        },
        {
            "@timestamp": "1234567891",
            "@timestamp_pretty": "12-12-2022",
            "missing_keyword": "test",
            "type_test": "abc",
            "ip": "10.0.0.1",
            "event_type": "alert",
            "host": "doom",
            "uptime": 0,
            "port": 1234,
            "os": "win10",
            "version": "1.0.0",
            "id": 11
        },
        {
            "index": {
                "_id": 2
            }
        },
        {
            "@timestamp": "1234567892",
            "@timestamp_pretty": "13-12-2022",
            "event_type": "alert",
            "type_test": "abc",
            "host": "CS",
            "uptime": 5,
            "port": 1,
            "os": "win10",
            "version": "1.2.0",
            "id": 12
        },
        {
            "index": {
                "_id": 3
            }
        },
        {
            "@timestamp": "1234567893",
            "@timestamp_pretty": "12-12-2022",
            "event_type": "alert",
            "type_test": "abc",
            "host": "farcry",
            "uptime": 1,
            "port": 1234,
            "bool": False,
            "os": "win10",
            "version": "2.0.0",
            "id": 13
        },
        {
            "index": {
                "_id": 4
            }
        },
        {
            "@timestamp": "1234567894",
            "@timestamp_pretty": "13-12-2022",
            "event_type": "alert",
            "type_test": "abc",
            "host": "GTA",
            "uptime": 3,
            "port": 12,
            "os": "slack",
            "version": "10.0.0",
            "id": 14
        },
        {
            "index": {
                "_id": 5
            }
        },
        {
            "@timestamp": "1234567895",
            "@timestamp_pretty": "17-12-2022",
            "event_type": "alert",
            "host": "sniper 3d",
            "uptime": 6,
            "port": 1234,
            "os": "fedora",
            "version": "20.1.0",
            "id": 15
        },
        {
            "index": {
                "_id": 6
            }
        },
        {
            "@timestamp": "1234568896",
            "@timestamp_pretty": "17-12-2022",
            "event_type": "alert",
            "host": "doom",
            "port": 65123,
            "bool": True,
            "os": "redhat",
            "version": "20.10.0",
            "id": 16
        },
        {
            "index": {
                "_id": 7
            }
        },
        {
            "@timestamp": "1234567897",
            "@timestamp_pretty": "17-12-2022",
            "missing_keyword": "yyy",
            "event_type": "failure",
            "host": "doom",
            "uptime": 15,
            "port": 1234,
            "bool": True,
            "os": "redhat",
            "version": "20.2.0",
            "id": 17
        },
        {
            "index": {
                "_id": 8
            }
        },
        {
            "@timestamp": "1234567898",
            "@timestamp_pretty": "12-12-2022",
            "missing_keyword": "test",
            "event_type": "success",
            "host": "doom",
            "uptime": 16,
            "port": 512,
            "os": "win10",
            "version": "1.2.3",
            "id": 18
        },
        {
            "index": {
                "_id": 9
            }
        },
        {
            "@timestamp": "1234567899",
            "@timestamp_pretty": "15-12-2022",
            "missing_keyword": "test",
            "event_type": "success",
            "host": "GTA",
            "port": 12,
            "bool": True,
            "os": "win10",
            "version": "1.2.3",
            "id": 19
        },
        {
            "index": {
                "_id": 10
            }
        },
        {
            "@timestamp": "1234567893",
            "missing_keyword": None,
            "ip": "10.0.0.5",
            "event_type": "alert",
            "host": "farcry",
            "uptime": 1,
            "port": 1234,
            "bool": True,
            "os": "win10",
            "version": "1.2.3",
            "id": 110
        }
    ],
)
print(resp3)

resp4 = client.bulk(
    index="my-index-000002",
    refresh=True,
    operations=[
        {
            "index": {
                "_id": 1
            }
        },
        {
            "@timestamp": "1234567991",
            "type_test": "abc",
            "ip": "10.0.0.1",
            "event_type": "alert",
            "host": "doom",
            "uptime": 0,
            "port": 1234,
            "op_sys": "win10",
            "id": 21
        },
        {
            "index": {
                "_id": 2
            }
        },
        {
            "@timestamp": "1234567992",
            "type_test": "abc",
            "event_type": "alert",
            "host": "CS",
            "uptime": 5,
            "port": 1,
            "op_sys": "win10",
            "id": 22
        },
        {
            "index": {
                "_id": 3
            }
        },
        {
            "@timestamp": "1234567993",
            "type_test": "abc",
            "@timestamp_pretty": "2022-12-17",
            "event_type": "alert",
            "host": "farcry",
            "uptime": 1,
            "port": 1234,
            "bool": False,
            "op_sys": "win10",
            "id": 23
        },
        {
            "index": {
                "_id": 4
            }
        },
        {
            "@timestamp": "1234567994",
            "event_type": "alert",
            "host": "GTA",
            "uptime": 3,
            "port": 12,
            "op_sys": "slack",
            "id": 24
        },
        {
            "index": {
                "_id": 5
            }
        },
        {
            "@timestamp": "1234567995",
            "event_type": "alert",
            "host": "sniper 3d",
            "uptime": 6,
            "port": 1234,
            "op_sys": "fedora",
            "id": 25
        },
        {
            "index": {
                "_id": 6
            }
        },
        {
            "@timestamp": "1234568996",
            "@timestamp_pretty": "2022-12-17",
            "ip": "10.0.0.5",
            "event_type": "alert",
            "host": "doom",
            "port": 65123,
            "bool": True,
            "op_sys": "redhat",
            "id": 26
        },
        {
            "index": {
                "_id": 7
            }
        },
        {
            "@timestamp": "1234567997",
            "@timestamp_pretty": "2022-12-17",
            "event_type": "failure",
            "host": "doom",
            "uptime": 15,
            "port": 1234,
            "bool": True,
            "op_sys": "redhat",
            "id": 27
        },
        {
            "index": {
                "_id": 8
            }
        },
        {
            "@timestamp": "1234567998",
            "ip": "10.0.0.1",
            "event_type": "success",
            "host": "doom",
            "uptime": 16,
            "port": 512,
            "op_sys": "win10",
            "id": 28
        },
        {
            "index": {
                "_id": 9
            }
        },
        {
            "@timestamp": "1234567999",
            "ip": "10.0.0.1",
            "event_type": "success",
            "host": "GTA",
            "port": 12,
            "bool": False,
            "op_sys": "win10",
            "id": 29
        }
    ],
)
print(resp4)

resp5 = client.bulk(
    index="my-index-000003",
    refresh=True,
    operations=[
        {
            "index": {
                "_id": 1
            }
        },
        {
            "@timestamp": "1334567891",
            "host_ip": "10.0.0.1",
            "event_type": "alert",
            "host": "doom",
            "uptime": 0,
            "port": 12,
            "os": "win10",
            "id": 31
        },
        {
            "index": {
                "_id": 2
            }
        },
        {
            "@timestamp": "1334567892",
            "event_type": "alert",
            "host": "CS",
            "os": "win10",
            "id": 32
        },
        {
            "index": {
                "_id": 3
            }
        },
        {
            "@timestamp": "1334567893",
            "event_type": "alert",
            "host": "farcry",
            "bool": True,
            "os": "win10",
            "id": 33
        },
        {
            "index": {
                "_id": 4
            }
        },
        {
            "@timestamp": "1334567894",
            "event_type": "alert",
            "host": "GTA",
            "os": "slack",
            "bool": True,
            "id": 34
        },
        {
            "index": {
                "_id": 5
            }
        },
        {
            "@timestamp": "1234567895",
            "event_type": "alert",
            "host": "sniper 3d",
            "os": "fedora",
            "id": 35
        },
        {
            "index": {
                "_id": 6
            }
        },
        {
            "@timestamp": "1234578896",
            "host_ip": "10.0.0.1",
            "event_type": "alert",
            "host": "doom",
            "bool": True,
            "os": "redhat",
            "id": 36
        },
        {
            "index": {
                "_id": 7
            }
        },
        {
            "@timestamp": "1234567897",
            "event_type": "failure",
            "missing_keyword": "test",
            "host": "doom",
            "bool": True,
            "os": "redhat",
            "id": 37
        },
        {
            "index": {
                "_id": 8
            }
        },
        {
            "@timestamp": "1234577898",
            "event_type": "success",
            "host": "doom",
            "os": "win10",
            "id": 38,
            "date": "1671235200000"
        },
        {
            "index": {
                "_id": 9
            }
        },
        {
            "@timestamp": "1234577899",
            "host_ip": "10.0.0.5",
            "event_type": "success",
            "host": "GTA",
            "bool": True,
            "os": "win10",
            "id": 39
        }
    ],
)
print(resp5)
response = client.indices.create(
  index: 'my-index-000001',
  body: {
    mappings: {
      properties: {
        ip: {
          type: 'ip'
        },
        version: {
          type: 'version'
        },
        missing_keyword: {
          type: 'keyword'
        },
        "@timestamp": {
          type: 'date'
        },
        type_test: {
          type: 'keyword'
        },
        "@timestamp_pretty": {
          type: 'date',
          format: 'dd-MM-yyyy'
        },
        event_type: {
          type: 'keyword'
        },
        event: {
          properties: {
            category: {
              type: 'alias',
              path: 'event_type'
            }
          }
        },
        host: {
          type: 'keyword'
        },
        os: {
          type: 'keyword'
        },
        bool: {
          type: 'boolean'
        },
        uptime: {
          type: 'long'
        },
        port: {
          type: 'long'
        }
      }
    }
  }
)
puts response

response = client.indices.create(
  index: 'my-index-000002',
  body: {
    mappings: {
      properties: {
        ip: {
          type: 'ip'
        },
        "@timestamp": {
          type: 'date'
        },
        "@timestamp_pretty": {
          type: 'date',
          format: 'yyyy-MM-dd'
        },
        type_test: {
          type: 'keyword'
        },
        event_type: {
          type: 'keyword'
        },
        event: {
          properties: {
            category: {
              type: 'alias',
              path: 'event_type'
            }
          }
        },
        host: {
          type: 'keyword'
        },
        op_sys: {
          type: 'keyword'
        },
        bool: {
          type: 'boolean'
        },
        uptime: {
          type: 'long'
        },
        port: {
          type: 'long'
        }
      }
    }
  }
)
puts response

response = client.indices.create(
  index: 'my-index-000003',
  body: {
    mappings: {
      properties: {
        host_ip: {
          type: 'ip'
        },
        "@timestamp": {
          type: 'date'
        },
        date: {
          type: 'date'
        },
        event_type: {
          type: 'keyword'
        },
        event: {
          properties: {
            category: {
              type: 'alias',
              path: 'event_type'
            }
          }
        },
        missing_keyword: {
          type: 'keyword'
        },
        host: {
          type: 'keyword'
        },
        os: {
          type: 'keyword'
        },
        bool: {
          type: 'boolean'
        },
        uptime: {
          type: 'long'
        },
        port: {
          type: 'long'
        }
      }
    }
  }
)
puts response

response = client.bulk(
  index: 'my-index-000001',
  refresh: true,
  body: [
    {
      index: {
        _id: 1
      }
    },
    {
      "@timestamp": '1234567891',
      "@timestamp_pretty": '12-12-2022',
      missing_keyword: 'test',
      type_test: 'abc',
      ip: '10.0.0.1',
      event_type: 'alert',
      host: 'doom',
      uptime: 0,
      port: 1234,
      os: 'win10',
      version: '1.0.0',
      id: 11
    },
    {
      index: {
        _id: 2
      }
    },
    {
      "@timestamp": '1234567892',
      "@timestamp_pretty": '13-12-2022',
      event_type: 'alert',
      type_test: 'abc',
      host: 'CS',
      uptime: 5,
      port: 1,
      os: 'win10',
      version: '1.2.0',
      id: 12
    },
    {
      index: {
        _id: 3
      }
    },
    {
      "@timestamp": '1234567893',
      "@timestamp_pretty": '12-12-2022',
      event_type: 'alert',
      type_test: 'abc',
      host: 'farcry',
      uptime: 1,
      port: 1234,
      bool: false,
      os: 'win10',
      version: '2.0.0',
      id: 13
    },
    {
      index: {
        _id: 4
      }
    },
    {
      "@timestamp": '1234567894',
      "@timestamp_pretty": '13-12-2022',
      event_type: 'alert',
      type_test: 'abc',
      host: 'GTA',
      uptime: 3,
      port: 12,
      os: 'slack',
      version: '10.0.0',
      id: 14
    },
    {
      index: {
        _id: 5
      }
    },
    {
      "@timestamp": '1234567895',
      "@timestamp_pretty": '17-12-2022',
      event_type: 'alert',
      host: 'sniper 3d',
      uptime: 6,
      port: 1234,
      os: 'fedora',
      version: '20.1.0',
      id: 15
    },
    {
      index: {
        _id: 6
      }
    },
    {
      "@timestamp": '1234568896',
      "@timestamp_pretty": '17-12-2022',
      event_type: 'alert',
      host: 'doom',
      port: 65_123,
      bool: true,
      os: 'redhat',
      version: '20.10.0',
      id: 16
    },
    {
      index: {
        _id: 7
      }
    },
    {
      "@timestamp": '1234567897',
      "@timestamp_pretty": '17-12-2022',
      missing_keyword: 'yyy',
      event_type: 'failure',
      host: 'doom',
      uptime: 15,
      port: 1234,
      bool: true,
      os: 'redhat',
      version: '20.2.0',
      id: 17
    },
    {
      index: {
        _id: 8
      }
    },
    {
      "@timestamp": '1234567898',
      "@timestamp_pretty": '12-12-2022',
      missing_keyword: 'test',
      event_type: 'success',
      host: 'doom',
      uptime: 16,
      port: 512,
      os: 'win10',
      version: '1.2.3',
      id: 18
    },
    {
      index: {
        _id: 9
      }
    },
    {
      "@timestamp": '1234567899',
      "@timestamp_pretty": '15-12-2022',
      missing_keyword: 'test',
      event_type: 'success',
      host: 'GTA',
      port: 12,
      bool: true,
      os: 'win10',
      version: '1.2.3',
      id: 19
    },
    {
      index: {
        _id: 10
      }
    },
    {
      "@timestamp": '1234567893',
      missing_keyword: nil,
      ip: '10.0.0.5',
      event_type: 'alert',
      host: 'farcry',
      uptime: 1,
      port: 1234,
      bool: true,
      os: 'win10',
      version: '1.2.3',
      id: 110
    }
  ]
)
puts response

response = client.bulk(
  index: 'my-index-000002',
  refresh: true,
  body: [
    {
      index: {
        _id: 1
      }
    },
    {
      "@timestamp": '1234567991',
      type_test: 'abc',
      ip: '10.0.0.1',
      event_type: 'alert',
      host: 'doom',
      uptime: 0,
      port: 1234,
      op_sys: 'win10',
      id: 21
    },
    {
      index: {
        _id: 2
      }
    },
    {
      "@timestamp": '1234567992',
      type_test: 'abc',
      event_type: 'alert',
      host: 'CS',
      uptime: 5,
      port: 1,
      op_sys: 'win10',
      id: 22
    },
    {
      index: {
        _id: 3
      }
    },
    {
      "@timestamp": '1234567993',
      type_test: 'abc',
      "@timestamp_pretty": '2022-12-17',
      event_type: 'alert',
      host: 'farcry',
      uptime: 1,
      port: 1234,
      bool: false,
      op_sys: 'win10',
      id: 23
    },
    {
      index: {
        _id: 4
      }
    },
    {
      "@timestamp": '1234567994',
      event_type: 'alert',
      host: 'GTA',
      uptime: 3,
      port: 12,
      op_sys: 'slack',
      id: 24
    },
    {
      index: {
        _id: 5
      }
    },
    {
      "@timestamp": '1234567995',
      event_type: 'alert',
      host: 'sniper 3d',
      uptime: 6,
      port: 1234,
      op_sys: 'fedora',
      id: 25
    },
    {
      index: {
        _id: 6
      }
    },
    {
      "@timestamp": '1234568996',
      "@timestamp_pretty": '2022-12-17',
      ip: '10.0.0.5',
      event_type: 'alert',
      host: 'doom',
      port: 65_123,
      bool: true,
      op_sys: 'redhat',
      id: 26
    },
    {
      index: {
        _id: 7
      }
    },
    {
      "@timestamp": '1234567997',
      "@timestamp_pretty": '2022-12-17',
      event_type: 'failure',
      host: 'doom',
      uptime: 15,
      port: 1234,
      bool: true,
      op_sys: 'redhat',
      id: 27
    },
    {
      index: {
        _id: 8
      }
    },
    {
      "@timestamp": '1234567998',
      ip: '10.0.0.1',
      event_type: 'success',
      host: 'doom',
      uptime: 16,
      port: 512,
      op_sys: 'win10',
      id: 28
    },
    {
      index: {
        _id: 9
      }
    },
    {
      "@timestamp": '1234567999',
      ip: '10.0.0.1',
      event_type: 'success',
      host: 'GTA',
      port: 12,
      bool: false,
      op_sys: 'win10',
      id: 29
    }
  ]
)
puts response

response = client.bulk(
  index: 'my-index-000003',
  refresh: true,
  body: [
    {
      index: {
        _id: 1
      }
    },
    {
      "@timestamp": '1334567891',
      host_ip: '10.0.0.1',
      event_type: 'alert',
      host: 'doom',
      uptime: 0,
      port: 12,
      os: 'win10',
      id: 31
    },
    {
      index: {
        _id: 2
      }
    },
    {
      "@timestamp": '1334567892',
      event_type: 'alert',
      host: 'CS',
      os: 'win10',
      id: 32
    },
    {
      index: {
        _id: 3
      }
    },
    {
      "@timestamp": '1334567893',
      event_type: 'alert',
      host: 'farcry',
      bool: true,
      os: 'win10',
      id: 33
    },
    {
      index: {
        _id: 4
      }
    },
    {
      "@timestamp": '1334567894',
      event_type: 'alert',
      host: 'GTA',
      os: 'slack',
      bool: true,
      id: 34
    },
    {
      index: {
        _id: 5
      }
    },
    {
      "@timestamp": '1234567895',
      event_type: 'alert',
      host: 'sniper 3d',
      os: 'fedora',
      id: 35
    },
    {
      index: {
        _id: 6
      }
    },
    {
      "@timestamp": '1234578896',
      host_ip: '10.0.0.1',
      event_type: 'alert',
      host: 'doom',
      bool: true,
      os: 'redhat',
      id: 36
    },
    {
      index: {
        _id: 7
      }
    },
    {
      "@timestamp": '1234567897',
      event_type: 'failure',
      missing_keyword: 'test',
      host: 'doom',
      bool: true,
      os: 'redhat',
      id: 37
    },
    {
      index: {
        _id: 8
      }
    },
    {
      "@timestamp": '1234577898',
      event_type: 'success',
      host: 'doom',
      os: 'win10',
      id: 38,
      date: '1671235200000'
    },
    {
      index: {
        _id: 9
      }
    },
    {
      "@timestamp": '1234577899',
      host_ip: '10.0.0.5',
      event_type: 'success',
      host: 'GTA',
      bool: true,
      os: 'win10',
      id: 39
    }
  ]
)
puts response
const response = await client.indices.create({
  index: "my-index-000001",
  mappings: {
    properties: {
      ip: {
        type: "ip",
      },
      version: {
        type: "version",
      },
      missing_keyword: {
        type: "keyword",
      },
      "@timestamp": {
        type: "date",
      },
      type_test: {
        type: "keyword",
      },
      "@timestamp_pretty": {
        type: "date",
        format: "dd-MM-yyyy",
      },
      event_type: {
        type: "keyword",
      },
      event: {
        properties: {
          category: {
            type: "alias",
            path: "event_type",
          },
        },
      },
      host: {
        type: "keyword",
      },
      os: {
        type: "keyword",
      },
      bool: {
        type: "boolean",
      },
      uptime: {
        type: "long",
      },
      port: {
        type: "long",
      },
    },
  },
});
console.log(response);

const response1 = await client.indices.create({
  index: "my-index-000002",
  mappings: {
    properties: {
      ip: {
        type: "ip",
      },
      "@timestamp": {
        type: "date",
      },
      "@timestamp_pretty": {
        type: "date",
        format: "yyyy-MM-dd",
      },
      type_test: {
        type: "keyword",
      },
      event_type: {
        type: "keyword",
      },
      event: {
        properties: {
          category: {
            type: "alias",
            path: "event_type",
          },
        },
      },
      host: {
        type: "keyword",
      },
      op_sys: {
        type: "keyword",
      },
      bool: {
        type: "boolean",
      },
      uptime: {
        type: "long",
      },
      port: {
        type: "long",
      },
    },
  },
});
console.log(response1);

const response2 = await client.indices.create({
  index: "my-index-000003",
  mappings: {
    properties: {
      host_ip: {
        type: "ip",
      },
      "@timestamp": {
        type: "date",
      },
      date: {
        type: "date",
      },
      event_type: {
        type: "keyword",
      },
      event: {
        properties: {
          category: {
            type: "alias",
            path: "event_type",
          },
        },
      },
      missing_keyword: {
        type: "keyword",
      },
      host: {
        type: "keyword",
      },
      os: {
        type: "keyword",
      },
      bool: {
        type: "boolean",
      },
      uptime: {
        type: "long",
      },
      port: {
        type: "long",
      },
    },
  },
});
console.log(response2);

const response3 = await client.bulk({
  index: "my-index-000001",
  refresh: "true",
  operations: [
    {
      index: {
        _id: 1,
      },
    },
    {
      "@timestamp": "1234567891",
      "@timestamp_pretty": "12-12-2022",
      missing_keyword: "test",
      type_test: "abc",
      ip: "10.0.0.1",
      event_type: "alert",
      host: "doom",
      uptime: 0,
      port: 1234,
      os: "win10",
      version: "1.0.0",
      id: 11,
    },
    {
      index: {
        _id: 2,
      },
    },
    {
      "@timestamp": "1234567892",
      "@timestamp_pretty": "13-12-2022",
      event_type: "alert",
      type_test: "abc",
      host: "CS",
      uptime: 5,
      port: 1,
      os: "win10",
      version: "1.2.0",
      id: 12,
    },
    {
      index: {
        _id: 3,
      },
    },
    {
      "@timestamp": "1234567893",
      "@timestamp_pretty": "12-12-2022",
      event_type: "alert",
      type_test: "abc",
      host: "farcry",
      uptime: 1,
      port: 1234,
      bool: false,
      os: "win10",
      version: "2.0.0",
      id: 13,
    },
    {
      index: {
        _id: 4,
      },
    },
    {
      "@timestamp": "1234567894",
      "@timestamp_pretty": "13-12-2022",
      event_type: "alert",
      type_test: "abc",
      host: "GTA",
      uptime: 3,
      port: 12,
      os: "slack",
      version: "10.0.0",
      id: 14,
    },
    {
      index: {
        _id: 5,
      },
    },
    {
      "@timestamp": "1234567895",
      "@timestamp_pretty": "17-12-2022",
      event_type: "alert",
      host: "sniper 3d",
      uptime: 6,
      port: 1234,
      os: "fedora",
      version: "20.1.0",
      id: 15,
    },
    {
      index: {
        _id: 6,
      },
    },
    {
      "@timestamp": "1234568896",
      "@timestamp_pretty": "17-12-2022",
      event_type: "alert",
      host: "doom",
      port: 65123,
      bool: true,
      os: "redhat",
      version: "20.10.0",
      id: 16,
    },
    {
      index: {
        _id: 7,
      },
    },
    {
      "@timestamp": "1234567897",
      "@timestamp_pretty": "17-12-2022",
      missing_keyword: "yyy",
      event_type: "failure",
      host: "doom",
      uptime: 15,
      port: 1234,
      bool: true,
      os: "redhat",
      version: "20.2.0",
      id: 17,
    },
    {
      index: {
        _id: 8,
      },
    },
    {
      "@timestamp": "1234567898",
      "@timestamp_pretty": "12-12-2022",
      missing_keyword: "test",
      event_type: "success",
      host: "doom",
      uptime: 16,
      port: 512,
      os: "win10",
      version: "1.2.3",
      id: 18,
    },
    {
      index: {
        _id: 9,
      },
    },
    {
      "@timestamp": "1234567899",
      "@timestamp_pretty": "15-12-2022",
      missing_keyword: "test",
      event_type: "success",
      host: "GTA",
      port: 12,
      bool: true,
      os: "win10",
      version: "1.2.3",
      id: 19,
    },
    {
      index: {
        _id: 10,
      },
    },
    {
      "@timestamp": "1234567893",
      missing_keyword: null,
      ip: "10.0.0.5",
      event_type: "alert",
      host: "farcry",
      uptime: 1,
      port: 1234,
      bool: true,
      os: "win10",
      version: "1.2.3",
      id: 110,
    },
  ],
});
console.log(response3);

const response4 = await client.bulk({
  index: "my-index-000002",
  refresh: "true",
  operations: [
    {
      index: {
        _id: 1,
      },
    },
    {
      "@timestamp": "1234567991",
      type_test: "abc",
      ip: "10.0.0.1",
      event_type: "alert",
      host: "doom",
      uptime: 0,
      port: 1234,
      op_sys: "win10",
      id: 21,
    },
    {
      index: {
        _id: 2,
      },
    },
    {
      "@timestamp": "1234567992",
      type_test: "abc",
      event_type: "alert",
      host: "CS",
      uptime: 5,
      port: 1,
      op_sys: "win10",
      id: 22,
    },
    {
      index: {
        _id: 3,
      },
    },
    {
      "@timestamp": "1234567993",
      type_test: "abc",
      "@timestamp_pretty": "2022-12-17",
      event_type: "alert",
      host: "farcry",
      uptime: 1,
      port: 1234,
      bool: false,
      op_sys: "win10",
      id: 23,
    },
    {
      index: {
        _id: 4,
      },
    },
    {
      "@timestamp": "1234567994",
      event_type: "alert",
      host: "GTA",
      uptime: 3,
      port: 12,
      op_sys: "slack",
      id: 24,
    },
    {
      index: {
        _id: 5,
      },
    },
    {
      "@timestamp": "1234567995",
      event_type: "alert",
      host: "sniper 3d",
      uptime: 6,
      port: 1234,
      op_sys: "fedora",
      id: 25,
    },
    {
      index: {
        _id: 6,
      },
    },
    {
      "@timestamp": "1234568996",
      "@timestamp_pretty": "2022-12-17",
      ip: "10.0.0.5",
      event_type: "alert",
      host: "doom",
      port: 65123,
      bool: true,
      op_sys: "redhat",
      id: 26,
    },
    {
      index: {
        _id: 7,
      },
    },
    {
      "@timestamp": "1234567997",
      "@timestamp_pretty": "2022-12-17",
      event_type: "failure",
      host: "doom",
      uptime: 15,
      port: 1234,
      bool: true,
      op_sys: "redhat",
      id: 27,
    },
    {
      index: {
        _id: 8,
      },
    },
    {
      "@timestamp": "1234567998",
      ip: "10.0.0.1",
      event_type: "success",
      host: "doom",
      uptime: 16,
      port: 512,
      op_sys: "win10",
      id: 28,
    },
    {
      index: {
        _id: 9,
      },
    },
    {
      "@timestamp": "1234567999",
      ip: "10.0.0.1",
      event_type: "success",
      host: "GTA",
      port: 12,
      bool: false,
      op_sys: "win10",
      id: 29,
    },
  ],
});
console.log(response4);

const response5 = await client.bulk({
  index: "my-index-000003",
  refresh: "true",
  operations: [
    {
      index: {
        _id: 1,
      },
    },
    {
      "@timestamp": "1334567891",
      host_ip: "10.0.0.1",
      event_type: "alert",
      host: "doom",
      uptime: 0,
      port: 12,
      os: "win10",
      id: 31,
    },
    {
      index: {
        _id: 2,
      },
    },
    {
      "@timestamp": "1334567892",
      event_type: "alert",
      host: "CS",
      os: "win10",
      id: 32,
    },
    {
      index: {
        _id: 3,
      },
    },
    {
      "@timestamp": "1334567893",
      event_type: "alert",
      host: "farcry",
      bool: true,
      os: "win10",
      id: 33,
    },
    {
      index: {
        _id: 4,
      },
    },
    {
      "@timestamp": "1334567894",
      event_type: "alert",
      host: "GTA",
      os: "slack",
      bool: true,
      id: 34,
    },
    {
      index: {
        _id: 5,
      },
    },
    {
      "@timestamp": "1234567895",
      event_type: "alert",
      host: "sniper 3d",
      os: "fedora",
      id: 35,
    },
    {
      index: {
        _id: 6,
      },
    },
    {
      "@timestamp": "1234578896",
      host_ip: "10.0.0.1",
      event_type: "alert",
      host: "doom",
      bool: true,
      os: "redhat",
      id: 36,
    },
    {
      index: {
        _id: 7,
      },
    },
    {
      "@timestamp": "1234567897",
      event_type: "failure",
      missing_keyword: "test",
      host: "doom",
      bool: true,
      os: "redhat",
      id: 37,
    },
    {
      index: {
        _id: 8,
      },
    },
    {
      "@timestamp": "1234577898",
      event_type: "success",
      host: "doom",
      os: "win10",
      id: 38,
      date: "1671235200000",
    },
    {
      index: {
        _id: 9,
      },
    },
    {
      "@timestamp": "1234577899",
      host_ip: "10.0.0.5",
      event_type: "success",
      host: "GTA",
      bool: true,
      os: "win10",
      id: 39,
    },
  ],
});
console.log(response5);
PUT /my-index-000001
{
    "mappings": {
        "properties": {
            "ip": {
                "type":"ip"
            },
            "version": {
                "type": "version"
            },
            "missing_keyword": {
                "type": "keyword"
            },
            "@timestamp": {
              "type": "date"
            },
            "type_test": {
                "type": "keyword"
            },
            "@timestamp_pretty": {
              "type": "date",
              "format": "dd-MM-yyyy"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "host": {
              "type": "keyword"
            },
            "os": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

PUT /my-index-000002
{
    "mappings": {
        "properties": {
            "ip": {
                "type":"ip"
            },
            "@timestamp": {
              "type": "date"
            },
            "@timestamp_pretty": {
              "type": "date",
              "format": "yyyy-MM-dd"
            },
            "type_test": {
                "type": "keyword"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "host": {
              "type": "keyword"
            },
            "op_sys": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

PUT /my-index-000003
{
    "mappings": {
        "properties": {
            "host_ip": {
                "type":"ip"
            },
            "@timestamp": {
              "type": "date"
            },
            "date": {
              "type": "date"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "missing_keyword": {
                "type": "keyword"
            },
            "host": {
              "type": "keyword"
            },
            "os": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

POST /my-index-000001/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1234567891","@timestamp_pretty":"12-12-2022","missing_keyword":"test","type_test":"abc","ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":1234,"os":"win10","version":"1.0.0","id":11}
{"index":{"_id":2}}
{"@timestamp":"1234567892","@timestamp_pretty":"13-12-2022","event_type":"alert","type_test":"abc","host":"CS","uptime":5,"port":1,"os":"win10","version":"1.2.0","id":12}
{"index":{"_id":3}}
{"@timestamp":"1234567893","@timestamp_pretty":"12-12-2022","event_type":"alert","type_test":"abc","host":"farcry","uptime":1,"port":1234,"bool":false,"os":"win10","version":"2.0.0","id":13}
{"index":{"_id":4}}
{"@timestamp":"1234567894","@timestamp_pretty":"13-12-2022","event_type":"alert","type_test":"abc","host":"GTA","uptime":3,"port":12,"os":"slack","version":"10.0.0","id":14}
{"index":{"_id":5}}
{"@timestamp":"1234567895","@timestamp_pretty":"17-12-2022","event_type":"alert","host":"sniper 3d","uptime":6,"port":1234,"os":"fedora","version":"20.1.0","id":15}
{"index":{"_id":6}}
{"@timestamp":"1234568896","@timestamp_pretty":"17-12-2022","event_type":"alert","host":"doom","port":65123,"bool":true,"os":"redhat","version":"20.10.0","id":16}
{"index":{"_id":7}}
{"@timestamp":"1234567897","@timestamp_pretty":"17-12-2022","missing_keyword":"yyy","event_type":"failure","host":"doom","uptime":15,"port":1234,"bool":true,"os":"redhat","version":"20.2.0","id":17}
{"index":{"_id":8}}
{"@timestamp":"1234567898","@timestamp_pretty":"12-12-2022","missing_keyword":"test","event_type":"success","host":"doom","uptime":16,"port":512,"os":"win10","version":"1.2.3","id":18}
{"index":{"_id":9}}
{"@timestamp":"1234567899","@timestamp_pretty":"15-12-2022","missing_keyword":"test","event_type":"success","host":"GTA","port":12,"bool":true,"os":"win10","version":"1.2.3","id":19}
{"index":{"_id":10}}
{"@timestamp":"1234567893","missing_keyword":null,"ip":"10.0.0.5","event_type":"alert","host":"farcry","uptime":1,"port":1234,"bool":true,"os":"win10","version":"1.2.3","id":110}

POST /my-index-000002/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1234567991","type_test":"abc","ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":1234,"op_sys":"win10","id":21}
{"index":{"_id":2}}
{"@timestamp":"1234567992","type_test":"abc","event_type":"alert","host":"CS","uptime":5,"port":1,"op_sys":"win10","id":22}
{"index":{"_id":3}}
{"@timestamp":"1234567993","type_test":"abc","@timestamp_pretty":"2022-12-17","event_type":"alert","host":"farcry","uptime":1,"port":1234,"bool":false,"op_sys":"win10","id":23}
{"index":{"_id":4}}
{"@timestamp":"1234567994","event_type":"alert","host":"GTA","uptime":3,"port":12,"op_sys":"slack","id":24}
{"index":{"_id":5}}
{"@timestamp":"1234567995","event_type":"alert","host":"sniper 3d","uptime":6,"port":1234,"op_sys":"fedora","id":25}
{"index":{"_id":6}}
{"@timestamp":"1234568996","@timestamp_pretty":"2022-12-17","ip":"10.0.0.5","event_type":"alert","host":"doom","port":65123,"bool":true,"op_sys":"redhat","id":26}
{"index":{"_id":7}}
{"@timestamp":"1234567997","@timestamp_pretty":"2022-12-17","event_type":"failure","host":"doom","uptime":15,"port":1234,"bool":true,"op_sys":"redhat","id":27}
{"index":{"_id":8}}
{"@timestamp":"1234567998","ip":"10.0.0.1","event_type":"success","host":"doom","uptime":16,"port":512,"op_sys":"win10","id":28}
{"index":{"_id":9}}
{"@timestamp":"1234567999","ip":"10.0.0.1","event_type":"success","host":"GTA","port":12,"bool":false,"op_sys":"win10","id":29}

POST /my-index-000003/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1334567891","host_ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":12,"os":"win10","id":31}
{"index":{"_id":2}}
{"@timestamp":"1334567892","event_type":"alert","host":"CS","os":"win10","id":32}
{"index":{"_id":3}}
{"@timestamp":"1334567893","event_type":"alert","host":"farcry","bool":true,"os":"win10","id":33}
{"index":{"_id":4}}
{"@timestamp":"1334567894","event_type":"alert","host":"GTA","os":"slack","bool":true,"id":34}
{"index":{"_id":5}}
{"@timestamp":"1234567895","event_type":"alert","host":"sniper 3d","os":"fedora","id":35}
{"index":{"_id":6}}
{"@timestamp":"1234578896","host_ip":"10.0.0.1","event_type":"alert","host":"doom","bool":true,"os":"redhat","id":36}
{"index":{"_id":7}}
{"@timestamp":"1234567897","event_type":"failure","missing_keyword":"test","host":"doom","bool":true,"os":"redhat","id":37}
{"index":{"_id":8}}
{"@timestamp":"1234577898","event_type":"success","host":"doom","os":"win10","id":38,"date":"1671235200000"}
{"index":{"_id":9}}
{"@timestamp":"1234577899","host_ip":"10.0.0.5","event_type":"success","host":"GTA","bool":true,"os":"win10","id":39}

样本查询使用 by 关键字 指定至少一个连接键，以及最多五个过滤器。

resp = client.eql.search(
    index="my-index*",
    query="\n    sample by host\n      [any where uptime > 0]\n      [any where port > 100]\n      [any where bool == true]\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-index*",
  query:
    "\n    sample by host\n      [any where uptime > 0]\n      [any where port > 100]\n      [any where bool == true]\n  ",
});
console.log(response);
GET /my-index*/_eql/search
{
  "query": """
    sample by host
      [any where uptime > 0]
      [any where port > 100]
      [any where bool == true]
  """
}

默认情况下，响应的 hits.sequences 属性包含最多 10 个样本。每个样本都有一组 join_keys 和一个数组，其中包含每个过滤器的每个匹配事件。事件按其匹配的过滤器顺序返回。

{
  ...
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          "doom"                                      
        ],
        "events": [
          {                                           
            "_index": "my-index-000001",
            "_id": "7",
            "_source": {
              "@timestamp": "1234567897",
              "@timestamp_pretty": "17-12-2022",
              "missing_keyword": "yyy",
              "event_type": "failure",
              "host": "doom",
              "uptime": 15,
              "port": 1234,
              "bool": true,
              "os": "redhat",
              "version": "20.2.0",
              "id": 17
            }
          },
          {                                           
            "_index": "my-index-000001",
            "_id": "1",
            "_source": {
              "@timestamp": "1234567891",
              "@timestamp_pretty": "12-12-2022",
              "missing_keyword": "test",
              "type_test": "abc",
              "ip": "10.0.0.1",
              "event_type": "alert",
              "host": "doom",
              "uptime": 0,
              "port": 1234,
              "os": "win10",
              "version": "1.0.0",
              "id": 11
            }
          },
          {                                           
            "_index": "my-index-000001",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568896",
              "@timestamp_pretty": "17-12-2022",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "os": "redhat",
              "version": "20.10.0",
              "id": 16
            }
          }
        ]
      },
      {
        "join_keys": [
          "farcry"                                    
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567893",
              "@timestamp_pretty": "12-12-2022",
              "event_type": "alert",
              "type_test": "abc",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "os": "win10",
              "version": "2.0.0",
              "id": 13
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "10",
            "_source": {
              "@timestamp": "1234567893",
              "missing_keyword": null,
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": true,
              "os": "win10",
              "version": "1.2.3",
              "id": 110
            }
          },
          {
            "_index": "my-index-000003",
            "_id": "3",
            "_source": {
              "@timestamp": "1334567893",
              "event_type": "alert",
              "host": "farcry",
              "bool": true,
              "os": "win10",
              "id": 33
            }
          }
        ]
      }
    ]
  }
}

第一个样本中的事件的 host 值为 doom。

此事件与第一个过滤器匹配。

此事件与第二个过滤器匹配。

此事件与第三个过滤器匹配。

第二个样本中的事件的 host 值为 farcry。

您可以指定多个连接键。

resp = client.eql.search(
    index="my-index*",
    query="\n    sample by host\n      [any where uptime > 0]   by os\n      [any where port > 100]   by op_sys\n      [any where bool == true] by os\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-index*",
  query:
    "\n    sample by host\n      [any where uptime > 0]   by os\n      [any where port > 100]   by op_sys\n      [any where bool == true] by os\n  ",
});
console.log(response);
GET /my-index*/_eql/search
{
  "query": """
    sample by host
      [any where uptime > 0]   by os
      [any where port > 100]   by op_sys
      [any where bool == true] by os
  """
}

此查询将返回每个事件共享 os 或 op_sys 的相同值以及 host 的相同值的样本。例如

{
  ...
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          "doom",                                      
          "redhat"
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "7",
            "_source": {
              "@timestamp": "1234567897",
              "@timestamp_pretty": "17-12-2022",
              "missing_keyword": "yyy",
              "event_type": "failure",
              "host": "doom",
              "uptime": 15,
              "port": 1234,
              "bool": true,
              "os": "redhat",
              "version": "20.2.0",
              "id": 17
            }
          },
          {
            "_index": "my-index-000002",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568996",
              "@timestamp_pretty": "2022-12-17",
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "op_sys": "redhat",
              "id": 26
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568896",
              "@timestamp_pretty": "17-12-2022",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "os": "redhat",
              "version": "20.10.0",
              "id": 16
            }
          }
        ]
      },
      {
        "join_keys": [
          "farcry",
          "win10"
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567893",
              "@timestamp_pretty": "12-12-2022",
              "event_type": "alert",
              "type_test": "abc",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "os": "win10",
              "version": "2.0.0",
              "id": 13
            }
          },
          {
            "_index": "my-index-000002",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567993",
              "type_test": "abc",
              "@timestamp_pretty": "2022-12-17",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "op_sys": "win10",
              "id": 23
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "10",
            "_source": {
              "@timestamp": "1234567893",
              "missing_keyword": null,
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": true,
              "os": "win10",
              "version": "1.2.3",
              "id": 110
            }
          }
        ]
      }
    ]
  }
}

此样本中的事件的 host 值为 doom，os 或 op_sys 值为 redhat。

默认情况下，样本查询的响应包含最多 10 个样本，每个唯一连接键集一个样本。使用 size 参数可以获取较小或较大的样本集。要为每个连接键集检索多个样本，请使用 max_samples_per_key 参数。样本查询不支持管道。

resp = client.eql.search(
    index="my-index*",
    max_samples_per_key=2,
    size=20,
    query="\n    sample\n      [any where uptime > 0]   by host,os\n      [any where port > 100]   by host,op_sys\n      [any where bool == true] by host,os\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-index*",
  max_samples_per_key: 2,
  size: 20,
  query:
    "\n    sample\n      [any where uptime > 0]   by host,os\n      [any where port > 100]   by host,op_sys\n      [any where bool == true] by host,os\n  ",
});
console.log(response);
GET /my-index*/_eql/search
{
  "max_samples_per_key": 2,     
  "size": 20,                   
  "query": """
    sample
      [any where uptime > 0]   by host,os
      [any where port > 100]   by host,op_sys
      [any where bool == true] by host,os
  """
}

为每个连接键集检索最多 2 个样本。

总共检索最多 20 个样本。

检索选定字段

编辑

默认情况下，搜索响应中的每个命中结果都包含文档 _source，它是索引文档时提供的整个 JSON 对象。

您可以使用 filter_path 查询参数来过滤 API 响应。例如，以下搜索仅返回每个匹配事件的 _source 中的时间戳和 PID。

resp = client.eql.search(
    index="my-data-stream",
    filter_path="hits.events._source.@timestamp,hits.events._source.process.pid",
    query="\n    process where process.name == \"regsvr32.exe\"\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  filter_path: "hits.events._source.@timestamp,hits.events._source.process.pid",
  query: '\n    process where process.name == "regsvr32.exe"\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search?filter_path=hits.events._source.@timestamp,hits.events._source.process.pid
{
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

API 返回以下响应。

{
  "hits": {
    "events": [
      {
        "_source": {
          "@timestamp": "2099-12-07T11:07:09.000Z",
          "process": {
            "pid": 2012
          }
        }
      },
      {
        "_source": {
          "@timestamp": "2099-12-07T11:07:10.000Z",
          "process": {
            "pid": 2012
          }
        }
      }
    ]
  }
}

您还可以使用 fields 参数来检索和格式化响应中的特定字段。此字段与搜索 API 的 fields 参数 相同。

因为它会查询索引映射，所以 fields 参数比直接引用 _source 提供了几个优势。具体来说，fields 参数

以下搜索请求使用 fields 参数来检索 event.type 字段的值、所有以 process. 开头的字段以及 @timestamp 字段。该请求还使用 filter_path 查询参数来排除每个命中的 _source。

resp = client.eql.search(
    index="my-data-stream",
    filter_path="-hits.events._source",
    query="\n    process where process.name == \"regsvr32.exe\"\n  ",
    fields=[
        "event.type",
        "process.*",
        {
            "field": "@timestamp",
            "format": "epoch_millis"
        }
    ],
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  filter_path: "-hits.events._source",
  query: '\n    process where process.name == "regsvr32.exe"\n  ',
  fields: [
    "event.type",
    "process.*",
    {
      field: "@timestamp",
      format: "epoch_millis",
    },
  ],
});
console.log(response);
GET /my-data-stream/_eql/search?filter_path=-hits.events._source
{
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "fields": [
    "event.type",
    "process.*",                
    {
      "field": "@timestamp",
      "format": "epoch_millis"  
    }
  ]
}

接受完整的字段名称和通配符模式。

使用 format 参数可以为字段的值应用自定义格式。

响应在每个命中的 fields 部分中包含值作为平面列表。

{
  ...
  "hits": {
    "total": ...,
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "fields": {
          "process.name": [
            "regsvr32.exe"
          ],
          "process.name.keyword": [
            "regsvr32.exe"
          ],
          "@timestamp": [
            "4100324829000"
          ],
          "process.command_line": [
            "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll"
          ],
          "process.command_line.keyword": [
            "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll"
          ],
          "process.executable.keyword": [
            "C:\\Windows\\System32\\regsvr32.exe"
          ],
          "process.pid": [
            2012
          ],
          "process.executable": [
            "C:\\Windows\\System32\\regsvr32.exe"
          ]
        }
      },
      ....
    ]
  }
}

使用运行时字段

编辑

使用 runtime_mappings 参数在搜索期间提取和创建 运行时字段。使用 fields 参数将运行时字段包含在响应中。

以下搜索从 @timestamp 创建一个 day_of_week 运行时字段并在响应中返回它。

resp = client.eql.search(
    index="my-data-stream",
    filter_path="-hits.events._source",
    runtime_mappings={
        "day_of_week": {
            "type": "keyword",
            "script": "emit(doc['@timestamp'].value.dayOfWeekEnum.toString())"
        }
    },
    query="\n    process where process.name == \"regsvr32.exe\"\n  ",
    fields=[
        "@timestamp",
        "day_of_week"
    ],
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  filter_path: "-hits.events._source",
  runtime_mappings: {
    day_of_week: {
      type: "keyword",
      script: "emit(doc['@timestamp'].value.dayOfWeekEnum.toString())",
    },
  },
  query: '\n    process where process.name == "regsvr32.exe"\n  ',
  fields: ["@timestamp", "day_of_week"],
});
console.log(response);
GET /my-data-stream/_eql/search?filter_path=-hits.events._source
{
  "runtime_mappings": {
    "day_of_week": {
      "type": "keyword",
      "script": "emit(doc['@timestamp'].value.dayOfWeekEnum.toString())"
    }
  },
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "fields": [
    "@timestamp",
    "day_of_week"
  ]
}

API 返回

{
  ...
  "hits": {
    "total": ...,
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "fields": {
          "@timestamp": [
            "2099-12-07T11:07:09.000Z"
          ],
          "day_of_week": [
            "MONDAY"
          ]
        }
      },
      ....
    ]
  }
}

指定时间戳或事件类别字段

编辑

EQL 搜索 API 默认情况下使用来自 ECS 的 @timestamp 和 event.category 字段。要指定不同的字段，请使用 timestamp_field 和 event_category_field 参数。

resp = client.eql.search(
    index="my-data-stream",
    timestamp_field="file.accessed",
    event_category_field="file.type",
    query="\n    file where (file.size > 1 and file.type == \"file\")\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  timestamp_field: "file.accessed",
  event_category_field: "file.type",
  query: '\n    file where (file.size > 1 and file.type == "file")\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "timestamp_field": "file.accessed",
  "event_category_field": "file.type",
  "query": """
    file where (file.size > 1 and file.type == "file")
  """
}

事件类别字段必须映射为 keyword 类型族字段类型。时间戳字段应映射为 date 字段类型。date_nanos 时间戳字段不受支持。您不能使用 nested 字段或 nested 字段的子字段作为时间戳或事件类别字段。

指定排序决胜符

编辑

默认情况下，EQL 搜索 API 按时间戳返回匹配的命中结果。如果两个或多个事件共享相同的时间戳，Elasticsearch 将使用决胜符字段值按升序对事件进行排序。Elasticsearch 将没有决胜符值的事件排序在有值的事件之后。

如果您没有指定决胜符字段或事件也共享相同的决胜符值，Elasticsearch 将认为这些事件是并发的，并且可能不会以一致的排序顺序返回它们。

要指定决胜符字段，请使用 tiebreaker_field 参数。如果您使用 ECS，我们建议使用 event.sequence 作为决胜符字段。

resp = client.eql.search(
    index="my-data-stream",
    tiebreaker_field="event.sequence",
    query="\n    process where process.name == \"cmd.exe\" and stringContains(process.executable, \"System32\")\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  tiebreaker_field: "event.sequence",
  query:
    '\n    process where process.name == "cmd.exe" and stringContains(process.executable, "System32")\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "tiebreaker_field": "event.sequence",
  "query": """
    process where process.name == "cmd.exe" and stringContains(process.executable, "System32")
  """
}

使用 Query DSL 过滤

编辑

filter 参数使用 Query DSL 来限制运行 EQL 查询的文档。

resp = client.eql.search(
    index="my-data-stream",
    filter={
        "range": {
            "@timestamp": {
                "gte": "now-1d/d",
                "lt": "now/d"
            }
        }
    },
    query="\n    file where (file.type == \"file\" and file.name == \"cmd.exe\")\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  filter: {
    range: {
      "@timestamp": {
        gte: "now-1d/d",
        lt: "now/d",
      },
    },
  },
  query:
    '\n    file where (file.type == "file" and file.name == "cmd.exe")\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "filter": {
    "range": {
      "@timestamp": {
        "gte": "now-1d/d",
        "lt": "now/d"
      }
    }
  },
  "query": """
    file where (file.type == "file" and file.name == "cmd.exe")
  """
}

运行异步 EQL 搜索

编辑

默认情况下，EQL 搜索请求是同步的，会在返回响应之前等待完整的搜索结果。但是，对于跨大型数据集或冻结数据的搜索，完整的搜索结果可能需要更长时间。

为避免长时间等待，请运行异步 EQL 搜索。将wait_for_completion_timeout设置为要等待同步结果的持续时间。

resp = client.eql.search(
    index="my-data-stream",
    wait_for_completion_timeout="2s",
    query="\n    process where process.name == \"cmd.exe\"\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  wait_for_completion_timeout: "2s",
  query: '\n    process where process.name == "cmd.exe"\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

如果请求在超时时间内未完成，则搜索变为异步，并返回包含以下内容的响应：

  • 搜索 ID
  • is_partial 值为 true，指示搜索结果不完整
  • is_running 值为 true，指示搜索正在进行中

异步搜索将在后台继续运行，而不会阻塞其他请求。

{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_partial": true,
  "is_running": true,
  "took": 2000,
  "timed_out": false,
  "hits": ...
}

要检查异步搜索的进度，请使用获取异步 EQL 搜索 API和搜索 ID。在wait_for_completion_timeout参数中指定您希望获得完整结果的时长。

resp = client.eql.get(
    id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
    wait_for_completion_timeout="2s",
)
print(resp)
response = client.eql.get(
  id: 'FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=',
  wait_for_completion_timeout: '2s'
)
puts response
const response = await client.eql.get({
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  wait_for_completion_timeout: "2s",
});
console.log(response);
GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?wait_for_completion_timeout=2s

如果响应的is_running值为false，则异步搜索已完成。如果is_partial值为false，则返回的搜索结果完整。

{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_partial": false,
  "is_running": false,
  "took": 2000,
  "timed_out": false,
  "hits": ...
}

另一种更轻量级的方法来检查异步搜索的进度是使用获取异步 EQL 状态 API和搜索 ID。

resp = client.eql.get_status(
    id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
)
print(resp)
response = client.eql.get_status(
  id: 'FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE='
)
puts response
const response = await client.eql.getStatus({
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
});
console.log(response);
GET /_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=
{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_running": false,
  "is_partial": false,
  "expiration_time_in_millis": 1611690295000,
  "completion_status": 200
}

更改搜索保留期

编辑

默认情况下，EQL 搜索 API 会存储异步搜索五天。此期限过后，任何搜索及其结果都将被删除。使用keep_alive参数更改此保留期。

resp = client.eql.search(
    index="my-data-stream",
    keep_alive="2d",
    wait_for_completion_timeout="2s",
    query="\n    process where process.name == \"cmd.exe\"\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  keep_alive: "2d",
  wait_for_completion_timeout: "2s",
  query: '\n    process where process.name == "cmd.exe"\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "keep_alive": "2d",
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

您可以使用获取异步 EQL 搜索 API的keep_alive参数稍后更改保留期。新的保留期从获取请求运行后开始。

resp = client.eql.get(
    id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
    keep_alive="5d",
)
print(resp)
response = client.eql.get(
  id: 'FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=',
  keep_alive: '5d'
)
puts response
const response = await client.eql.get({
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  keep_alive: "5d",
});
console.log(response);
GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?keep_alive=5d

使用删除异步 EQL 搜索 API在keep_alive期限结束之前手动删除异步 EQL 搜索。如果搜索仍在进行中，Elasticsearch 将取消搜索请求。

resp = client.eql.delete(
    id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
)
print(resp)
response = client.eql.delete(
  id: 'FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE='
)
puts response
const response = await client.eql.delete({
  id: "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
});
console.log(response);
DELETE /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=

存储同步 EQL 搜索

编辑

默认情况下，EQL 搜索 API 仅存储异步搜索。要保存同步搜索，请将keep_on_completion设置为true。

resp = client.eql.search(
    index="my-data-stream",
    keep_on_completion=True,
    wait_for_completion_timeout="2s",
    query="\n    process where process.name == \"cmd.exe\"\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "my-data-stream",
  keep_on_completion: true,
  wait_for_completion_timeout: "2s",
  query: '\n    process where process.name == "cmd.exe"\n  ',
});
console.log(response);
GET /my-data-stream/_eql/search
{
  "keep_on_completion": true,
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

响应包含搜索 ID。is_partial和is_running为false，指示 EQL 搜索是同步的并返回了完整的搜索结果。

{
  "id": "FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY=",
  "is_partial": false,
  "is_running": false,
  "took": 52,
  "timed_out": false,
  "hits": ...
}

稍后使用获取异步 EQL 搜索 API获取相同的结果。

resp = client.eql.get(
    id="FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY=",
)
print(resp)
response = client.eql.get(
  id: 'FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY='
)
puts response
const response = await client.eql.get({
  id: "FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY=",
});
console.log(response);
GET /_eql/search/FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY=

已保存的同步搜索仍受keep_alive参数的保留期限制。当此期限结束时，搜索及其结果将被删除。

您也可以仅使用获取异步 EQL 状态 API检查已保存同步搜索的状态（无结果）。

您也可以使用删除异步 EQL 搜索 API手动删除已保存的同步搜索。

跨集群运行 EQL 搜索

编辑

此功能处于技术预览阶段，可能会在将来的版本中更改或删除。Elastic 将致力于修复任何问题，但技术预览中的功能不受官方 GA 功能的支持 SLA 的约束。

EQL 搜索 API 支持跨集群搜索。但是，如果本地和远程集群使用的是 7.17.7（含）之前的版本或 8.5.1（含）之前的版本，则必须使用相同的 Elasticsearch 版本。

以下集群更新设置请求添加了两个远程集群：cluster_one和cluster_two。

resp = client.cluster.put_settings(
    persistent={
        "cluster": {
            "remote": {
                "cluster_one": {
                    "seeds": [
                        "127.0.0.1:9300"
                    ]
                },
                "cluster_two": {
                    "seeds": [
                        "127.0.0.1:9301"
                    ]
                }
            }
        }
    },
)
print(resp)
response = client.cluster.put_settings(
  body: {
    persistent: {
      cluster: {
        remote: {
          cluster_one: {
            seeds: [
              '127.0.0.1:9300'
            ]
          },
          cluster_two: {
            seeds: [
              '127.0.0.1:9301'
            ]
          }
        }
      }
    }
  }
)
puts response
const response = await client.cluster.putSettings({
  persistent: {
    cluster: {
      remote: {
        cluster_one: {
          seeds: ["127.0.0.1:9300"],
        },
        cluster_two: {
          seeds: ["127.0.0.1:9301"],
        },
      },
    },
  },
});
console.log(response);
PUT /_cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_one": {
          "seeds": [
            "127.0.0.1:9300"
          ]
        },
        "cluster_two": {
          "seeds": [
            "127.0.0.1:9301"
          ]
        }
      }
    }
  }
}

要定位远程集群上的数据流或索引，请使用<cluster>:<target>语法。

resp = client.eql.search(
    index="cluster_one:my-data-stream,cluster_two:my-data-stream",
    query="\n    process where process.name == \"regsvr32.exe\"\n  ",
)
print(resp)
const response = await client.eql.search({
  index: "cluster_one:my-data-stream,cluster_two:my-data-stream",
  query: '\n    process where process.name == "regsvr32.exe"\n  ',
});
console.log(response);
GET /cluster_one:my-data-stream,cluster_two:my-data-stream/_eql/search
{
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

EQL 断路器设置

编辑

相关的断路器设置可以在断路器页面中找到。

« 正则表达式语法 EQL 语法参考 »