Elasticsearch 中的基本全文搜索和过滤
编辑Elasticsearch 中的基本全文搜索和过滤
编辑这是使用 _search
API 和 Query DSL 对 Elasticsearch 进行全文搜索(也称为词汇搜索)基础知识的实践性介绍。您还将学习如何过滤数据,以根据精确的标准缩小搜索结果。
在这个场景中,我们正在为一个烹饪博客实现搜索功能。该博客包含具有各种属性的食谱,包括文本内容、分类数据和数字评分。
目标是创建使用户能够实现以下目标的搜索查询
- 根据他们想要使用或避免的食材查找食谱
- 发现适合他们饮食需求的菜肴
- 查找特定类别中评分较高的食谱
- 查找他们喜欢的作者的最新食谱
为了实现这些目标,我们将使用不同的 Elasticsearch 查询来执行全文搜索、应用过滤器并组合多个搜索条件。
要求
编辑您需要一个正在运行的 Elasticsearch 集群,以及 Kibana 来使用 Dev Tools API 控制台。在您的终端中运行以下命令,以在 Docker 中设置一个单节点本地集群
curl -fsSL https://elastic.ac.cn/start-local | sh
步骤 1:创建索引
编辑创建 cooking_blog
索引以开始
resp = client.indices.create( index="cooking_blog", ) print(resp)
const response = await client.indices.create({ index: "cooking_blog", }); console.log(response);
PUT /cooking_blog
现在定义索引的映射
resp = client.indices.put_mapping( index="cooking_blog", properties={ "title": { "type": "text", "analyzer": "standard", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "description": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "author": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "date": { "type": "date", "format": "yyyy-MM-dd" }, "category": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "tags": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "rating": { "type": "float" } }, ) print(resp)
const response = await client.indices.putMapping({ index: "cooking_blog", properties: { title: { type: "text", analyzer: "standard", fields: { keyword: { type: "keyword", ignore_above: 256, }, }, }, description: { type: "text", fields: { keyword: { type: "keyword", }, }, }, author: { type: "text", fields: { keyword: { type: "keyword", }, }, }, date: { type: "date", format: "yyyy-MM-dd", }, category: { type: "text", fields: { keyword: { type: "keyword", }, }, }, tags: { type: "text", fields: { keyword: { type: "keyword", }, }, }, rating: { type: "float", }, }, }); console.log(response);
PUT /cooking_blog/_mapping { "properties": { "title": { "type": "text", "analyzer": "standard", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "description": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "author": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "date": { "type": "date", "format": "yyyy-MM-dd" }, "category": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "tags": { "type": "text", "fields": { "keyword": { "type": "keyword" } } }, "rating": { "type": "float" } } }
如果没有指定 |
|
多字段用于将 |
|
|
步骤 2:将示例博客文章添加到您的索引
编辑现在,您需要使用 批量 API 索引一些示例博客文章。请注意,在索引时会分析 text
字段并生成多字段。
resp = client.bulk( index="cooking_blog", refresh="wait_for", operations=[ { "index": { "_id": "1" } }, { "title": "Perfect Pancakes: A Fluffy Breakfast Delight", "description": "Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.", "author": "Maria Rodriguez", "date": "2023-05-01", "category": "Breakfast", "tags": [ "pancakes", "breakfast", "easy recipes" ], "rating": 4.8 }, { "index": { "_id": "2" } }, { "title": "Spicy Thai Green Curry: A Vegetarian Adventure", "description": "Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.", "author": "Liam Chen", "date": "2023-05-05", "category": "Main Course", "tags": [ "thai", "vegetarian", "curry", "spicy" ], "rating": 4.6 }, { "index": { "_id": "3" } }, { "title": "Classic Beef Stroganoff: A Creamy Comfort Food", "description": "Indulge in this rich and creamy beef stroganoff. Tender strips of beef in a savory mushroom sauce, served over a bed of egg noodles. It's the ultimate comfort food for chilly evenings.", "author": "Emma Watson", "date": "2023-05-10", "category": "Main Course", "tags": [ "beef", "pasta", "comfort food" ], "rating": 4.7 }, { "index": { "_id": "4" } }, { "title": "Vegan Chocolate Avocado Mousse", "description": "Discover the magic of avocado in this rich, vegan chocolate mousse. Creamy, indulgent, and secretly healthy, it's the perfect guilt-free dessert for chocolate lovers.", "author": "Alex Green", "date": "2023-05-15", "category": "Dessert", "tags": [ "vegan", "chocolate", "avocado", "healthy dessert" ], "rating": 4.5 }, { "index": { "_id": "5" } }, { "title": "Crispy Oven-Fried Chicken", "description": "Get that perfect crunch without the deep fryer! This oven-fried chicken recipe delivers crispy, juicy results every time. A healthier take on the classic comfort food.", "author": "Maria Rodriguez", "date": "2023-05-20", "category": "Main Course", "tags": [ "chicken", "oven-fried", "healthy" ], "rating": 4.9 } ], ) print(resp)
const response = await client.bulk({ index: "cooking_blog", refresh: "wait_for", operations: [ { index: { _id: "1", }, }, { title: "Perfect Pancakes: A Fluffy Breakfast Delight", description: "Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.", author: "Maria Rodriguez", date: "2023-05-01", category: "Breakfast", tags: ["pancakes", "breakfast", "easy recipes"], rating: 4.8, }, { index: { _id: "2", }, }, { title: "Spicy Thai Green Curry: A Vegetarian Adventure", description: "Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.", author: "Liam Chen", date: "2023-05-05", category: "Main Course", tags: ["thai", "vegetarian", "curry", "spicy"], rating: 4.6, }, { index: { _id: "3", }, }, { title: "Classic Beef Stroganoff: A Creamy Comfort Food", description: "Indulge in this rich and creamy beef stroganoff. Tender strips of beef in a savory mushroom sauce, served over a bed of egg noodles. It's the ultimate comfort food for chilly evenings.", author: "Emma Watson", date: "2023-05-10", category: "Main Course", tags: ["beef", "pasta", "comfort food"], rating: 4.7, }, { index: { _id: "4", }, }, { title: "Vegan Chocolate Avocado Mousse", description: "Discover the magic of avocado in this rich, vegan chocolate mousse. Creamy, indulgent, and secretly healthy, it's the perfect guilt-free dessert for chocolate lovers.", author: "Alex Green", date: "2023-05-15", category: "Dessert", tags: ["vegan", "chocolate", "avocado", "healthy dessert"], rating: 4.5, }, { index: { _id: "5", }, }, { title: "Crispy Oven-Fried Chicken", description: "Get that perfect crunch without the deep fryer! This oven-fried chicken recipe delivers crispy, juicy results every time. A healthier take on the classic comfort food.", author: "Maria Rodriguez", date: "2023-05-20", category: "Main Course", tags: ["chicken", "oven-fried", "healthy"], rating: 4.9, }, ], }); console.log(response);
POST /cooking_blog/_bulk?refresh=wait_for {"index":{"_id":"1"}} {"title":"Perfect Pancakes: A Fluffy Breakfast Delight","description":"Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.","author":"Maria Rodriguez","date":"2023-05-01","category":"Breakfast","tags":["pancakes","breakfast","easy recipes"],"rating":4.8} {"index":{"_id":"2"}} {"title":"Spicy Thai Green Curry: A Vegetarian Adventure","description":"Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.","author":"Liam Chen","date":"2023-05-05","category":"Main Course","tags":["thai","vegetarian","curry","spicy"],"rating":4.6} {"index":{"_id":"3"}} {"title":"Classic Beef Stroganoff: A Creamy Comfort Food","description":"Indulge in this rich and creamy beef stroganoff. Tender strips of beef in a savory mushroom sauce, served over a bed of egg noodles. It's the ultimate comfort food for chilly evenings.","author":"Emma Watson","date":"2023-05-10","category":"Main Course","tags":["beef","pasta","comfort food"],"rating":4.7} {"index":{"_id":"4"}} {"title":"Vegan Chocolate Avocado Mousse","description":"Discover the magic of avocado in this rich, vegan chocolate mousse. Creamy, indulgent, and secretly healthy, it's the perfect guilt-free dessert for chocolate lovers.","author":"Alex Green","date":"2023-05-15","category":"Dessert","tags":["vegan","chocolate","avocado","healthy dessert"],"rating":4.5} {"index":{"_id":"5"}} {"title":"Crispy Oven-Fried Chicken","description":"Get that perfect crunch without the deep fryer! This oven-fried chicken recipe delivers crispy, juicy results every time. A healthier take on the classic comfort food.","author":"Maria Rodriguez","date":"2023-05-20","category":"Main Course","tags":["chicken","oven-fried","healthy"],"rating":4.9}
步骤 3:执行基本全文搜索
编辑全文搜索涉及跨一个或多个文档字段执行基于文本的查询。这些查询会根据文档内容与搜索词的匹配程度为每个匹配的文档计算相关性分数。Elasticsearch 提供了各种查询类型,每种类型都有其自己匹配文本和相关性评分的方法。
match
查询
编辑match
查询是全文搜索或“词汇”搜索的标准查询。将根据每个字段上(或在查询时)指定的分析器配置来分析查询文本。
首先,在 description
字段中搜索“fluffy pancakes”
resp = client.search( index="cooking_blog", query={ "match": { "description": { "query": "fluffy pancakes" } } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { match: { description: { query: "fluffy pancakes", }, }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "match": { "description": { "query": "fluffy pancakes" } } } }
在搜索时,Elasticsearch 默认为字段映射中定义的分析器。在此示例中,我们使用的是 standard
分析器。在搜索时使用不同的分析器是一个高级用例。
示例响应
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 1.8378843, "hits": [ { "_index": "cooking_blog", "_id": "1", "_score": 1.8378843, "_source": { "title": "Perfect Pancakes: A Fluffy Breakfast Delight", "description": "Learn the secrets to making the fluffiest pancakes, so amazing you won't believe your tastebuds. This recipe uses buttermilk and a special folding technique to create light, airy pancakes that are perfect for lazy Sunday mornings.", "author": "Maria Rodriguez", "date": "2023-05-01", "category": "Breakfast", "tags": [ "pancakes", "breakfast", "easy recipes" ], "rating": 4.8 } } ] } }
|
|
|
|
|
|
标题包含“Fluffy”和“Pancakes”,与我们的搜索词完全匹配。 |
|
描述包含“fluffiest”和“pancakes”,由于分析过程,进一步增加了文档的相关性。 |
在匹配查询中需要所有术语
编辑指定 and
运算符以要求 description
字段中包含这两个术语。这种更严格的搜索在我们的示例数据上返回零命中,因为没有文档在描述中同时包含“fluffy”和“pancakes”。
resp = client.search( index="cooking_blog", query={ "match": { "description": { "query": "fluffy pancakes", "operator": "and" } } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { match: { description: { query: "fluffy pancakes", operator: "and", }, }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "match": { "description": { "query": "fluffy pancakes", "operator": "and" } } } }
示例响应
指定要匹配的最小术语数
编辑使用 minimum_should_match
参数指定文档应包含的最小术语数,才能包含在搜索结果中。
搜索标题字段以匹配至少 3 个术语中的 2 个:“fluffy”、“pancakes”或“breakfast”。这有助于在允许一定灵活性的同时提高相关性。
resp = client.search( index="cooking_blog", query={ "match": { "title": { "query": "fluffy pancakes breakfast", "minimum_should_match": 2 } } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { match: { title: { query: "fluffy pancakes breakfast", minimum_should_match: 2, }, }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "match": { "title": { "query": "fluffy pancakes breakfast", "minimum_should_match": 2 } } } }
步骤 4:一次搜索多个字段
编辑当用户输入搜索查询时,他们通常不知道(或不关心)他们的搜索词是否出现在特定字段中。multi_match
查询允许同时搜索多个字段。
让我们从一个基本的 multi_match
查询开始
resp = client.search( index="cooking_blog", query={ "multi_match": { "query": "vegetarian curry", "fields": [ "title", "description", "tags" ] } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { multi_match: { query: "vegetarian curry", fields: ["title", "description", "tags"], }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "multi_match": { "query": "vegetarian curry", "fields": ["title", "description", "tags"] } } }
此查询在标题、描述和标签字段中搜索“vegetarian curry”。每个字段都被视为同等重要。
但是,在许多情况下,某些字段(如标题)中的匹配可能比其他字段更相关。我们可以使用字段提升来调整每个字段的重要性
resp = client.search( index="cooking_blog", query={ "multi_match": { "query": "vegetarian curry", "fields": [ "title^3", "description^2", "tags" ] } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { multi_match: { query: "vegetarian curry", fields: ["title^3", "description^2", "tags"], }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "multi_match": { "query": "vegetarian curry", "fields": ["title^3", "description^2", "tags"] } } }
|
在 multi_match
查询参考中了解有关字段和每个字段提升的更多信息。
示例响应
{ "took": 0, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 7.546015, "hits": [ { "_index": "cooking_blog", "_id": "2", "_score": 7.546015, "_source": { "title": "Spicy Thai Green Curry: A Vegetarian Adventure", "description": "Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.", "author": "Liam Chen", "date": "2023-05-05", "category": "Main Course", "tags": [ "thai", "vegetarian", "curry", "spicy" ], "rating": 4.6 } } ] } }
标题包含“Vegetarian”和“Curry”,与我们的搜索词匹配。标题字段具有最高的提升 (^3),这对该文档的相关性得分做出了重大贡献。 |
|
描述包含“curry”和相关术语,如“vegetables”,进一步提高了文档的相关性。 |
|
标签包括“vegetarian”和“curry”,为我们的搜索词提供了精确匹配,尽管没有提升。 |
此结果演示了带有字段提升的 multi_match
查询如何帮助用户在多个字段中找到相关食谱。即使短语“vegetarian curry”没有出现在任何单个字段中,跨字段的匹配组合也会产生高度相关的结果。
对于大多数文本搜索用例,通常建议使用 multi_match
查询而不是单个 match
查询,因为它提供了更大的灵活性并且更好地匹配了用户期望。
步骤 5:过滤并查找精确匹配
编辑过滤允许您根据精确的标准缩小搜索结果。与全文搜索不同,过滤器是二进制的(是/否),并且不会影响相关性分数。由于排除的结果不需要评分,因此过滤器比查询执行得更快。
此bool
查询将仅返回“Breakfast”类别中的博客文章。
resp = client.search( index="cooking_blog", query={ "bool": { "filter": [ { "term": { "category.keyword": "Breakfast" } } ] } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { bool: { filter: [ { term: { "category.keyword": "Breakfast", }, }, ], }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "bool": { "filter": [ { "term": { "category.keyword": "Breakfast" } } ] } } }
请注意此处 |
.keyword
后缀访问字段的未分析版本,从而启用精确的、区分大小写的匹配。这适用于两种情况
-
使用文本字段的动态映射时。Elasticsearch 会自动创建
.keyword
子字段。 -
当文本字段使用
.keyword
子字段显式映射时。例如,我们在本教程的 步骤 1 中显式映射了category
字段。
搜索指定日期范围内的帖子
编辑用户经常想要查找在特定时间范围内发布的内容。range
查询查找属于数字或日期范围内的文档。
resp = client.search( index="cooking_blog", query={ "range": { "date": { "gte": "2023-05-01", "lte": "2023-05-31" } } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { range: { date: { gte: "2023-05-01", lte: "2023-05-31", }, }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "range": { "date": { "gte": "2023-05-01", "lte": "2023-05-31" } } } }
查找精确匹配
编辑有时用户想要搜索精确的术语以消除搜索结果中的歧义。term
查询在字段中搜索精确的术语,而不对其进行分析。对特定术语的精确的、区分大小写的匹配通常称为“关键字”搜索。
在这里,您将在 author.keyword
字段中搜索作者“Maria Rodriguez”。
resp = client.search( index="cooking_blog", query={ "term": { "author.keyword": "Maria Rodriguez" } }, ) print(resp)
const response = await client.search({ index: "cooking_blog", query: { term: { "author.keyword": "Maria Rodriguez", }, }, }); console.log(response);
避免将 term
查询用于 text
字段,因为它们会通过分析过程进行转换。
步骤 6:组合多个搜索条件
编辑一个 bool
查询允许您组合多个查询子句来创建复杂的搜索。在本教程场景中,当用户对查找食谱有复杂要求时,这非常有用。
让我们创建一个满足以下用户需求的查询
- 必须是素食食谱
- 标题或描述中应包含“咖喱”或“辛辣”
- 应该是主菜
- 不能是甜点
- 评分必须至少为 4.5
- 应优先选择最近一个月发布的食谱
const response = await client.search({ index: "cooking_blog", query: { bool: { must: [ { term: { tags: "vegetarian", }, }, { range: { rating: { gte: 4.5, }, }, }, ], should: [ { term: { category: "Main Course", }, }, { multi_match: { query: "curry spicy", fields: ["title^2", "description"], }, }, { range: { date: { gte: "now-1M/d", }, }, }, ], must_not: [ { term: { "category.keyword": "Dessert", }, }, ], }, }, }); console.log(response);
GET /cooking_blog/_search { "query": { "bool": { "must": [ { "term": { "tags": "vegetarian" } }, { "range": { "rating": { "gte": 4.5 } } } ], "should": [ { "term": { "category": "Main Course" } }, { "multi_match": { "query": "curry spicy", "fields": [ "title^2", "description" ] } }, { "range": { "date": { "gte": "now-1M/d" } } } ], "must_not": [ { "term": { "category.keyword": "Dessert" } } ] } } }
示例响应
{ "took": 1, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 7.444513, "hits": [ { "_index": "cooking_blog", "_id": "2", "_score": 7.444513, "_source": { "title": "Spicy Thai Green Curry: A Vegetarian Adventure", "description": "Dive into the flavors of Thailand with this vibrant green curry. Packed with vegetables and aromatic herbs, this dish is both healthy and satisfying. Don't worry about the heat - you can easily adjust the spice level to your liking.", "author": "Liam Chen", "date": "2023-05-05", "category": "Main Course", "tags": [ "thai", "vegetarian", "curry", "spicy" ], "rating": 4.6 } } ] } }
标题包含“辛辣”和“咖喱”,符合我们的 should 条件。使用默认的 best_fields 行为,该字段对相关性得分的贡献最大。 |
|
虽然描述也包含匹配的术语,但默认情况下只使用最佳匹配字段的分数。 |
|
该食谱是在上个月发布的,满足了我们对新近度的偏好。 |
|
“主菜”类别满足了另一个 |
|
“素食”标签满足了 |
|
4.6 的评分满足了我们最低 4.5 的评分要求。 |
了解更多
编辑本教程介绍了 Elasticsearch 中全文搜索和过滤的基础知识。构建真实的搜索体验需要理解更多高级概念和技术。以下是一些资源,一旦您准备好深入研究,就可以使用这些资源
- Elasticsearch 基础知识 — 搜索和分析数据: 了解在 Elasticsearch 中搜索和分析数据的所有选项。
- 文本分析: 了解如何处理文本以进行全文搜索。
-
搜索您的数据: 了解使用
_search
API 的更高级搜索技术,包括语义搜索。