字母分词器

编辑

letter 字母分词器在遇到非字母字符时将文本拆分为词项。对于大多数欧洲语言来说,它的效果还算合理,但对于某些亚洲语言来说,效果很差,因为这些语言的词语之间没有空格分隔。

示例输出

编辑
resp = client.indices.analyze(
    tokenizer="letter",
    text="The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.",
)
print(resp)
response = client.indices.analyze(
  body: {
    tokenizer: 'letter',
    text: "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
  }
)
puts response
const response = await client.indices.analyze({
  tokenizer: "letter",
  text: "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone.",
});
console.log(response);
POST _analyze
{
  "tokenizer": "letter",
  "text": "The 2 QUICK Brown-Foxes jumped over the lazy dog's bone."
}

上面的句子将产生以下词项:

[ The, QUICK, Brown, Foxes, jumped, over, the, lazy, dog, s, bone ]

配置

编辑

letter 字母分词器不可配置。