diff options
Diffstat (limited to 'docs/reference/analysis/tokenizers/pathhierarchy-tokenizer.asciidoc')
| -rw-r--r-- | docs/reference/analysis/tokenizers/pathhierarchy-tokenizer.asciidoc | 32 |
1 files changed, 32 insertions, 0 deletions
diff --git a/docs/reference/analysis/tokenizers/pathhierarchy-tokenizer.asciidoc b/docs/reference/analysis/tokenizers/pathhierarchy-tokenizer.asciidoc new file mode 100644 index 0000000..e6876f5 --- /dev/null +++ b/docs/reference/analysis/tokenizers/pathhierarchy-tokenizer.asciidoc @@ -0,0 +1,32 @@ +[[analysis-pathhierarchy-tokenizer]] +=== Path Hierarchy Tokenizer + +The `path_hierarchy` tokenizer takes something like this: + +------------------------- +/something/something/else +------------------------- + +And produces tokens: + +------------------------- +/something +/something/something +/something/something/else +------------------------- + +[cols="<,<",options="header",] +|======================================================================= +|Setting |Description +|`delimiter` |The character delimiter to use, defaults to `/`. + +|`replacement` |An optional replacement character to use. Defaults to +the `delimiter`. + +|`buffer_size` |The buffer size to use, defaults to `1024`. + +|`reverse` |Generates tokens in reverse order, defaults to `false`. + +|`skip` |Controls initial tokens to skip, defaults to `0`. +|======================================================================= + |