Elasticsearch Edge NGram 2 Filter
The EdgeNGramFilter2 analysis plugin provides with an extension over the mainstream EdgeNGramTokenFilter.
DEPRECATION NOTICE
This plugin is not compatible with ElasticSearch v0.90.3. Moreover, it is no longer needed since ElasticSearch v0.90.1, as the latter uses Lucene 4.3.0, which contains the fix for bug LUCENE-4810.
Installation
Simply run at the root of your ElasticSearch v0.20.2+ installation:
bin/plugin -install com.yakaz.elasticsearch.plugins/elasticsearch-analysis-edgengram2/1.1.0
This will download the plugin from the Central Maven Repository.
For older versions of ElasticSearch, you can still use the longer:
bin/plugin -url http://oss.sonatype.org/content/repositories/releases/com/yakaz/elasticsearch/plugins/elasticsearch-analysis-edgengram2/1.1.0/elasticsearch-analysis-edgengram2-1.1.0.zip install elasticsearch-analysis-edgengram2
In order to declare this plugin as a dependency, add the following to your pom.xml
:
<dependency>
<groupId>com.yakaz.elasticsearch.plugins</groupId>
<artifactId>elasticsearch-analysis-edgengram2</artifactId>
<version>1.1.0</version>
</dependency>
Version matrix:
-------------------------------------------------
| HashSplitter Analysis Plugin | ElasticSearch |
-------------------------------------------------
| (builtin fixed) | 0.90.1 and up |
-------------------------------------------------
| master | 0.90 -> 0.90.2 |
-------------------------------------------------
| 1.1.0 | 0.90 -> 0.90.2 |
-------------------------------------------------
| 1.0.0 | 0.19 -> 0.20 |
-------------------------------------------------
Description
This plugin exposes an extension over the Edge NGram filter, packaged as an ElasticSearch 0.19.0+ plugin.
See Lucene EdgeNGramTokenFilter JavaDoc for more information about the base functionality.
Added features
Currently there is a single added feature:
-
Preserve input positions
When the filter splits the source token, it generates additional tokens, usually each of them takes a new position on its own. This breaks the structure of the source stream by making two sibling tokens far away, position-wise. Eg.
"foo bar"
becomes0:f 1:fo 2:foo 3:b 4:ba 5:bar
.This new feature permits to output all grams tokens at the same position as the source token, hence
"foo bar"
will yield0:f,fo,foo 1:b,ba,bar
.This is particularly useful when merging with other analysis, using the Combo Analyzer, to prevent position jitter, or when using phrase queries for suggestions.
Please always be aware of the impact of terms positions with regard to your queries.
Configuration
The plugin provides you with the edge_ngram_2
token filter type.
It accepts the same list of parameters as the edge_ngram
token filter, plus:
preserve_positions
:false
by default.
See also
ElasticSearch EdgeNGramFilter doc