如何添加自己的词典?
Opened this issue · 1 comments
zz1559152814 commented
如何添加自己的词典?
jimichan commented
MemCustomDictionary memCustomDictionary = new MemCustomDictionary();
FluentLexerBuilder builder = Lexers.coreBuilder();
builder.with(new CustomDictionaryPlugin(memCustomDictionary));
Lexer tokenizer = builder.build();
System.out.println(tokenizer);
System.out.println(tokenizer.scan("欢迎来到松江临港科技城"));
memCustomDictionary.addWord("临港科技城");
memCustomDictionary.rebuild();
System.out.println(tokenizer.scan("欢迎来到松江临港科技城"));
Output:
PipelineTokenizer
BestPathAlgorithm = ViterbiBestPathAlgorithm
CharNormalize = DefaultCharNormalize
WordTermCollector = SentenceCollector
WordSplitAlgorithm = CoreDictionarySplitAlgorithm,AtomSplitAlgorithm
WordpathProcessor =
CustomDictionaryProcessor
欢迎 来到 松江 临港 科技城
欢迎 来到 松江 临港科技城