mayabot/mynlp

如何添加自己的词典?

Opened this issue · 1 comments

如何添加自己的词典?
        MemCustomDictionary memCustomDictionary = new MemCustomDictionary();

        FluentLexerBuilder builder = Lexers.coreBuilder();

        builder.with(new CustomDictionaryPlugin(memCustomDictionary));

        Lexer tokenizer = builder.build();

        System.out.println(tokenizer);

        System.out.println(tokenizer.scan("欢迎来到松江临港科技城"));

        memCustomDictionary.addWord("临港科技城");
        memCustomDictionary.rebuild();

        System.out.println(tokenizer.scan("欢迎来到松江临港科技城"));

Output:

PipelineTokenizer

BestPathAlgorithm = ViterbiBestPathAlgorithm
CharNormalize = DefaultCharNormalize
WordTermCollector = SentenceCollector
WordSplitAlgorithm = CoreDictionarySplitAlgorithm,AtomSplitAlgorithm
WordpathProcessor = 
	CustomDictionaryProcessor

欢迎 来到 松江 临港 科技城
欢迎 来到 松江 临港科技城