dlite-tools/NLPiper

Custom Tokenizer

Opened this issue ยท 0 comments

๐Ÿš€ Feature

Allow the user to add a custom Tokenizer with minor implementation.

Motivation

It is impossible to integrate every tool, but that does not mean that it shouldn't exist a CustomTokenizer that allows the user to use its own implementation.

Needs: callable function or object
Warnings: If Compose.create_from_steps should throw an error since it cannot load its implementation