/textator

Build your own tokenizer.

Primary LanguageJavaScript

Textator (!WORK IN PROGRESS)

This repository contains a tool too easily build your own tokenizer.

Simple usage

// Preperation BEGIN. Just do the following once.
const myTokens = [
    {   // detecting links
        regExp: /\[(.+?)\]\((.+?)\)/, // two vars: text & url
        process(meta, text, href) {
            return `<a href="${href}">${text}</a>`;
        }
    },
    {   // detecting line breaks
        regExp: /(\n)/,
        process(meta, linebreak) {
            return `<br />`;
        }
    },
];
const customParser = new Textator(myTokens);
// Preperation END. customParser is ready to go.

const input = 'Some super cool text which contains a [link](http://example.com) and... \n also a line break.';

// maybe better serialize your input before processing it.
const output = customParser(input);

console.log(output);
// Some super cool text which contains a <a href="http://example.com">link</a>and... <br /> also a line break.

Questions & Answers

  • Can it be used for inline and block elements? — Yes and no. At the moment there is no feature for recognizing two patterns for one substring. So I recommend you to build a customInlineParser and a customBlockParser and let them run one after the other.