h0tk3y/better-parse

[BUG] literal token with DefaultTokenizer can't find a match - 'noneMatched'

Closed this issue · 1 comments

I a trying to figure out how this library works but I simply can't get passed the simple literal token example.
Here is my test code:

    @Test
    fun `should match foo`() {
        val literal = literalToken("foo")
        val tokenizer = DefaultTokenizer(listOf(literal))
        val sequence: TokenMatchesSequence = tokenizer.tokenize("my foo")

        assertThat(sequence.toList()).isNotEmpty() // success
        assertThat(sequence[0]!!.input).isEqualTo("my foo") // success
        assertThat(sequence[0]!!.type).isNotEqualTo(noneMatched) // failure
        assertThat(sequence[0]!!.text).isEqualTo("foo") // never reached
    }

This fails with assertThat(sequence[0]!!.type).isNotEqualTo(noneMatched) since there is no match for the simple input of my foo and the literal token looking for foo.

Any idea what I am doing wrong?

Infos:

implementation("com.github.h1tk3y.betterParse:better-parse:0.4.4")

Greetings!

Sorry I just realized I need to call parse on the Token - not check the sequence for a result