apertium/lexd

crosstalk between multiple filters with `--minimize --tags`

nlhowell opened this issue · 1 comments

It seems multiple filters can interfere with each other when compiled
with --minimize --tags. This behaviour was observed on master and
on the original commit introducing --tags,
e69a777 compile tag filters as flags

Here's an example:

PATTERNS
Phrase[nofruit,-nocolor] # pattern A
Phrase[-nofruit,nocolor] # pattern B

PATTERN Phrase
Adj Noun

LEXICON Adj
bright[nofruit]

LEXICON Noun
apple[nocolor]
orange[nofruit]

We expect pattern A to generate brightorange and pattern B to
generate nothing. Indeed, separately we get (all compilation modes):

$ lexd onlyA.lexd | hfst-txt2fst | hfst-fst2strings
brightorange
$ lexd onlyB.lexd | hfst-txt2fst | hfst-fst2strings
Error: empty set of final states

With both patterns included in the source:

$ lexd both.lexd | hfst-txt2fst | hfst-fst2strings
brightorange
$ lexd --minimize both.lexd | hfst-txt2fst | hfst-fst2strings -X obey-flags
brightorange
brightorange
$ lexd --minimize --tags both.lexd | hfst-txt2fst | hfst-fst2strings -X obey-flags | uniq -c
      1 brightorange
      8 brightapple
      1 brightorange

Notice the 8 copies of brightapple.

I don't know the flag diacritic codepaths well enough; Daniel, can you
help?

I didn't entirely work out what was actually causing the issue, but I made the handling of --tags a little more consistent, which gets rid of the incorrect output, though the doubling of the correct output is still present.