crosstalk between multiple filters with `--minimize --tags`
nlhowell opened this issue · 1 comments
It seems multiple filters can interfere with each other when compiled
with --minimize --tags
. This behaviour was observed on master
and
on the original commit introducing --tags
,
e69a777 compile tag filters as flags
Here's an example:
PATTERNS
Phrase[nofruit,-nocolor] # pattern A
Phrase[-nofruit,nocolor] # pattern B
PATTERN Phrase
Adj Noun
LEXICON Adj
bright[nofruit]
LEXICON Noun
apple[nocolor]
orange[nofruit]
We expect pattern A to generate brightorange
and pattern B to
generate nothing. Indeed, separately we get (all compilation modes):
$ lexd onlyA.lexd | hfst-txt2fst | hfst-fst2strings
brightorange
$ lexd onlyB.lexd | hfst-txt2fst | hfst-fst2strings
Error: empty set of final states
With both patterns included in the source:
$ lexd both.lexd | hfst-txt2fst | hfst-fst2strings
brightorange
$ lexd --minimize both.lexd | hfst-txt2fst | hfst-fst2strings -X obey-flags
brightorange
brightorange
$ lexd --minimize --tags both.lexd | hfst-txt2fst | hfst-fst2strings -X obey-flags | uniq -c
1 brightorange
8 brightapple
1 brightorange
Notice the 8 copies of brightapple
.
I don't know the flag diacritic codepaths well enough; Daniel, can you
help?
I didn't entirely work out what was actually causing the issue, but I made the handling of --tags
a little more consistent, which gets rid of the incorrect output, though the doubling of the correct output is still present.