High performance for right-branching strategy

[marcwww]

Really appreciate for releasing the code.

I found when I testing the baseline of right-branching strategy on WSJ test set, the F1 is really high (39.87), which does not match the result in the paper (16.5).

I have just changed the code

gates = model.gates.squeeze().data.cpu().numpy()

into

gates = numpy.arange(len(sen), 0, -1)

, which represent a right-branching strategy.

And the result on WSJ test set is:

So, what my be the reason? Thanks a lot if u could help me out.

[phu-pmh]

Hi, it might be because I don't do pre-processing, and keep punctuations in calculating F1 for the test set, whereas the test_phrase_grammar.py filters them here. I use https://github.com/nyu-mll/spinn/blob/master/scripts/parse_comparison.py code for computing RB, LB, and the other stats (such as NP, PP). Hope this helps.