soap117/DeepRule

Recreate numbers in table 2

Opened this issue · 3 comments

Hi,
is there an evaluation script to re-create numbers of table 2 from paper.

The test_pipe_type_cloud.py only gives json as output. Where are the eval metrics ?

This would be really helpful.
Used pre-trained models to benchmark on https://chartinfo.github.io/
the performance score is 0.34 (task 6b) for line charts specifically where their and your metrics described in the paper match exactly.

@crazysal how about the score of task 6a

out of box is 0.73