How to adapt this framework to a new dataset?

[luoxindi]

Hello, thanks for your repository, it inspires me a lot. For the QA dataset, I wonder how to adapt this framework to the GrailQA dataset? What changes do I need to make to this framework？

[WuXuan374]

Hi, we haven't tried to adapt this framework to the GrailQA dataset, so we can only provide some suggestions based on our understanding. We'd like to follow the sequence in README.md.

• (2) Parse SPARQL queries to S-expressions.
  This part of code is modified based on RNG-KBQA. Since GrailQA is also based on Freebase, this part of code should work. But it will be better if you run a coverage test on SPARQLs from GrailQA, to check if there are SPARQLs our code fails to handle.
• (3) Retrieve Candidate Entities and (4) Retrieve Candidate Relations.
  These two steps aims to provide candidate entities/relations for our multi-task model. Both migrating our method for GrailQA and using results of other linking tools should work.
• (5) Prepare data for multi-task model.
  We guess there should be some file format issue for a new dataset.
• (6) Generate Logical Forms through multi-task learning.
  We think the multi-task model itself should work without modification. But similar as (2), when converting S-expression back to SPARQL in eval_topk_prediction_final.py, you may need to run a coverage test for unhandled cases from GrailQA.

These are possible changes we can think of at current stage. Hope this helps.

[luoxindi]

Thanks for your guidance!