jasonwu0731/trade-dst

Modifications required for running experiments on MultiWOZ2.1

ashutoshml opened this issue · 2 comments

ashutoshml commented

Hi,

I am wondering what all files need to be changed for evaluating on MultiWOZ2.1 dataset.

Based on preliminary analysis I found the following files:

  1. create_data.py
  2. utils/utils_multiWOZ_DST.py and
  3. evaluation part in models/TRADE.py.
    Please let me know if there are any other files that need to be modified.

Also, incase the changes already exist, please point me to them.

Thanks.

jasonwu0731 commented

It should work if you change this function as MWOZ 2.1 and 2.0 have the same format.

ashutoshml commented

The slot labels are different too. restaurant-semi-price in MultiWOZ2.1 as compared to restaurant-price MultiWOZ2.0.
Due to this difference in labels, class Lang needs to be changed too in utils/utils_multWOZ_DST.py
The evaluation also might need tweaking, since I am getting 0.12 Joint Accuracy(MultiWOZ2.1) as compared to ~0.48 (MultiWOZ2).

In case these issues have already been addressed, please let me know.