jasonwu0731/trade-dst

why not consider the request accuracy?

ha-lins opened this issue · 1 comments

Hi, @jasonwu0731 :

I have a question about your nice work: why not eval the request accu in the experiment? In the original dataset, it consist not only inform slots (as states in your paper) but also request slots(domain + slots). It seems to be achieved by only using the slot gates without the state generator. So why not report the request accuracy?

@ha-lins

request and inform are the dialogue acts.

We evaluated on all the possible slots in each domain but I did not take the dialogue acts into account. If you want, you can also get those request slots out to compute the acc separately.

Hope this help :)