why not consider the request accuracy?
ha-lins opened this issue · 1 comments
ha-lins commented
Hi, @jasonwu0731 :
I have a question about your nice work: why not eval the request accu in the experiment? In the original dataset, it consist not only inform slots (as states in your paper) but also request slots(domain + slots). It seems to be achieved by only using the slot gates without the state generator. So why not report the request accuracy?
jasonwu0731 commented
request and inform are the dialogue acts.
We evaluated on all the possible slots in each domain but I did not take the dialogue acts into account. If you want, you can also get those request slots out to compute the acc separately.
Hope this help :)