Issues about generation from test dataset
leyi-123 opened this issue · 2 comments
leyi-123 commented
Hello! What do I do if I want to generate responses from a test dataset using a trained model? Thanks!
SivilTaram commented
If you want to evaluate the response using predefined metrics such as F1
, you could use eval_f1.py
to achieve it;
If you want to obtain all generated responses, you could modify the option(line43-line49) in eval_f1.py
as:
parser.set_params(
model='agents.transmitter.transmitter:TransformerAgent',
model_file=model_name,
gpu=1,
batchsize=1,
beam_size=2,
report_freq=1.0
)
And the Text
and Prediction
will be printed as following:
TEXT: <t1start> that is great , me too ! i'm married and my husband and i've 2 children . <t1end>
PREDICTION: i ' m a nurse and i love to spend time with my family
And you could extract them from the log.
leyi-123 commented
Thanks!