HugAILab/HugNLP

eval metrics返回结果中eval_macro_f1问题

Opened this issue · 0 comments

1、使用run_extractive_unified_ie.sh启动训练,运行正常,结果如下:
***** eval metrics *****
epoch = 2.0
eval_ = 0.0
eval_loss = 8.7072
eval_macro_f1 = 0.0
eval_micro_f1 = 0.0
eval_num = 16
eval_runtime = 0:00:01.11
eval_samples = 16
eval_samples_per_second = 14.409
eval_steps_per_second = 1.801

2、将hugnlp_runner.py文件中parser.parse_args_into_dataclasses()改为 parser.parse_json_file(‘config.json'),训练结果如下:
***** eval metrics *****
epoch = 2.0
eval_runtime = 0:00:01.05
eval_samples = 16
eval_samples_per_second = 15.134
eval_steps_per_second = 1.892

eval_macro_f1属性不见了。请问是什么原因?

config.json结构如下:
{
"model_name_or_path": "/chinese-macbert-large",
"data_dir": "/dataset",
"output_dir": "/output",
"seed": 42,
"exp_name": "unified-ie-wjn",
"max_seq_length": 512,
"max_eval_seq_length": 512,
"do_train": true,
"do_eval": true,
"per_device_train_batch_size": 8,
"per_device_eval_batch_size": 8,
"gradient_accumulation_steps": 1,
"evaluation_strategy": "steps",
"learning_rate": 2e-05,
"num_train_epochs": 20,
"logging_steps": 100000000,
"eval_steps": 500,
"save_steps": 500,
"save_total_limit": 1,
"warmup_steps": 200,
"load_best_model_at_end": true,
"report_to": "none",
"task_name": "zh_mrc_instruction",
"task_type": "global_pointer",
"model_type": "bert",
"metric_for_best_model": "macro_f1",
"pad_to_max_length": true,
"remove_unused_columns": false,
"overwrite_output_dir": true,
"fp16": true,
"label_names": "short_labels",
"keep_predict_labels": true,
"cache_dir": "/cache"
}