"omegaconf.errors.InterpolationKeyError: Interpolation key 'metric' not found" in NucelotideTransformer Benchmark
Closed this issue · 1 comments
Hi,
When I tried to reproduce NT benchmark's results, I met an error that seemed to be related to the train.metric not set in configuration.
I used the script run_nucleotide_transformer.sh you provided and only added some configs needed by this script. I did't change any configs in the default config files.
I know this might be a simple question, but I'm really not familiar with HYDRA configuration parameters, so I'd like to seek your help this time.
Additionally, I would like to ask about the evaluation standards for the NT benchmark. Do you report the average performance on the validation set directly? If so, does that mean you didn't use the test set provided by NT? The NT paper doesn't seem to be very clear on this point—it mentions some evaluations on the test set, which I find confusing. Thank you!