Reproduce the result in Supplementary Table 1

Question

Reproduce the result in Supplementary Table 1

Closed this issue a year ago · 6 comments

Hi, I found it extremely slow and also problematic when I ran train_model.py with the basic training command (shown in the figure attached), and I want to reproduce the result in your Supplementary Table 1, could you please tell me how I can do it?

Answer 1 · 2022-12-08T19:08:45.000Z

Hi, could you share the loss curve? Maybe the learning rate isn't working, hard to tell from the epoch logs.

Answer 2 · 2022-12-15T05:12:04.000Z

Hi, this is my loss curve for 30 epochs.

Answer 3 · 2022-12-15T09:42:57.000Z

And this is for 140 epochs

Answer 4 · 2022-12-15T13:35:55.000Z

I think the learning rate here is too high.

train_model.py --data data/train_set.fasta --test_partition 0 --validation_partition 1 --output_dir testruns --experiment_name testrun1 --remove_top_layers 1 --kingdom_as_token --sp_region_labels --region_regularization_alpha 0.5 --constrain_crf --average_per_kingdom --optimizer adamax --lr 0.000008

shows performance after one epoch.

Answer 5 · 2022-12-16T05:48:19.000Z

Thank you very much! It works now. And may I know if this repo has the code for producing the benchmark result in Supplementary Table 1?

Answer 6 · 2022-12-16T09:08:24.000Z

Unfortunately there is no code for this. Most of these tools are only available via an online submission interface, so you will have submit the data to the services and parse their outputs.