Train Xtransformer using PubMedBert in 2016-2019
Closed this issue · 0 comments
nsorros commented
This experiment increases the amount of data from 2018-2019 to 2016-2019 which increases the data from ~1M to ~2M.
The results here were 0.59 which is close to 0.57 that XLinear squeezes from the same data but better. My gut feeling is that there is some further optimisation of hyper parameters that can increase performance here that we need to think carefully.