ltgoslo/simple_elmo

Questions about integrating your models & code

Closed this issue · 4 comments

kylepjohnson commented

Thanks for this repo. I am interested in integrating some of your ELMo models (eg, ID 162 for Latin [http://vectors.nlpl.eu/repository/20/162.zip]) into the CLTK (an NLP framework for ancient/dead languages).

Would you be the right one to answer a few questions about loading these models? I have some questions that may seem very simple to you :)

  1. You have pinned your code to tensorflow to 1.15.2. Is there a specific reason for this? If including TF for my users, I would prefer it to be a little newer. Also, do you have any idea whether keras could load these files?

  2. What other libraries are available to load these models? I ask because there are multiple Python projects capable of using (and sometimes fine-tuning) ELMo, however their conventions for naming files are different. For example, the ELMo directory from NLPL has (config.json, meta.json, word.dic, char.dic, encoder.pkl, token_embedder.pkl) yet I do not see such file types when looking at the "big" ELMo libraries like https://github.com/allenai/allennlp/ .

akutuzov commented

Hi,

  1. The Latin model you mentioned (ID 162) and in fact all the ELMo models trained on the 'CoNLL17 corpus' were not trained by us. As described in their meta.json files, they come from the
    ELMoForManyLangs project. Their code and models are indeed a bit different from the standard ones, and I would advise to use their repository to handle these models.
    The repository we are now in (simple_elmo) contains code to work with more standard ELMo models, like the ones starting from the ID 193 in the NLPL Vector Repository, or the reference models by the ELMo authors.
kylepjohnson commented

Understood, thank you for the fast reply! I will check them out.

akutuzov commented
  1. The code in this repository is pinned to TensorFlow 1.15.2, since this is the most recent release of TensorFlow 1.*. In the future, we do plan to upgrade the code base so that it is fully compatible with TensorFlow 2. But this will require some non-trivial changes.
  • 👍1
kylepjohnson commented

But this will require some non-trivial changes

I understand and have heard this before. Thanks again!