Research `spacy pretrain`
Closed this issue · 2 comments
DoD:
- understand meaning of command
- sample usage on one of models
Sentence: don't use it now
This function is described as experimental, we can't assume it is doing anything useful. For future experiments we may try to test it, but for now it's waste of valuable time.
Description from spacy documentation
Pre-train the “token to vector” (tok2vec) layer of pipeline components, using an approximate language-modeling objective. Specifically, we load pre-trained vectors, and train a component like a CNN, BiLSTM, etc to predict vectors which match the pre-trained ones. The weights are saved to a directory after each epoch. You can then pass a path to one of these pre-trained weights files to the ‘spacy train’ command.
This technique may be especially helpful if you have little labelled data. However, it’s still quite experimental, so your mileage may vary. To load the weights back in during spacy train, you need to ensure all settings are the same between pretraining and training. The API and errors around this need some improvement.
I can't test it, because our machine is still unavailable :(