Online learning

Question

Online learning

sademakn opened this issue 4 years ago · 5 comments

Is there any way to update the model on the fly with new data?

Answer 1 · 2021-05-04T19:15:58.000Z

The embeddings are stable, which means, you can re-run on (old + new) data, and the results will be aligned with the old embeddings, for any downstream purposes.

Currently on-the-fly updates are not implemented, but what would be required is as follows:

incremental creation of the sparse transition matrix,
periodic recalculation of the embeddings.

Answer 2 · 2021-05-05T12:29:30.000Z

Thanks for your well-organized answer,
If someday the on-the-fly updates will be available, that would help a lot with large-scale and non-stop data streams. In my case, I use a cronjob to train my model on whole data(old + new) and each time it takes hours to build a new model and during the training also I'm receiving a bunch of new data!
Is there any hope to achieve on-the-fly updates anytime soon?

Answer 3 · 2021-05-05T16:59:01.000Z

For now there is no plan but I will come back to you next week because I have to check the pipeline. We'll see what we can do.

Answer 4 · 2021-05-14T13:01:34.000Z

Hi @sademakn !

I checked and unfortunately there is no chance to provide the feature within 1 or 2 months.

We will be updating if something change.

Answer 5 · 2021-11-16T11:47:59.000Z

@sademakn Hi!
We encourage you to contribute. Unfortunately, it will not be possible to implement such a solution on our side. If it happens that a large group of users will expect such a feature, we will try to allocate time for implementation.