Inference on large data

[ujjwalx]

Hi,

I am trying to use the pretrained 'absa/classifier-rest-0.2' model for generating labels for approximately 300K reviews. In the README examples, I could not find a way to provide batches of data (batches or list of reviews) to the model. Feeding batches would allow me to potentially speed up the inference process.

I am merely looking to generate labels from your model as a means for a different analysis. There is no intention to train as my task is exactly what you model has been trained on.

Any other ideas on how to speed up the inference process would also be welcome!

[rolczynski]

hmm look at the pipeline.predict method that consumes the input_batch and easily you can modify a pipeline (or build your own) to do what you want 😊

[trinanjan12]

@ujjwalx Hey did you figure out a way to do it?

[ujjwalx]

@trinanjan12 I’m sorry, No.