nlpodyssey/spago

Better BERT Server Capabilities

bonedaddy opened this issue · 4 comments

bonedaddy commented

(apologies if the many issues are annoying, really liking the library so far)

Overview

Right now when using the BERT server, a set of defaults are used without any control from the user:

  • HTTP server lacks customizability
  • HTTP server listens on 0.0.0.0
  • No ability to enable TLS

Additionally it's not possible to build "external servers" as the functions used by the BERT http router (discriminateHandler, predictionHandler, qaHandler are private functions). If this was changed to instead export the handler functions (DiscriminateHandler, PredictionHandler, QaHandler) it would allow people to have more control over the BERT server, better middleware capabilities, TLS, etc...

By using public handler functions, users would be able to define their own routers, say using chi, and overall have more control of the BERT server.

I'd be more than happy to open a PR that implements these suggestions

matteo-grella commented

Dear @bonedaddy,

many thanks for your suggestions.

Please go ahead with your PR!

I know that @evanmcclure also plans to improve the server, including:

  • Return the elapsed time it takes to complete an API function in the results, in order to compare implementations, doc sizes, models, etc.

  • Support HTTPS by default, to maintain best practices in delivering cloud-native applications.

What about organizing a chat on slack?

Best,

Matteo

evanmcclure commented
bonedaddy commented

Slack works for me, although my personal preference is discord. email me slack invite postables@rtradetechnologies.com, or a discord handle

matteo-grella commented

I'd say we've implemented everything discussed here on the BERT server.

I'm closing this issue.