Internal Server Error when attempting to train Gliner (docker build)
Opened this issue · 0 comments
Hello,
I am getting the following error logs when trying to finetune Gliner, based on annotations which have been corrected via Label Studio. I am using docker for the ml backend - it works fine for suggesting annotations, and the model connects successfully, but fails to start training when requested.
2024-08-15 09:04:01 gliner | [2024-08-15 08:04:01,991] [INFO] [model::fit::235] Downloaded 233 labeled tasks from Label Studio
2024-08-15 09:04:02 gliner | [2024-08-15 08:04:01,995] [ERROR] [label_studio_ml.api::log_exception::1414] Exception on /webhook [POST]
2024-08-15 09:04:02 gliner | Traceback (most recent call last):
2024-08-15 09:04:02 gliner | File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 2190, in wsgi_app
2024-08-15 09:04:02 gliner | response = self.full_dispatch_request()
2024-08-15 09:04:02 gliner | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-15 09:04:02 gliner | File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1486, in full_dispatch_request
2024-08-15 09:04:02 gliner | rv = self.handle_user_exception(e)
2024-08-15 09:04:02 gliner | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-15 09:04:02 gliner | File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1484, in full_dispatch_request
2024-08-15 09:04:02 gliner | rv = self.dispatch_request()
2024-08-15 09:04:02 gliner | ^^^^^^^^^^^^^^^^^^^^^^^
2024-08-15 09:04:02 gliner | File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1469, in dispatch_request
2024-08-15 09:04:02 gliner | return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
2024-08-15 09:04:02 gliner | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-15 09:04:02 gliner | File "/usr/local/lib/python3.11/site-packages/label_studio_ml/api.py", line 126, in webhook
2024-08-15 09:04:02 gliner | model.fit(event, data)
2024-08-15 09:04:02 gliner | File "/app/model.py", line 239, in fit
2024-08-15 09:04:02 gliner | tokens, ner = self.process_training_data(task)
2024-08-15 09:04:02 gliner | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-08-15 09:04:02 gliner | File "/app/model.py", line 132, in process_training_data
2024-08-15 09:04:02 gliner | tokens = task['data']['tokens']
2024-08-15 09:04:02 gliner | ~~~~~~~~~~~~^^^^^^^^^^
2024-08-15 09:04:02 gliner | KeyError: 'tokens'
2024-08-15 09:04:02 gliner | [2024-08-15 08:04:02,089] [DEBUG] [label_studio_ml.api::log_response_info::185] Response status: 500 INTERNAL SERVER ERROR
2024-08-15 09:04:02 gliner | [2024-08-15 08:04:02,089] [DEBUG] [label_studio_ml.api::log_response_info::186] Response headers: Content-Type: text/html; charset=utf-8
2024-08-15 09:04:02 gliner | Content-Length: 265
2024-08-15 09:04:02 gliner |
2024-08-15 09:04:02 gliner |
2024-08-15 09:04:02 gliner | [2024-08-15 08:04:02,090] [DEBUG] [label_studio_ml.api::log_response_info::187] Response body: b'<!doctype html>\n\n<title>500 Internal Server Error</title>\n
Internal Server Error
\nThe server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
\n'