Connect Model Validation error
Closed this issue · 1 comments
I met the problem when setting my own ML model, can you help me ?
Validation error
Successfully connected to http://localhost:9090 but it doesn't look like a valid ML backend. Reason: HTTPConnectionPool(host='localhost', port=9090): Read timed out. (read timeout=3.0).
Check the ML backend server console logs to check the status.There might be something wrong with your model or it might be incompatible with the current labeling configuration.
below is my predict part of model.py
def predict(self, tasks: List[Dict], context: Optional[Dict] = None, **kwargs) -> ModelResponse:
""" Write your inference logic here
:param tasks: [Label Studio tasks in JSON format](https://labelstud.io/guide/task_format.html)
:param context: [Label Studio context in JSON format](https://labelstud.io/guide/ml_create#Implement-prediction-logic)
:return model_response
ModelResponse(predictions=predictions) with
predictions: [Predictions array in JSON format](https://labelstud.io/guide/export.html#Label-Studio-JSON-format-of-annotated-tasks)
"""
predictions = []
for task in tasks:
text = task['data']['text']
token = self.tokenizer(text, add_special_tokens=True, padding='max_length', truncation=True, max_length=512)
input_ids = token['input_ids']
attention_mask = token['attention_mask']
token_type_ids = token['token_type_ids']
input_ids = torch.tensor([input_ids], dtype=torch.long)
attention_mask = torch.tensor([attention_mask], dtype=torch.long)
token_type_ids = torch.tensor([token_type_ids], dtype=torch.long)
with torch.no_grad():
outputs = self.model(
input_ids,
attention_mask,
token_type_ids,
)
probabilities = F.softmax(outputs, dim=1)
pred_label_index = torch.argmax(probabilities, dim=1).item()
confidence = probabilities[0][pred_label_index].item()
pred_label = self.labels[pred_label_index]
prediction = {
"model_version": self.get("model_version"),
"score": confidence,
"result": [{
"from_name": "sentiment",
"to_name": "text",
"type": "choices",
"value": {
"choices": [pred_label]
}
}]
}
predictions.append(prediction)
return ModelResponse(predictions=predictions)
below is label-studio-ml log
(label_studio) [root@localhost label-studio-ml-backend]# LOG_LEVEL=DEBUG label-studio-ml start my_ml_backend --debug
There are 2 GPU(s) available.
We will use the GPU: NVIDIA GeForce GTX 1080 Ti
- Serving Flask app 'label_studio_ml.api'
- Debug mode: on
[2024-09-25 15:30:45,742] [INFO] [werkzeug::_log::97] WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead. - Running on all addresses (0.0.0.0)
- Running on
- Running on
[2024-09-25 15:30:45,743] [INFO] [werkzeug::_log::97] Press CTRL+C to quit
[2024-09-25 15:30:45,744] [INFO] [werkzeug::_log::97] * Restarting with stat
There are 2 GPU(s) available.
We will use the GPU: NVIDIA GeForce GTX 1080 Ti
[2024-09-25 15:30:49,281] [WARNING] [werkzeug::_log::97] * Debugger is active!
[2024-09-25 15:30:49,283] [INFO] [werkzeug::_log::97] * Debugger PIN: 433-440-643
[2024-09-25 15:30:54,547] [DEBUG] [label_studio_ml.api::log_request_info::185] Request headers: Host: localhost:9090
Sentry-Trace: 1b921b038c8542e8a5be3350b16a43f1-a6fe26ea5aa7dbe7-0
Baggage: sentry-trace_id=1b921b038c8542e8a5be3350b16a43f1,sentry-environment=opensource,sentry-release=label-studio%401.13.1,sentry-public_key=68b045ab408a4d32a910d339be8591a4,sentry-transaction=/api/ml/,sentry-sample_rate=0.02,sentry-sampled=false
User-Agent: heartex/1.13.1+0.gd9b816a.dirty
Accept-Encoding: gzip, deflate
Accept: /
Connection: keep-alive
[2024-09-25 15:30:54,547] [DEBUG] [label_studio_ml.api::log_request_info::186] Request body: b''
[2024-09-25 15:30:54,548] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 200 OK
[2024-09-25 15:30:54,548] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json
Content-Length: 50
[2024-09-25 15:30:54,548] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{\n "model_class": "NewModel",\n "status": "UP"\n}\n'
[2024-09-25 15:30:54,549] [INFO] [werkzeug::_log::97] 127.0.0.1 - - [25/Sep/2024 15:30:54] "GET /health HTTP/1.1" 200 -
[2024-09-25 15:30:54,554] [DEBUG] [label_studio_ml.api::log_request_info::185] Request headers: Host: localhost:9090
Sentry-Trace: 1b921b038c8542e8a5be3350b16a43f1-917d1085e6837a6e-0
Baggage: sentry-trace_id=1b921b038c8542e8a5be3350b16a43f1,sentry-environment=opensource,sentry-release=label-studio%401.13.1,sentry-public_key=68b045ab408a4d32a910d339be8591a4,sentry-transaction=/api/ml/,sentry-sample_rate=0.02,sentry-sampled=false
User-Agent: heartex/1.13.1+0.gd9b816a.dirty
Accept-Encoding: gzip, deflate
Accept: /
Connection: keep-alive
Content-Length: 1003
Content-Type: application/json
[2024-09-25 15:30:54,555] [DEBUG] [label_studio_ml.api::log_request_info::186] Request body: b'{"project": "3.1727088695", "schema": "\n <Text name=\"text\" value=\"$text\"/>\n <View style=\"box-shadow: 2px 2px 5px #999; padding: 20px; margin-top: 2em; border-radius: 5px;\">\n <Header value=\"Choose text sentiment\"/>\n <Choices name=\"sentiment\" toName=\"text\" choice=\"single\" showInLine=\"true\">\n \n \n \n <Choice value=\"\u5176\u5b83\u6587\u672c\"/><Choice value=\"\u6deb\u79fd\u8272\u60c5\"/><Choice value=\"\u6b3a\u8bc8\u6587\u672c\"/><Choice value=\"\u66b4\u6050\u8fdd\u6cd5\"/><Choice value=\"\u653f\u6cbb\u654f\u611f\"/><Choice value=\"\u4fae\u8fb1\u8c29\u9a82\"/><Choice value=\"\u8d4c\u535a\u5f69\u7968\"/><Choice value=\"\u5e7f\u544a\u63a8\u9500\"/>\n \n", "hostname": "http://localhost:8080", "access_token": "f9fc6984680f507112daeed288bedbb6081911c6", "extra_params": ""}'
[2024-09-25 15:30:59,948] [DEBUG] [label_studio_ml.api::log_response_info::191] Response status: 200 OK
[2024-09-25 15:30:59,949] [DEBUG] [label_studio_ml.api::log_response_info::192] Response headers: Content-Type: application/json
Content-Length: 41
[2024-09-25 15:30:59,949] [DEBUG] [label_studio_ml.api::log_response_info::193] Response body: b'{\n "model_version": "NewModel-v0.0.1"\n}\n'
[2024-09-25 15:30:59,949] [INFO] [werkzeug::_log::97] 127.0.0.1 - - [25/Sep/2024 15:30:59] "POST /setup HTTP/1.1" 200 -
Closed,
just set the ML_TIMEOUT_SETUP and restart label studio.