Unexpected error in aml-deploy action step
an185144 opened this issue · 6 comments
Hi - we are experiencing an unexpected error which we haven't been able to solve for a while now. All the steps before aml-deploy run just fine, but aml-deploy is suddenly not being able to pull the registered model. The name and version of the model coming from aml-registermodel have been cross-checked and are being correctly passed to aml-deploy. Based on the logs looks like there are also no issues with workspace authorization.
Hoping you've seen anything similar happen with the template and can help us.
Attaching the action log, deploy.json parameters and aml_arm_config json. We have kept the workflow largely the same except for adding some debugging actions.
deploy.json
{
"name": "test",
"deployment_compute_target": "akscompute"
}
aml_arm_config from aml-workspace (found in $GITHUB_WORKSPACE/.azureml/)
{"Id": null, "Scope": "/subscriptions/{subscriptionid}/resourceGroups/{resourcegroup}/providers/Microsoft.MachineLearningServices/workspaces/{workspacename}"}
Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.84. Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.
Hi @an185144,
We are running unit and integration tests and I also have a separate repo, where I test all actions end to end and I was not able to reproduce the issue. Let me take a look tomorrow or next week and get back to you!
Hi @an185144,
The issue you are describing sounds very similar to this one: Azure/aml-deploy#41 (comment)
Can you confirm that it is the same issue?
Hi @marvinbuss - Thanks for looking into this. It does look very similar to the issue you mentioned above. I solved it by switching from 'v1' branch of aml-deploy to 'azureml_1_18 branch'. I think the issue is happening with the Dockerfile with the older version of the SDK.
It occurred to me to look at the Azure ML SDK version because a lot of the issues here are related to the SDK version and thankfully, you already had a branch with the change. I would be looking more into the conditions which result in the issue this week. Thanks for the help.
Hi @an185144,
Thanks for your reply and your investigation. I will release the new version later today. Please switch back to v1, once this is done.
Have a great day!
Hi @an185144,
I just released a new version. Please switch back to v1 tag, whenever you are ready. The latest release makes use of the latest azureml-sdk 1.18.0.