az ml online-endpoint create raises Resource Not Found Error
kmr0877 opened this issue · 5 comments
kmr0877 commented
Describe the bug
When trying to create an endpoint using AML CLI raises an error shown below
Related command
az ml online-endpoint create \
-f kubernetes.yml \
--subscription ${SUBSCRIPTION} \
--resource-group ${RESOURCE_GROUP} \
--workspace-name ${WORKSPACE_NAME} \
--auth-mode key \
--debugErrors
2023-12-20T04:48:32.2079609Z Response content:
2023-12-20T04:48:32.2080089Z {
2023-12-20T04:48:32.2080533Z "error": {
2023-12-20T04:48:32.2081040Z "code": "NotFound",
2023-12-20T04:48:32.2081707Z "message": "The specified resource was not found.",
2023-12-20T04:48:32.2082278Z "details": [
2023-12-20T04:48:32.2083075Z {
2023-12-20T04:48:32.2083645Z "code": "InferencingClientCallFailed",
2023-12-20T04:48:32.2084259Z "message": "{{\"title\":\"Not Found\"}}",
2023-12-20T04:48:32.2084924Z "details": [],
2023-12-20T04:48:32.2085381Z "additionalInfo": []
2023-12-20T04:48:32.2085817Z }
2023-12-20T04:48:32.2086197Z ],
2023-12-20T04:48:32.2086617Z "additionalInfo": [
2023-12-20T04:48:32.2087061Z {
2023-12-20T04:48:32.2087492Z "type": "ComponentName",
2023-12-20T04:48:32.2087944Z "info": {
2023-12-20T04:48:32.2088403Z "value": "managementfrontend"
2023-12-20T04:48:32.2088860Z }
2023-12-20T04:48:32.2089255Z },
2023-12-20T04:48:32.2089653Z {
2023-12-20T04:48:32.2090072Z "type": "Correlation",
2023-12-20T04:48:32.2090510Z "info": {
2023-12-20T04:48:32.2090947Z "value": {
2023-12-20T04:48:32.2091491Z "operation": "22f61e***f8789bf3db42462ee64ee7b48",
2023-12-20T04:48:32.2092040Z "request": "d***633f8acdb3ae57"
2023-12-20T04:48:32.2092499Z }
2023-12-20T04:48:32.2092904Z }
2023-12-20T04:48:32.2093289Z },
2023-12-20T04:48:32.2093689Z {
2023-12-20T04:48:32.2094124Z "type": "Environment",
2023-12-20T04:48:32.2094588Z "info": {
2023-12-20T04:48:32.2095037Z "value": "australiaeast"
2023-12-20T04:48:32.2095474Z }
2023-12-20T04:48:32.2095862Z },
2023-12-20T04:48:32.2096273Z {
2023-12-20T04:48:32.2096696Z "type": "Location",
2023-12-20T04:48:32.2097140Z "info": {
2023-12-20T04:48:32.2097610Z "value": "australiaeast"
2023-12-20T04:48:32.2098055Z }
2023-12-20T04:48:32.2098461Z },
2023-12-20T04:48:32.2098856Z {
2023-12-20T04:48:32.2099288Z "type": "Time",
2023-12-20T04:48:32.2099728Z "info": {
2023-12-20T04:48:32.2100400Z "value": "2***23-12-2***T***4:48:31.7776546+***:***"
2023-12-20T04:48:32.2100888Z }
2023-12-20T04:48:32.2101295Z }
2023-12-20T04:48:32.2101696Z ]
2023-12-20T04:48:32.2102083Z }
2023-12-20T04:48:32.2102710Z }
2023-12-20T04:48:32.2900402Z Traceback (most recent call last):
2023-12-20T04:48:32.2901430Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/custom/online_endpoint.py", line 122, in ml_online_endpoint_create
2023-12-20T04:48:32.2902400Z endpoint = ml_client.begin_create_or_update(endpoint, local=local)
2023-12-20T04:48:32.2902968Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2903740Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_ml_client.py", line 1***87, in begin_create_or_update
2023-12-20T04:48:32.2904772Z return _begin_create_or_update(entity, self._operation_container.all_operations, **kwargs)
2023-12-20T04:48:32.2905368Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2907895Z File "/opt/az/lib/python3.11/functools.py", line 9***9, in wrapper
2023-12-20T04:48:32.2908539Z return dispatch(args[***].__class__)(*args, **kw)
2023-12-20T04:48:32.2909077Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2909700Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_ml_client.py", line 1175, in _
2023-12-20T04:48:32.2910450Z return operations[AzureMLResourceType.ONLINE_ENDPOINT].begin_create_or_update(entity, **kwargs)
2023-12-20T04:48:32.2911042Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2911999Z File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
2023-12-20T04:48:32.2912623Z return func(*args, **kwargs)
2023-12-20T04:48:32.2913110Z ^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2913743Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_telemetry/activity.py", line 275, in wrapper
2023-12-20T04:48:32.2914365Z return f(*args, **kwargs)
2023-12-20T04:48:32.2914837Z ^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2915538Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 256, in begin_create_or_update
2023-12-20T04:48:32.2916208Z raise ex
2023-12-20T04:48:32.2916878Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 251, in begin_create_or_update
2023-12-20T04:48:32.2917548Z raise ex
2023-12-20T04:48:32.2918232Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 24***, in begin_create_or_update
2023-12-20T04:48:32.2920295Z poller = self._online_operation.begin_create_or_update(
2023-12-20T04:48:32.2920844Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2921604Z File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
2023-12-20T04:48:32.2922234Z return func(*args, **kwargs)
2023-12-20T04:48:32.2922695Z ^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2923798Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_restclient/v2***22_***2_***1_preview/operations/_online_endpoints_operations.py", line 921, in begin_create_or_update
2023-12-20T04:48:32.2924590Z raw_result = self._create_or_update_initial(
2023-12-20T04:48:32.2925110Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2925896Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_restclient/v2***22_***2_***1_preview/operations/_online_endpoints_operations.py", line 855, in _create_or_update_initial
2023-12-20T04:48:32.2926764Z map_error(status_code=response.status_code, response=response, error_map=error_map)
2023-12-20T04:48:32.2927595Z File "/opt/az/lib/python3.11/site-packages/azure/core/exceptions.py", line 1***7, in map_error
2023-12-20T04:48:32.2928150Z raise error
2023-12-20T04:48:32.2928927Z azure.core.exceptions.ResourceNotFoundError: (NotFound) The specified resource was not found.
2023-12-20T04:48:32.2929473Z Code: NotFound
2023-12-20T04:48:32.2929979Z Message: The specified resource was not found.
2023-12-20T04:48:32.2930618Z Exception Details: (InferencingClientCallFailed) {{"title":"Not Found"}}
2023-12-20T04:48:32.2931217Z Code: InferencingClientCallFailed
2023-12-20T04:48:32.2931715Z Message: {{"title":"Not Found"}}
2023-12-20T04:48:32.2932185Z ERROR: cli: None
2023-12-20T04:48:32.2932892Z DEBUG: cli: Received HttpResponseError: Traceback (most recent call last):
2023-12-20T04:48:32.2933631Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/custom/online_endpoint.py", line 122, in ml_online_endpoint_create
2023-12-20T04:48:32.2934340Z endpoint = ml_client.begin_create_or_update(endpoint, local=local)
2023-12-20T04:48:32.2934893Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2935587Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_ml_client.py", line 1***87, in begin_create_or_update
2023-12-20T04:48:32.2936359Z return _begin_create_or_update(entity, self._operation_container.all_operations, **kwargs)
2023-12-20T04:48:32.2936969Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2937517Z File "/opt/az/lib/python3.11/functools.py", line 9***9, in wrapper
2023-12-20T04:48:32.2938126Z return dispatch(args[***].__class__)(*args, **kw)
2023-12-20T04:48:32.2938639Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2939266Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_ml_client.py", line 1175, in _
2023-12-20T04:48:32.2939993Z return operations[AzureMLResourceType.ONLINE_ENDPOINT].begin_create_or_update(entity, **kwargs)
2023-12-20T04:48:32.2940602Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2941362Z File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
2023-12-20T04:48:32.2941962Z return func(*args, **kwargs)
2023-12-20T04:48:32.2942426Z ^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2943071Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_telemetry/activity.py", line 275, in wrapper
2023-12-20T04:48:32.2943699Z return f(*args, **kwargs)
2023-12-20T04:48:32.2944164Z ^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2944871Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 256, in begin_create_or_update
2023-12-20T04:48:32.2945535Z raise ex
2023-12-20T04:48:32.2946199Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 251, in begin_create_or_update
2023-12-20T04:48:32.2946843Z raise ex
2023-12-20T04:48:32.2947508Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 24***, in begin_create_or_update
2023-12-20T04:48:32.2948270Z poller = self._online_operation.begin_create_or_update(
2023-12-20T04:48:32.2948794Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2949525Z File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
2023-12-20T04:48:32.2950123Z return func(*args, **kwargs)
2023-12-20T04:48:32.2950598Z ^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2951380Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_restclient/v2***22_***2_***1_preview/operations/_online_endpoints_operations.py", line 921, in begin_create_or_update
2023-12-20T04:48:32.2952161Z raw_result = self._create_or_update_initial(
2023-12-20T04:48:32.2952672Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.2953554Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_restclient/v2***22_***2_***1_preview/operations/_online_endpoints_operations.py", line 855, in _create_or_update_initial
2023-12-20T04:48:32.2954430Z map_error(status_code=response.status_code, response=response, error_map=error_map)
2023-12-20T04:48:32.2955255Z File "/opt/az/lib/python3.11/site-packages/azure/core/exceptions.py", line 1***7, in map_error
2023-12-20T04:48:32.2955808Z raise error
2023-12-20T04:48:32.2956475Z azure.core.exceptions.ResourceNotFoundError: (NotFound) The specified resource was not found.
2023-12-20T04:48:32.2957035Z Code: NotFound
2023-12-20T04:48:32.2957545Z Message: The specified resource was not found.
2023-12-20T04:48:32.2958147Z Exception Details: (InferencingClientCallFailed) {{"title":"Not Found"}}
2023-12-20T04:48:32.2958739Z Code: InferencingClientCallFailed
2023-12-20T04:48:32.2959233Z Message: {{"title":"Not Found"}}
2023-12-20T04:48:32.2959434Z
2023-12-20T04:48:32.3408677Z DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
2023-12-20T04:48:32.3409844Z File "/opt/az/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
2023-12-20T04:48:32.3410700Z cmd_result = self.invocation.execute(args)
2023-12-20T04:48:32.3411274Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3412104Z File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
2023-12-20T04:48:32.3412737Z raise ex
2023-12-20T04:48:32.3413596Z File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
2023-12-20T04:48:32.3414368Z results.append(self._run_job(expanded_arg, cmd_copy))
2023-12-20T04:48:32.3414983Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3415788Z File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
2023-12-20T04:48:32.3416564Z result = cmd_copy(params)
2023-12-20T04:48:32.3417032Z ^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3417732Z File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
2023-12-20T04:48:32.3418355Z return self.handler(*args, **kwargs)
2023-12-20T04:48:32.3418848Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3419559Z File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
2023-12-20T04:48:32.3420179Z return op(**command_args)
2023-12-20T04:48:32.3420641Z ^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3421267Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/custom/online_endpoint.py", line 136, in ml_online_endpoint_create
2023-12-20T04:48:32.3421986Z log_and_raise_error(err, debug, yaml_operation=yaml_operation)
2023-12-20T04:48:32.3422652Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/custom/raise_error.py", line 181, in log_and_raise_error
2023-12-20T04:48:32.3423246Z raise cli_error
2023-12-20T04:48:32.3423876Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/custom/online_endpoint.py", line 122, in ml_online_endpoint_create
2023-12-20T04:48:32.3424601Z endpoint = ml_client.begin_create_or_update(endpoint, local=local)
2023-12-20T04:48:32.3425145Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3425903Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_ml_client.py", line 1***87, in begin_create_or_update
2023-12-20T04:48:32.3426737Z return _begin_create_or_update(entity, self._operation_container.all_operations, **kwargs)
2023-12-20T04:48:32.3427313Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3427886Z File "/opt/az/lib/python3.11/functools.py", line 9***9, in wrapper
2023-12-20T04:48:32.3428474Z return dispatch(args[***].__class__)(*args, **kw)
2023-12-20T04:48:32.3429008Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3429878Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_ml_client.py", line 1175, in _
2023-12-20T04:48:32.3430645Z return operations[AzureMLResourceType.ONLINE_ENDPOINT].begin_create_or_update(entity, **kwargs)
2023-12-20T04:48:32.3431242Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3432004Z File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
2023-12-20T04:48:32.3432688Z return func(*args, **kwargs)
2023-12-20T04:48:32.3433174Z ^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3433808Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_telemetry/activity.py", line 275, in wrapper
2023-12-20T04:48:32.3434441Z return f(*args, **kwargs)
2023-12-20T04:48:32.3434890Z ^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3435606Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 256, in begin_create_or_update
2023-12-20T04:48:32.3436273Z raise ex
2023-12-20T04:48:32.3436961Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 251, in begin_create_or_update
2023-12-20T04:48:32.3437589Z raise ex
2023-12-20T04:48:32.3438281Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/operations/_online_endpoint_operations.py", line 24***, in begin_create_or_update
2023-12-20T04:48:32.3439079Z poller = self._online_operation.begin_create_or_update(
2023-12-20T04:48:32.3439607Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3440329Z File "/opt/az/lib/python3.11/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
2023-12-20T04:48:32.3440944Z return func(*args, **kwargs)
2023-12-20T04:48:32.3441408Z ^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3442206Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_restclient/v2***22_***2_***1_preview/operations/_online_endpoints_operations.py", line 921, in begin_create_or_update
2023-12-20T04:48:32.3443166Z raw_result = self._create_or_update_initial(
2023-12-20T04:48:32.3443675Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-12-20T04:48:32.3444446Z File "/opt/az/azcliextensions/ml/azext_mlv2/manual/vendored_curated_sdk/azure/ai/ml/_restclient/v2***22_***2_***1_preview/operations/_online_endpoints_operations.py", line 855, in _create_or_update_initial
2023-12-20T04:48:32.3445326Z map_error(status_code=response.status_code, response=response, error_map=error_map)
2023-12-20T04:48:32.3446136Z File "/opt/az/lib/python3.11/site-packages/azure/core/exceptions.py", line 1***7, in map_error
2023-12-20T04:48:32.3446695Z raise error
2023-12-20T04:48:32.3447271Z azure.core.exceptions.ResourceNotFoundError: (NotFound) The specified resource was not found.
2023-12-20T04:48:32.3447840Z Code: NotFound
2023-12-20T04:48:32.3448329Z Message: The specified resource was not found.
2023-12-20T04:48:32.3448935Z Exception Details: (InferencingClientCallFailed) {{"title":"Not Found"}}
2023-12-20T04:48:32.3449549Z Code: InferencingClientCallFailed
2023-12-20T04:48:32.3450058Z Message: {{"title":"Not Found"}}
2023-12-20T04:48:32.3450262Z
2023-12-20T04:48:32.3450790Z ERROR: cli.azure.cli.core.azclierror: (NotFound) The specified resource was not found.
2023-12-20T04:48:32.3451328Z Code: NotFound
2023-12-20T04:48:32.3451817Z Message: The specified resource was not found.
2023-12-20T04:48:32.3452435Z Exception Details: (InferencingClientCallFailed) {{"title":"Not Found"}}
2023-12-20T04:48:32.3453007Z Code: InferencingClientCallFailed
2023-12-20T04:48:32.3453515Z Message: {{"title":"Not Found"}}
2023-12-20T04:48:32.3454075Z ERROR: az_command_data_logger: (NotFound) The specified resource was not found.
2023-12-20T04:48:32.3454756Z Code: NotFound
2023-12-20T04:48:32.3455259Z Message: The specified resource was not found.
2023-12-20T04:48:32.3455856Z Exception Details: (InferencingClientCallFailed) {{"title":"Not Found"}}
2023-12-20T04:48:32.3456444Z Code: InferencingClientCallFailed
2023-12-20T04:48:32.3456944Z Message: {{"title":"Not Found"}}
2023-12-20T04:48:32.3457625Z DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at ***x7fc3611e***ea***>]
2023-12-20T04:48:32.3458372Z INFO: az_command_data_logger: exit code: 3
2023-12-20T04:48:32.3459096Z INFO: cli.__main__: Command ran in 13.948 seconds (init: ***.18***, invoke: 13.769)
2023-12-20T04:48:32.4143302Z INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
2023-12-20T04:48:32.4144241Z INFO: telemetry.client: Accumulated *** events. Flush the clients.
2023-12-20T04:48:32.4144906Z INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
2023-12-20T04:48:32.4149925Z INFO: telemetry.save: Save telemetry record of length 41***9 in cache
2023-12-20T04:48:32.4151054Z INFO: telemetry.main: Begin creating telemetry upload process.
2023-12-20T04:48:32.4155359Z INFO: telemetry.process: Creating upload process: "/opt/az/bin/python3 /opt/az/lib/python3.11/site-packages/azure/cli/telemetry/__init__.py /home/vsts/work/_temp/.azclitask"
2023-12-20T04:48:32.4157249Z INFO: telemetry.process: Return from creating process
2023-12-20T04:48:32.4158320Z INFO: telemetry.main: Finish creating telemetry upload process.
2023-12-20T04:48:32.9890779Z ##[error]Script failed with exit code: 3
2023-12-20T04:48:32.9910024Z [command]/usr/bin/az account clear
2023-12-20T04:48:33.6781580Z ##[section]Finishing: az_ml_endpoint_create_or_updateIssue script & Debug output
See above for detailed error message
kubernetes.yml used in above command:
$schema: https://azuremlschemas.azureedge.net/latest/kubernetesOnlineEndpoint.schema.json
name: nml-hydrocyclone-roping-qa
compute: azureml:****
auth_mode: keyExpected behavior
Create an endpoint in provided AML workspace
Environment Summary
$ az --version
azure-cli 2.55.0
core 2.55.0
telemetry 1.1.0
Extensions:
ml 2.22.0
Dependencies:
msal 1.24.0b2
azure-mgmt-resource 23.1.0b2Additional context
No response
yonzhan commented
Thank you for opening this issue, we will look into it.
microsoft-github-policy-service commented
Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.