elastic/azure-marketplace

Unable to deploy elasticsearch template on Azure Marketplace

tonkatsu7 opened this issue · 18 comments

tonkatsu7 commented

Hi, I'm unable to successfully deploy the Azure marketplace template https://azuremarketplace.microsoft.com/en-au/marketplace/apps/elastic.elasticsearch with several of the VMs failing to deploy successfully....

error output 
"code": "Conflict",
  "message": "{
    "status": "Failed",
    "error": {
      "code": "ResourceDeploymentFailure",
      "message": "The resource operation completed with terminal provisioning state 'Failed'.",
      "details": [
        {
          "code": "DeploymentFailed",
          "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
          "details": [
            {
              "code": "Conflict",
              "message": "{
    "status": "Failed",
    "error": {
      "code": "ResourceDeploymentFailure",
      "message": "The resource operation completed with terminal provisioning state 'Failed'.",
      "details": [
        {
          "code": "DeploymentFailed",
          "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
          "details": [
            {
              "code": "Conflict",
              "message": "{
    "status": "Failed",
    "error": {
      "code": "ResourceDeploymentFailure",
      "message": "The resource operation completed with terminal provisioning state 'Failed'.",
      "details": [
        {
          "code": "DeploymentFailed",
          "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
          "details": [
            {
              "code": "Conflict",
              "message": "{
    "status": "Failed",
    "error": {
      "code": "ResourceDeploymentFailure",
      "message": "The resource operation completed with terminal provisioning state 'Failed'.",
      "details": [
        {
          "code": "VMExtensionProvisioningError",
          "message": "VM has reported a failure when processing extension 'script'. Error message: "Enable failed: failed to execute command: command terminated with exit status=10
          [stdout]
  [10102020-10:59:23] [apply_security_settings] could not update the built-in elastic user

  [stderr]
  E: Unable to correct problems, you have held broken packages.
  E: Unable to correct problems, you have held broken packages.
  Failed to start ntp.service: Unit ntp.service not found.
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  Synchronizing state of elasticsearch.service with SysV init with /lib/systemd/systemd-sysv-install...
  Executing /lib/systemd/systemd-sysv-install enable elasticsearch
  Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service.
  run-parts: executing /usr/share/netfilter-persistent/plugins.d/15-ip4tables save
  run-parts: executing /usr/share/netfilter-persistent/plugins.d/25-ip6tables save
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
                                     0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0100   293  100   265  100    28    882     93 --:--:-- --:--:-- --:--:--   883
  HTTP 503
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
                                     0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0100   387  100   387    0     0   3234      0 --:--:-- --:--:-- --:--:--  3252
  HTTP 401
  "More information on troubleshooting is available at https://aka.ms/VMExtensionCSELinuxTroubleshoot "
        }
      ]
    }
    }"
            }
          ]
        }
      ]
    }
    }"
            },
            {
              "code": "Conflict",
              "message": "{
    "status": "Failed",
    "error": {
      "code": "ResourceDeploymentFailure",
      "message": "The resource operation completed with terminal provisioning state 'Failed'.",
      "details": [
        {
          "code": "DeploymentFailed",
          "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
          "details": [
            {
              "code": "Conflict",
              "message": "{
    "status": "Failed",
    "error": {
      "code": "ResourceDeploymentFailure",
      "message": "The resource operation completed with terminal provisioning state 'Failed'.",
      "details": [
        {
          "code": "VMExtensionProvisioningError",
          "message": "VM has reported a failure when processing extension 'script'. Error message: "Enable failed: failed to execute command: command terminated with exit status=10
          [stdout]
  [10102020-10:59:23] [apply_security_settings] could not update the built-in elastic user

  [stderr]
  E: Unable to correct problems, you have held broken packages.
  E: Unable to correct problems, you have held broken packages.
  Failed to start ntp.service: Unit ntp.service not found.
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  sent invalidate(passwd) request, exiting
  sent invalidate(group) request, exiting
  Synchronizing state of elasticsearch.service with SysV init with /lib/systemd/systemd-sysv-install...
  Executing /lib/systemd/systemd-sysv-install enable elasticsearch
  Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service.
  run-parts: executing /usr/share/netfilter-persistent/plugins.d/15-ip4tables save
  run-parts: executing /usr/share/netfilter-persistent/plugins.d/25-ip6tables save
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current  Dload  Upload   Total   Spent    Left  Speed
         0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0100   293  100   265  100    28    540     57 --:--:-- --:--:-- --:--:--   540
  HTTP 503
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
                                     0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0100   387  100   387    0     0   1663      0 --:--:-- --:--:-- --:--:--  1660
  HTTP 401
  "More information on troubleshooting is available at https://aka.ms/VMExtensionCSELinuxTroubleshoot"
        }
      ]
    }
    }"
            }
          ]
        }
      ]
    }
    }"
            }
          ]
        }
      ]
    }
    }"
            }
          ]
        }
      ]
    }
    }"

</details>
AlexCastroAlex commented

Hello ,

I have the same error.
Error on creating the master nodes.
image

image

image

and the details :
Master details :
{
"code": "DeploymentFailed",
"message": "Au moins une opération de déploiement de ressource a échoué. Listez les opérations de déploiement pour plus de détails. Consultez https://aka.ms/DeployOperations pour les détails d'utilisation.",
"details": [{
"code": "Conflict",
"message": "{\r\n "status": "Failed",\r\n "error": {\r\n "code": "ResourceDeploymentFailure",\r\n "message": "L'opération de ressource s'est achevée avec l'état d'approvisionnement terminal « Failed ».",\r\n "details": [\r\n {\r\n "code": "DeploymentFailed",\r\n "message": "Au moins une opération de déploiement de ressource a échoué. Listez les opérations de déploiement pour plus de détails. Consultez https://aka.ms/DeployOperations pour les détails d'utilisation.",\r\n "details": [\r\n {\r\n "code": "Conflict",\r\n "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"L'opération de ressource s'est achevée avec l'état d'approvisionnement terminal « Failed ».\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"message\": \"Au moins une opération de déploiement de ressource a échoué. Listez les opérations de déploiement pour plus de détails. Consultez https://aka.ms/DeployOperations pour les détails d'utilisation.\",\r\n \"details\": [\r\n {\r\n \"code\": \"Conflict\",\r\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"L'opération de ressource s'est achevée avec l'état d'approvisionnement terminal « Failed ».\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"VMExtensionProvisioningError\\\",\\r\\n \\\"message\\\": \\\"La machine virtuelle a signalé une erreur pendant le traitement de l'extension « script ». Message d'erreur : « Enable failed: failed to execute command: command terminated with exit status=10\\\\n[stdout]\\\\n[23102020-15:46:05] [apply_security_settings] could not update the built-in elastic user\\\\n\\\\n[stderr]\\\\nE: Unable to correct problems, you have held broken packages.\\\\nE: Unable to correct problems, you have held broken packages.\\\\nFailed to start ntp.service: Unit ntp.service not found.\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(passwd) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(passwd) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(passwd) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(passwd) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(passwd) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(passwd) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(passwd) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nsent invalidate(passwd) request, exiting\\\\nsent invalidate(group) request, exiting\\\\nSynchronizing state of elasticsearch.service with SysV init with /lib/systemd/systemd-sysv-install...\\\\nExecuting /lib/systemd/systemd-sysv-install enable elasticsearch\\\\nCreated symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service.\\\\nrun-parts: executing /usr/share/netfilter-persistent/plugins.d/15-ip4tables save\\\\nrun-parts: executing /usr/share/netfilter-persistent/plugins.d/25-ip6tables save\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\n Dload Upload Total Spent Left Speed\\\\n\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\r100 31 0 0 100 31 0 151 --:--:-- --:--:-- --:--:-- 151\\\\r100 296 100 265 100 31 584 68 --:--:-- --:--:-- --:--:-- 583\\\\nHTTP 503\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\n Dload Upload Total Spent Left Speed\\\\n\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\r100 387 100 387 0 0 892 0 --:--:-- --:--:-- --:--:-- 893\\\\nHTTP 401\\\\n »\\\\r\\\\n\\\\r\\\\nPour plus d'informations sur la résolution des problèmes, accédez à https://aka.ms/VMExtensionCSELinuxTroubleshoot \\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"
}
]
}

Lower script details :

{
"code": "DeploymentFailed",
"message": "Au moins une opération de déploiement de ressource a échoué. Listez les opérations de déploiement pour plus de détails. Consultez https://aka.ms/DeployOperations pour les détails d'utilisation.",
"details": [{
"code": "Conflict",
"message": "{\r\n "status": "Failed",\r\n "error": {\r\n "code": "ResourceDeploymentFailure",\r\n "message": "L'opération de ressource s'est achevée avec l'état d'approvisionnement terminal « Failed ».",\r\n "details": [\r\n {\r\n "code": "VMExtensionProvisioningError",\r\n "message": "La machine virtuelle a signalé une erreur pendant le traitement de l'extension « script ». Message d'erreur : « Enable failed: failed to execute command: command terminated with exit status=10\n[stdout]\n[23102020-15:46:05] [apply_security_settings] could not update the built-in elastic user\n\n[stderr]\nE: Unable to correct problems, you have held broken packages.\nE: Unable to correct problems, you have held broken packages.\nFailed to start ntp.service: Unit ntp.service not found.\nsent invalidate(group) request, exiting\nsent invalidate(passwd) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(passwd) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(passwd) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(passwd) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(passwd) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(passwd) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(passwd) request, exiting\nsent invalidate(group) request, exiting\nsent invalidate(passwd) request, exiting\nsent invalidate(group) request, exiting\nSynchronizing state of elasticsearch.service with SysV init with /lib/systemd/systemd-sysv-install...\nExecuting /lib/systemd/systemd-sysv-install enable elasticsearch\nCreated symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service.\nrun-parts: executing /usr/share/netfilter-persistent/plugins.d/15-ip4tables save\nrun-parts: executing /usr/share/netfilter-persistent/plugins.d/25-ip6tables save\n % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 31 0 0 100 31 0 151 --:--:-- --:--:-- --:--:-- 151\r100 296 100 265 100 31 584 68 --:--:-- --:--:-- --:--:-- 583\nHTTP 503\n % Total % Received % Xferd Average Speed Time Time Time Current\n Dload Upload Total Spent Left Speed\n\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\r100 387 100 387 0 0 892 0 --:--:-- --:--:-- --:--:-- 893\nHTTP 401\n »\r\n\r\nPour plus d'informations sur la résolution des problèmes, accédez à https://aka.ms/VMExtensionCSELinuxTroubleshoot "\r\n }\r\n ]\r\n }\r\n}"
}
]
}

Anybody able to help for this deployment please ?

Regards

russcam commented

Hi @tonkatsu7 / @AlexCastroAlex,

The underlying issue is

could not update the built-in elastic user

When the template deployment cannot update the built-in elastic user, the deployment fails, rather than finishing as successful but leaving the cluster in a state where not all setup has been performed successfully. There are a number of reasons why this can happen, some of which can be intermittent, so I would recommend trying to deploy again. Please report back if you see this issue on every attempted deployment.

mdrakiburrahman commented

I ran into the same issue about 8 times (trying different versions on the marketplace with different permutations). Also tried to deploy using the exact ARM template available here - without changing the passwords/config even, same error every time:

image

Found the root cause in this repo here: azure-marketplace/src/scripts/elasticsearch-install.sh
image

Cloned the repo, commented out the exit 10, and deployed from powershell.

Works fine, able to access Kibana now with the password passed in through the ARM template.

Seems like a post-deployment check to check for the elastic account password got updated (which it did - not sure if the exit 10 is needed?)

russcam commented

Cloned the repo, commented out the exit 10, and deployed from powershell.

Works fine, able to access Kibana now with the password passed in through the ARM template.

Seems like a post-deployment check to check for the elastic account password got updated (which it did - not sure if the exit 10 is needed?)

The check is needed as if this node fails to update the elastic user password, it should check to ensure that another node has successfully updated the password, and fail if the call fails.

One thing that could happen though, is that the cluster state may not be ready when performing the check, if another node has just updated the password, so the check could fail. To remedy, the node should wait for the health of security index to be green before checking. I've opened #372 to address.

tomconte commented

Hi, I just tried the new branch:

template_base_uri=https://raw.githubusercontent.com/elastic/azure-marketplace
#template_version="7.9.0"
template_version="wait-for-green-elastic-password-check"
es_version="7.9.0"

az group deployment create \
    --resource-group "$rg_name" \
    --template-uri $template_base_uri/$template_version/src/mainTemplate.json \
    --parameters _artifactsLocation=$template_base_uri/$template_version/src/ \
                 esVersion=$es_version esClusterName=elasticsearch \
                 vmDataDiskCount=3 vmSizeDataNodes="Standard_DS3_v2" vmDataNodeCount=3 \
                 adminUsername=esadmin adminPassword="admin${rand_string}+" \
                 securityBootstrapPassword="bootstrap${rand_string}" \
                 securityAdminPassword="admin${rand_string}" \
                 securityKibanaPassword="kibana${rand_string}" \
                 securityLogstashPassword="logstash${rand_string}" \
                 securityBeatsPassword="beats${rand_string}" \
                 securityApmPassword="apm${rand_string}" \
                 securityRemoteMonitoringPassword="remoteMonitoring${rand_string}"

And got this error:

{"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.","details":[{"code":"VMExtensionProvisioningError","message":"VM has reported a failure when processing extension 'script'. Error message: \"Enable failed: failed to execute command: command terminated with exit status=10
[stdout]
[02122020-08:51:59] [apply_security_settings] could not update the built-in elastic user

[stderr]
sent invalidate(group) request, exiting
sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting
sent invalidate(group) request, exiting
sent invalidate(group) request, exiting
sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting
sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting
sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting
sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting
sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting
sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting
sent invalidate(passwd) request, exiting
sent invalidate(group) request, exiting
Synchronizing state of elasticsearch.service with SysV init with /lib/systemd/systemd-sysv-install...
Executing /lib/systemd/systemd-sysv-install enable elasticsearch
Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service.
run-parts: executing /usr/share/netfilter-persistent/plugins.d/15-ip4tables save
run-parts: executing /usr/share/netfilter-persistent/plugins.d/25-ip6tables save
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 293 100 265 100 28 512 54 --:--:-- --:--:-- --:--:-- 513
HTTP 503
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 541 100 541 0 0 2357 0 --:--:-- --:--:-- --:--:-- 2352
elasticsearch-install.sh: line 541: {\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"unable: command not found
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 387 100 387 0 0 3296 0 --:--:-- --:--:-- --:--:-- 3307
HTTP 401
\"

Looking at https://github.com/elastic/azure-marketplace/blob/master/src/scripts/elasticsearch-install.sh#L541, shouldn't it use echo $response... ?

tomconte commented

Update: I have made some modifications to the script and was able to deploy... You can see my changes here:

wait-for-green-elastic-password-check...tomconte:fix-wait-for-green

Basically I added a retry loop for the 401 case... Also added echo commands for the head/tail split. I bumped the timeout to 15 minutes but I am not sure that was required.

russcam commented

Hey @tomconte, it's been a while 😃

shouldn't it use echo $response... ?

Yes it should 😞 Would you mind opening a PR for this change please?

Basically I added a retry loop for the 401 case... Also added echo commands for the head/tail split. I bumped the timeout to 15 minutes but I am not sure that was required.

The wait_for_status=green and timeout=5m will wait up to 5 minutes for the .security index to be green. It'll return as soon as it's green, or report the status after the timeout. With this setup, I don't think we need to have the retries, and the timeout of 5 minutes should be more than enough time for a green status; it should be green in the order of seconds.

I opened #372 to address an issue where a node should wait for the security index to be green after the request to update the elastic built-in user password fails (i.e. another node has already updated it), and before checking that the node can send a request with the updated password. That might be the issue happening here too, and I think with the echo bug fix in addition, I think this should be resolved.

tomconte commented

Hey @russcam, long time indeed, hope you are well!

I will open a PR to add the echo commands, but looking at the logs it seemed to me that the nodes were running the CURL command and immediately receiving a 401 error instead of waiting for the status to turn green... Could there be some kind of race condition going on there?

russcam commented

looking at the logs it seemed to me that the nodes were running the CURL command and immediately receiving a 401 error instead of waiting for the status to turn green... Could there be some kind of race condition going on there?

That does indeed sound like a race condition:

  1. node A updates elastic user password using the bootstrap password to authenticate
  2. node B tries to update the elastic user password using the bootstrap password, but fails, Expected failures are that
    1. node A change is already reflected and a 401 response is returned
    2. security index is not yet green, and an unavailable_shards_exception 503 response is returned
  3. node B tries to ping the cluster the cluster with the updated elastic user password

It looks like the expected failure in 2.2 would fail immediately with a 401 response because the security index isn't yet ready, meaning the cluster expects the bootstrap password to authenticate. But, at the point the security index is ready, the cluster would return a 401 response when using the bootstrap password and return the status when using the updated password.

In summary, I think you are correct in adding retries here. I don't think we need to wait for the security index to be green before trying step 3, just need to retry the pinging. Considering timezone differences, I'll open a PR to address this now.

russcam commented

I've opened #374.

Would you mind testing this, @tomconte, @mdrakiburrahman? I've run testing on it, deploying a 6 node cluster. Checking the /var/log/arm-install.log file on each node, I see the following similar output on each node:

[03122020-02:49:02] Begin execution of Elasticsearch script extension on data-3
[03122020-02:49:02] data-3 not found in /etc/hosts
[03122020-02:49:02] hostname data-3 added to /etc/hosts
[03122020-02:49:02] Option l set
[03122020-02:49:02] Option n set
[03122020-02:49:02] Option v set
[03122020-02:49:02] Option m set
[03122020-02:49:02] Option A set
[03122020-02:49:02] Option R set
[03122020-02:49:02] Option K set
[03122020-02:49:02] Option S set
[03122020-02:49:02] Option F set
[03122020-02:49:02] Option M set
[03122020-02:49:02] Option B set
[03122020-02:49:02] Option Z set
[03122020-02:49:02] Option p set
[03122020-02:49:02] Option a set
[03122020-02:49:02] Option k set
[03122020-02:49:02] Option E set
[03122020-02:49:02] Option L set
[03122020-02:49:02] Option C set
[03122020-02:49:02] Option D set
[03122020-02:49:02] Option H set
[03122020-02:49:02] Option G set
[03122020-02:49:02] Option V set
[03122020-02:49:02] Option J set
[03122020-02:49:02] Option T set
[03122020-02:49:02] Option W set
[03122020-02:49:02] Option N set
[03122020-02:49:02] Option O set
[03122020-02:49:02] Option P set
[03122020-02:49:02] using bootstrap password as the seed password
[03122020-02:49:02] bootstrapping an Elasticsearch 7.9.0 cluster named 'my-azure-cluster' with minimum_master_nodes set to 4
[03122020-02:49:02] cluster uses dedicated master nodes is set to 0 and unicast goes to ["data-0","data-1","data-2","data-3","data-4","data-5"]
[03122020-02:49:02] cluster install X-Pack plugin is set to 1
[03122020-02:49:02] cluster basic security is set to 1
[03122020-02:49:02] [format_data_disks] checking node role
[03122020-02:49:02] [format_data_disks] data node, data disks may be attached
[03122020-02:49:02] [format_data_disks] starting partition and format attached disks
[03122020-02:49:02] [format_and_partition_disks] Option s set with value
[03122020-02:49:03] [format_and_partition_disks] installing or updating mdadm
[03122020-02:49:10] [format_and_partition_disks] apt-get updated installing mdadm now
[03122020-02:49:10] [format_and_partition_disks] apt-get installed mdadm and can be found returns: 0
[03122020-02:49:10] [format_and_partition_disks] Begin creating striped volume
[03122020-02:49:10] [format_and_partition_disks] Disks are /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn /dev/sdo /dev/sdp /dev/sdq /dev/sdr
[03122020-02:49:10] [format_and_partition_disks] Working on /dev/sdc
[03122020-02:49:11] [format_and_partition_disks] /dev/sdc is not partitioned, partitioning
[03122020-02:49:11] [format_and_partition_disks] create partition for /dev/sdc with fdisk
[03122020-02:49:13] [format_and_partition_disks] Working on /dev/sdd
[03122020-02:49:13] [format_and_partition_disks] /dev/sdd is not partitioned, partitioning
[03122020-02:49:13] [format_and_partition_disks] create partition for /dev/sdd with fdisk
[03122020-02:49:13] [format_and_partition_disks] Working on /dev/sde
[03122020-02:49:14] [format_and_partition_disks] /dev/sde is not partitioned, partitioning
[03122020-02:49:14] [format_and_partition_disks] create partition for /dev/sde with fdisk
[03122020-02:49:14] [format_and_partition_disks] Working on /dev/sdf
[03122020-02:49:14] [format_and_partition_disks] /dev/sdf is not partitioned, partitioning
[03122020-02:49:14] [format_and_partition_disks] create partition for /dev/sdf with fdisk
[03122020-02:49:14] [format_and_partition_disks] Working on /dev/sdg
[03122020-02:49:14] [format_and_partition_disks] /dev/sdg is not partitioned, partitioning
[03122020-02:49:14] [format_and_partition_disks] create partition for /dev/sdg with fdisk
[03122020-02:49:14] [format_and_partition_disks] Working on /dev/sdh
[03122020-02:49:14] [format_and_partition_disks] /dev/sdh is not partitioned, partitioning
[03122020-02:49:14] [format_and_partition_disks] create partition for /dev/sdh with fdisk
[03122020-02:49:15] [format_and_partition_disks] Working on /dev/sdi
[03122020-02:49:15] [format_and_partition_disks] /dev/sdi is not partitioned, partitioning
[03122020-02:49:15] [format_and_partition_disks] create partition for /dev/sdi with fdisk
[03122020-02:49:15] [format_and_partition_disks] Working on /dev/sdj
[03122020-02:49:15] [format_and_partition_disks] /dev/sdj is not partitioned, partitioning
[03122020-02:49:15] [format_and_partition_disks] create partition for /dev/sdj with fdisk
[03122020-02:49:15] [format_and_partition_disks] Working on /dev/sdk
[03122020-02:49:15] [format_and_partition_disks] /dev/sdk is not partitioned, partitioning
[03122020-02:49:15] [format_and_partition_disks] create partition for /dev/sdk with fdisk
[03122020-02:49:15] [format_and_partition_disks] Working on /dev/sdl
[03122020-02:49:15] [format_and_partition_disks] /dev/sdl is not partitioned, partitioning
[03122020-02:49:15] [format_and_partition_disks] create partition for /dev/sdl with fdisk
[03122020-02:49:15] [format_and_partition_disks] Working on /dev/sdm
[03122020-02:49:15] [format_and_partition_disks] /dev/sdm is not partitioned, partitioning
[03122020-02:49:15] [format_and_partition_disks] create partition for /dev/sdm with fdisk
[03122020-02:49:15] [format_and_partition_disks] Working on /dev/sdn
[03122020-02:49:15] [format_and_partition_disks] /dev/sdn is not partitioned, partitioning
[03122020-02:49:15] [format_and_partition_disks] create partition for /dev/sdn with fdisk
[03122020-02:49:15] [format_and_partition_disks] Working on /dev/sdo
[03122020-02:49:16] [format_and_partition_disks] /dev/sdo is not partitioned, partitioning
[03122020-02:49:16] [format_and_partition_disks] create partition for /dev/sdo with fdisk
[03122020-02:49:16] [format_and_partition_disks] Working on /dev/sdp
[03122020-02:49:16] [format_and_partition_disks] /dev/sdp is not partitioned, partitioning
[03122020-02:49:16] [format_and_partition_disks] create partition for /dev/sdp with fdisk
[03122020-02:49:16] [format_and_partition_disks] Working on /dev/sdq
[03122020-02:49:16] [format_and_partition_disks] /dev/sdq is not partitioned, partitioning
[03122020-02:49:16] [format_and_partition_disks] create partition for /dev/sdq with fdisk
[03122020-02:49:16] [format_and_partition_disks] Working on /dev/sdr
[03122020-02:49:16] [format_and_partition_disks] /dev/sdr is not partitioned, partitioning
[03122020-02:49:16] [format_and_partition_disks] create partition for /dev/sdr with fdisk
[03122020-02:49:16] [format_and_partition_disks] Using 16 partitions /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 /dev/sdi1 /dev/sdj1 /dev/sdk1 /dev/sdl1 /dev/sdm1 /dev/sdn1 /dev/sdo1 /dev/sdp1 /dev/sdq1 /dev/sdr1
[03122020-02:49:16] [format_and_partition_disks] Next mount point appears to be /datadisks/disk1
[03122020-02:49:16] [format_and_partition_disks] 16 disks are attached. RAID0-ing them using mdadm
[03122020-02:49:16] [format_and_partition_disks] Next md device is /dev/md0
[03122020-02:49:19] [format_and_partition_disks] attempting to get UUID from /dev/md0
[03122020-02:49:19] [format_and_partition_disks] adding UUID: 2d9565ed-0dd7-4ea8-9085-cad3d1154c50 to fstab /dev/md0
[03122020-02:49:19] [format_and_partition_disks] calling fstab with UUID: 2d9565ed-0dd7-4ea8-9085-cad3d1154c50 and mount point: /datadisks/disk1
[03122020-02:49:19] [format_and_partition_disks] add entry to  /etc/mdadm/mdadm.conf for RAID array
[03122020-02:49:19] [format_and_partition_disks] update update-initramfs
[03122020-02:49:28] [format_data_disks] finished partition and format attached disks
[03122020-02:49:28] [apt-get] updating apt-get
[03122020-02:49:30] [apt-get] updated apt-get
[03122020-02:49:30] [install_ntp] installing ntp
[03122020-02:49:38] [install_ntp] installed ntp
[03122020-02:49:38] [install_ntpdate] installing ntpdate
[03122020-02:49:40] [install_ntpdate] installed ntpdate
[03122020-02:49:47] [install_java] Option v set with value 7.9.0
[03122020-02:49:47] [install_java] updating apt-get
[03122020-02:49:49] [install_java] updated apt-get
[03122020-02:49:49] [install_java] not installing java, using JDK bundled with distribution
[03122020-02:49:49] [install_es] installing Elasticsearch 7.9.0
[03122020-02:49:49] [install_es] download location - https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.9.0-amd64.deb?ultron=msft&gambit=azure
[03122020-02:49:54] [install_es] downloaded Elasticsearch 7.9.0
[03122020-02:50:07] [install_es] installed Elasticsearch 7.9.0
[03122020-02:50:07] [setup_data_disk] configuring disk /datadisks/disk1/elasticsearch/data
[03122020-02:50:07] [setup_bootstrap_password] adding bootstrap.password to keystore
[03122020-02:50:08] [setup_bootstrap_password] added bootstrap.password to keystore
[03122020-02:50:08] [configure_elasticsearch_yaml] configure elasticsearch.yml file
[03122020-02:50:08] [configure_elasticsearch_yaml] update configuration with data path list of /datadisks/disk1/elasticsearch/data
[03122020-02:50:08] [configure_elasticsearch_yaml] update configuration with discovery.seed_hosts and cluster.initial_master_nodes set to ["data-0","data-1","data-2","data-3","data-4","data-5"]
[03122020-02:50:08] [configure_elasticsearch_yaml] configure master/client/data node type flags only master-0 only data-0
[03122020-02:50:08] [configure_elasticsearch_yaml] configure node as master and data
[03122020-02:50:08] [install_jq] installing jq
[03122020-02:50:10] [install_jq] installed jq
[03122020-02:50:10] [configure_awareness_attributes] configure fault and update domain attributes
[03122020-02:50:10] [configure_awareness_attributes] configure shard allocation awareness using fault_domain and update_domain
[03122020-02:50:10] [configure_elasticsearch_yaml] Set generated license type to trial
[03122020-02:50:10] [configure_elasticsearch_yaml] Set X-Pack Security enabled
[03122020-02:50:10] [configure_elasticsearch_yaml] setting bootstrap.memory_lock: true
[03122020-02:50:10] [configure_elasticsearch] configuring elasticsearch default configuration
[03122020-02:50:10] [configure_elasticsearch] configuring heap size from available memory
[03122020-02:50:10] [configure_elasticsearch] configure elasticsearch heap size - 7013 megabytes
[03122020-02:50:10] [configure_os_properties] configuring operating system level configuration
[03122020-02:50:10] [configure_os_properties] configure systemd to start Elasticsearch service automatically when system boots
[03122020-02:50:11] [configure_os_properties] configured operating system level configuration
[03122020-02:50:11] [port_forward] setting up port forwarding from 9201 to 9200
[03122020-02:50:11] [port_forward] installing iptables-persistent
[03122020-02:50:15] [port_forward] installed iptables-persistent
[03122020-02:50:15] [port_forward] port forwarding configured
[03122020-02:50:15] [start_systemd] starting Elasticsearch
[03122020-02:50:28] [start_systemd] started Elasticsearch
[03122020-02:50:28] [wait_for_started] node is up!
[03122020-02:50:29] [apply_security_settings] start updating roles and users
[03122020-02:50:35] [wait_for_green_security_index] security index is green
[03122020-02:50:35] [apply_security_settings] updated built-in elastic superuser password
[03122020-02:50:35] [wait_for_green_security_index] security index is green
[03122020-02:50:35] [apply_security_settings] updated built-in kibana_system user password
[03122020-02:50:35] [apply_security_settings] updated built-in logstash_system user password
[03122020-02:50:36] [apply_security_settings] updated built-in beats_system user password
[03122020-02:50:36] [apply_security_settings] updated built-in apm_system user password
[03122020-02:50:36] [apply_security_settings] updated built-in remote_monitoring_user user password
[03122020-02:50:36] End execution of Elasticsearch script extension on data-3 in 0h:1m:34s

I haven't seen the issue reported here in test runs before the last release and now, so I'm keen to understand if #374 fixes the issue for someone that is seeing it. Hopefully the additional log provides a better picture of events.

  • 👍1
tomconte commented

Just gave the PR branch a spin and it was able to deploy my 6-node cluster without issues 🎉

Looking at the logs, some of the nodes did need to retry the curl a few times:

[03122020-08:13:08] [apply_security_settings] start updating roles and users
[03122020-08:13:08] [wait_for_green_security_index] status code 401 waiting for security index to be green
[03122020-08:13:18] [wait_for_green_security_index] status code 401 waiting for security index to be green
[03122020-08:13:29] [wait_for_green_security_index] status code 401 waiting for security index to be green
[03122020-08:13:39] [wait_for_green_security_index] security index is green
[03122020-08:13:39] [apply_security_settings] updated built-in elastic superuser password
[03122020-08:13:39] [wait_for_green_security_index] security index is green
[03122020-08:13:39] [apply_security_settings] updated built-in kibana_system user password
[03122020-08:13:39] [apply_security_settings] updated built-in logstash_system user password
[03122020-08:13:39] [apply_security_settings] updated built-in beats_system user password
[03122020-08:13:39] [apply_security_settings] updated built-in apm_system user password
[03122020-08:13:39] [apply_security_settings] updated built-in remote_monitoring_user user password

Thanks @russcam for the quick fix!

  • 👍1
russcam commented

Great to hear, @tomconte! We'll get a new version published to the Azure Marketplace shortly

tmatup commented

@russcam what's the ETA for publishing a new version to Azure Marketplace? I am running into the same issue with v7.9.0, tried multiple times.

russcam commented

@tmatup I don't have an ETA for when the new version goes live on the Azure Marketplace. It's been submitted to Microsoft for approval, and I'm working with them to get it approved.

  • 👍1
russcam commented

@tmatup The new version has now gone live on the Azure Marketplace.

AdelKhaled commented

Hi, I am hitting this bug while deploying a new self-managed elastic search cluster. Error details:

"statusMessage": {
"status": "Failed",
"error": {
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "Conflict",
"message": "{\r\n "status": "Failed",\r\n "error": {\r\n "code": "ResourceDeploymentFailure",\r\n "message": "The resource operation completed with terminal provisioning state 'Failed'.",\r\n "details": [\r\n {\r\n "code": "DeploymentFailed",\r\n "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",\r\n "details": [\r\n {\r\n "code": "Conflict",\r\n "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource operation completed with terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"Conflict\",\r\n \"message\": \"{\\r\\n \\\"status\\\": \\\"Failed\\\",\\r\\n \\\"error\\\": {\\r\\n \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n \\\"message\\\": \\\"The resource operation completed with terminal provisioning state 'Failed'.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.\\\",\\r\\n \\\"details\\\": [\\r\\n {\\r\\n \\\"code\\\": \\\"Conflict\\\",\\r\\n \\\"message\\\": \\\"{\\\\r\\\\n \\\\\\\"status\\\\\\\": \\\\\\\"Failed\\\\\\\",\\\\r\\\\n \\\\\\\"error\\\\\\\": {\\\\r\\\\n \\\\\\\"code\\\\\\\": \\\\\\\"ResourceDeploymentFailure\\\\\\\",\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"The resource operation completed with terminal provisioning state 'Failed'.\\\\\\\",\\\\r\\\\n \\\\\\\"details\\\\\\\": [\\\\r\\\\n {\\\\r\\\\n \\\\\\\"code\\\\\\\": \\\\\\\"VMExtensionProvisioningError\\\\\\\",\\\\r\\\\n \\\\\\\"message\\\\\\\": \\\\\\\"VM has reported a failure when processing extension 'script'. Error message: \\\\\\\\\\\\\\\"Enable failed: failed to execute command: command terminated with exit status=10\\\\\\\\n[stdout]\\\\\\\\n[26032021-10:37:50] [apply_security_settings] could not update the built-in elastic user\\\\\\\\n\\\\\\\\n[stderr]\\\\\\\\n- --:--:-- 1578\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4658 0 --:--:-- --:--:-- --:--:-- 4641\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4846 0 --:--:-- --:--:-- --:--:-- 4891\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4441 0 --:--:-- --:--:-- --:--:-- 4450\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4741 0 --:--:-- --:--:-- --:--:-- 4763\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4688 0 --:--:-- --:--:-- --:--:-- 4721\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4113 0 --:--:-- --:--:-- --:--:-- 4145\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4816 0 --:--:-- --:--:-- --:--:-- 4848\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4720 0 --:--:-- --:--:-- --:--:-- 4721\\\\\\\\r100 543 100 543 0 0 4715 0 --:--:-- --:--:-- --:--:-- 4681\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4779 0 --:--:-- --:--:-- --:--:-- 4805\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4311 0 --:--:-- --:--:-- --:--:-- 4344\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 543 100 543 0 0 4976 0 --:--:-- --:--:-- --:--:-- 4981\\\\\\\\nHTTP 401\\\\\\\\n % Total % Received % Xferd Average Speed Time Time Time Current\\\\\\\\n Dload Upload Total Spent Left Speed\\\\\\\\n\\\\\\\\r 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0\\\\\\\\r100 387 100 387 0 0 3175 0 --:--:-- --:--:-- --:--:-- 3198\\\\\\\\nHTTP 401\\\\\\\\n\\\\\\\\\\\\\\\"\\\\\\\\r\\\\\\\\n\\\\\\\\r\\\\\\\\nMore information on troubleshooting is available at https://aka.ms/VMExtensionCSELinuxTroubleshoot \\\\\\\"\\\\r\\\\n }\\\\r\\\\n ]\\\\r\\\\n }\\\\r\\\\n}\\\"\\r\\n }\\r\\n ]\\r\\n }\\r\\n ]\\r\\n }\\r\\n}\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}"
}
]
}
}

  • 👍2
ColeSiegelTR commented

+1 still getting error code 10 when attempting to install extension using https://raw.githubusercontent.com/elastic/azure-marketplace/master/src/scripts/elasticsearch-install.sh on master node using version 7.11.1

The command executed by VM and result is below

bash elasticsearch-install.sh -xdn 'elasticsearch' -v '7.11.1' -m 0 -A 'Abcdefg1234567!' -R 'Abcdefg1234567!' -K 'Abcdefg1234567!' -S 'Abcdefg1234567!' -F 'Abcdefg1234567!' -M 'Abcdefg1234567!' -B 'Abcdefg1234567!' -Z 3 -p 'cluster1' -L '' -C '' -D 'XX.XXX.XXX.X' -H '' -G '' -V '' -J '' -T '' -W '' -N '' -O '' -P ''

{
  "code": "DeploymentFailed",
  "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
  "details": [
    {
      "code": "VMExtensionProvisioningError",
      "message": "
VM has reported a failure when processing extension 'script'. Error message: 
\"Enable failed: failed to execute command: command terminated with exit status=10
[stdout]
[04052021-20:51:32] [wait_for_started] seeing if node is up after sleeping 5 seconds, retry 60/60
[04052021-20:51:33] [wait_for_started] never saw elasticsearch go up locally

[stderr]
  0     0    0     0    0     0 --:--:--  0:00:14 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:16 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:17 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:18 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:19 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:20 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:21 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:22 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:23 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:24 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:25 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:26 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:27 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:28 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:29 --:--:--     0
100   151  100   151    0     0      5      0  0:00:30  0:00:30 --:--:--    31
100   151  100   151    0     0      5      0  0:00:30  0:00:30 --:--:--    39
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:04 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:05 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:06 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:07 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:08 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:10 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:11 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:12 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:13 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:14 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:15 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:16 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:17 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:18 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:19 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:20 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:21 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:22 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:23 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:24 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:25 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:26 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:27 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:28 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:29 --:--:--     0
100   151  100   151    0     0      5      0  0:00:30  0:00:30 --:--:--    31
100   151  100   151    0     0      5      0  0:00:30  0:00:30 --:--:--    39
\"



More information on troubleshooting is available at https://aka.ms/VMExtensionCSELinuxTroubleshoot "
    }
  ]
}
mikepetridisz commented

+1, same issue