Provisioning of elastic search only results in incorrect value of vmSizeKibana
rychuhardy opened this issue · 7 comments
Hey
I'm trying to create Elasticsearch without Kibana, Logstash and others. My setup is as in the screenshots
The result of validation is error: Deployment template validation failed: 'The provided value '' for the template parameter 'vmSizeKibana' at line '223' and column '21' is not valid. The parameter value is not part of the allowed value(s): 'Standard_A2,Standard_A3,Standard_A4,Standard_A5,Standard_A6,Standard_A7,Standard_A8,Standard_A9,Standard_A10,Standard_A11,Standard_D1,Standard_D2,Standard_D3,Standard_D4,Standard_D11,Standard_D12,Standard_D13,Standard_D14,Standard_D1_v2,Standard_D2_v2,Standard_D3_v2,Standard_D4_v2,Standard_D5_v2,Standard_D11_v2,Standard_D12_v2,Standard_D13_v2,Standard_D14_v2,Standard_D15_v2,Standard_D2_v3,Standard_D4_v3,Standard_D8_v3,Standard_D16_v3,Standard_D32_v3,Standard_D64_v3,Standard_DS1,Standard_DS2,Standard_DS3,Standard_DS4,Standard_DS11,Standard_DS12,Standard_DS13,Standard_DS14,Standard_DS1_v2,Standard_DS2_v2,Standard_DS3_v2,Standard_DS4_v2,Standard_DS5_v2,Standard_DS11_v2,Standard_DS12_v2,Standard_DS13_v2,Standard_DS14_v2,Standard_DS15_v2,Standard_D2s_v3,Standard_D4s_v3,Standard_D8s_v3,Standard_D16s_v3,Standard_D32s_v3,Standard_D64s_v3,Standard_E2_v3,Standard_E4_v3,Standard_E8_v3,Standard_E16_v3,Standard_E32_v3,Standard_E64_v3,Standard_E64i_v3,Standard_E2s_v3,Standard_E4s_v3,Standard_E8s_v3,Standard_E16s_v3,Standard_E32s_v3,Standard_E64s_v3,Standard_E64is_v3,Standard_F1,Standard_F2,Standard_F4,Standard_F8,Standard_F16,Standard_F1s,Standard_F2s,Standard_F4s,Standard_F8s,Standard_F16s,Standard_F2s_v2,Standard_F4s_v2,Standard_F8s_v2,Standard_F16s_v2,Standard_F32s_v2,Standard_F64s_v2,Standard_F72s_v2,Standard_G1,Standard_G2,Standard_G3,Standard_G4,Standard_G5,Standard_GS1,Standard_GS2,Standard_GS3,Standard_GS4,Standard_GS5,Standard_L4s,Standard_L8s,Standard_L16s,Standard_L32s,Standard_L8s_v2,Standard_L16s_v2,Standard_L32s_v2,Standard_L64s_v2,Standard_L80s_v2,Standard_M8ms,Standard_M16ms,Standard_M32ts,Standard_M32ls,Standard_M32ms,Standard_M64s,Standard_M64ls,Standard_M64ms,Standard_M128s,Standard_M128ms,Standard_M64,Standard_M64m,Standard_M128,Standard_M128m'.'.
However I don't want to have kibana vm.
This looks like a change in the way the Azure Marketplace portal sends input values through; Historically, the portal would always send through the default value, whether Kibana was chosen or not. Now, it looks like "" is sent when Kibana is not selected. This could possibly be related to the move to using a different createUI def handler in 3569e48.
Thanks for opening @rychuhardy. for now, you would be able to deploy using the ARM template and GitHub repository. This is the same underlying ARM template used by the Marketplace portal. The latest template version in the Marketplace is 7.3.1, so I would recommend targeting the 7.3.1 tagged release.
I found you can workaround it by saying "yes" for installing them, then select valid sizes, then select "no" again and then you can continue. However, I'm getting another error about vmSizeClientNodes which I'll create a separate issue for.
I got past the error, but now I'm concerned. I said "no" for kibana, but I'm seeing this:
Does that mean it ignored my selection and provisioned kibana anyway? I better not get charged for this because the template is broken.
kibana-ip is a deployment resource. If Kibana has been selected to be deployed, this deployment resource will deploy the IP resource needed for Kibana. If Kibana has not been selected to be deployed, this deployment resource is a no-op. See:
azure-marketplace/src/mainTemplate.json
Line 2030 in f92550a
Per my previous comment, the Azure Marketplace infrastructure appears to have changed behaviour, and requires further investigation. Other behaviour has also changed recently too - see
And
I’ll raise this with our contacts at Microsoft.
For now, you may want to consider deploying the template using Azure CLI or PowerShell, targeting the 7.4.0 tagged release.
It's borderline impossible to deploy this template right now if you do not want to install the whole stack.
It literally asks for every setting on kibana, logstash, etc. although they are all set not to be deployed.
I am not very experienced in writing azure rm templates but to me it looks like adding conditionals to the corresponding setting/parameter assignments should solve this.
Am I missing something?
Update: Just tried to deploy via web-ui, with a little fiddling I was able to deploy this. On cli that's a whole different story.
@torvitas Most of the settings are optional, and many apply only when a resource is selected to be deployed. For example, all settings related to Kibana are relevant and still optional only if "kibana" parameter has a value of "Yes".
An example of a deployment that deploys only Elasticsearch (single node cluster, with an external loadbalancer to be able to connect from outside Azure) is
az login
az account set --subscription "<subscription id>"
az group create --name "<rg name>" --location "<location>"
template_base_uri="https://raw.githubusercontent.com/elastic/azure-marketplace"
template_version="7.5.0"
az group deployment create --resource-group "<rg name>" --template-uri "$template_base_uri/$template_version/src/mainTemplate.json" --parameters @parameters.json
// after deployment, get the outputs
outputs=$(az group deployment show --name mainTemplate \
--resource-group "<name>" \
--query properties.outputs)where the parameters.json file is
{
"_artifactsLocation":{"value":"https://raw.githubusercontent.com/elastic/azure-marketplace/7.5.0/src/"},
"esVersion":{"value":"7.5.0"},
"esClusterName":{"value":"my-cluster"},
"loadBalancerType":{"value":"external"},
"kibana":{"value":"No"},
"vmDataNodeCount":{"value":1},
"dataNodesAreMasterEligible":{"value":"Yes"},
"adminUsername":{"value":"<admin username>"},
"authenticationType":{"value":"password"},
"adminPassword":{"value":"<admin password>"},
"securityBootstrapPassword":{"value":"<password>"},
"securityAdminPassword":{"value":"<password>"},
"securityKibanaPassword":{"value":"<password>"},
"securityLogstashPassword":{"value":"<password>"},
"securityBeatsPassword":{"value":"<password>"},
"securityApmPassword":{"value":"<password>"},
"securityRemoteMonitoringPassword":{"value":"<password>"},
}I am not very experienced in writing azure rm templates but to me it looks like adding conditionals to the corresponding setting/parameter assignments should solve this.
Am I missing something?
The template already makes heavy use of conditions to determine whether to deploy resources. Where a resource produces an output that is referenced in the top level outputs however, that output must resolve to a value for the template to be valid. For example, the deployment of Kibana results in a "kibana" output that is the public URI that Kibana can be accessed from. If Kibana is not deployed, the value of this output is "N/A". An empty kibana resource linked template must be used for the latter case, as opposed to a "condition" on the kibana resource template, because the main template references the output section of the kibana template, and an error results if there is no output section to resolve.
Closing this issue. This was indeed a bug in the Azure portal, confirmed by folks at Microsoft, which is now addressed.