Incomplete configuration after cloud-init in Azure VM
spoerli opened this issue · 3 comments
Hi Team,
I am trying to deploy fortigate on azure automatically via Terraform. I leveraged the code from the repository and added my own configuration, fitting my environment. Using a byol-template, it works fine and the system comes up without issues.
# set firewall specific parameters as variables
locals {
vm_name_prefix = "confgt${var.ENVIRONMENT_SHORT}"
admin_name ="fgtadmin"
publisher = "fortinet"
offer = "fortinet_fortigate-vm_v5"
sku_byol = "fortinet_fg-vm"
sku_payg = "fortinet_fg-vm_payg_20190624"
version = "7.0.0"
}
resource "azurerm_virtual_machine" "Connectivity_VM_FGT1" {
name ="${local.vm_name_prefix}001"
resource_group_name = azurerm_resource_group.Connectivity_RG.name
location = azurerm_resource_group.Connectivity_RG.location
network_interface_ids = [azurerm_network_interface.Connectivity_NIC_Port1_FGT1.id, azurerm_network_interface.Connectivity_NIC_Port2_FGT1.id, azurerm_network_interface.Connectivity_NIC_Port3_FGT1.id]
primary_network_interface_id = azurerm_network_interface.Connectivity_NIC_Port1_FGT1.id
vm_size = "Standard_F4s"
availability_set_id = azurerm_availability_set.Connectivity-AVAIL.id
storage_image_reference {
publisher = local.publisher
offer = local.offer
sku = var.LICENSE_TYPE == "byol" ? local.sku_byol : local.sku_payg
version = local.version
}
plan {
name = var.LICENSE_TYPE == "byol" ? local.sku_byol : local.sku_payg
publisher = local.publisher
product = local.offer
}
storage_os_disk {
name = "osdisk-${local.vm_name_prefix}001"
caching = "ReadWrite"
managed_disk_type = "StandardSSD_LRS"
create_option = "FromImage"
}
storage_data_disk {
name = "datadisk-${local.vm_name_prefix}001"
managed_disk_type = "StandardSSD_LRS"
create_option = "Empty"
lun = 0
disk_size_gb = "30"
}
os_profile {
computer_name = "${local.vm_name_prefix}001"
admin_username = local.admin_name
admin_password = data.azurerm_key_vault_secret.Connectivity_KVS_FGTSecret.value
#custom_data = base64encode(data.azurerm_key_vault_secret.Connectivity_KVS_FGTLicense.value)
custom_data = data.template_file.Connectivity_TEMPLATE_CONF_FGT1.rendered
}
os_profile_linux_config {
disable_password_authentication = false
}
boot_diagnostics {
enabled = true
storage_uri = azurerm_storage_account.Connectivity_ST.primary_blob_endpoint
}
}
However, for the testing environment I try to bring the same setup online, but using a payg-template. Unfortunately I am facing a strange issue. Wenn bringing up the VM, it comes up with an imcomplete configuration. Analysing the logs, it looked like that cloud-init is executing the script twice, so that the second execution overlaps the inital one:
Fortigate1 # diagnose debug cloudinit show
>> Checking metadata source azure
>> Azure waiting for customdata file
>> Azure waiting for customdata file
>> Azure waiting for customdata file
>> Azure customdata file found
>> Azure cloudinit decrypt successfully
>> MIME parsed config script
>> Azure customdata processed successfully
>> Run config script
>> Finish running script
>> confgttst001 $ config system probe-response
>> confgttst001 (probe-response) $ set http-probe-value OK
>> confgttst001 (probe-response) $ set mode http-probe
>> confgttst001 (probe-response) $ end
>> confgttst001 $ config system global
>> confgttst001 (global) $ set admintimeout 15
>> confgttst001 (global) $ set alias "Fortigate1"
>> confgttst001 (global) $ set allow-traffic-redirect disable
[...]
>> Fortigate1 (filters) $ edit 7
>> Fortigate1 (7) $ set category 59
>> Fortigate1 (7) $ set action block
>> Fortigate1 (7) $ next
>> Fortigate1 (filters) $ edit 8
>> Fortigate1 (8) $ sconfig system interface <-- here it seems like it is overlapping, since it starts set action, but also starts config
>> Unknown action 0
>> Fortigate1 (8) $ edit port1
>> Unknown action 0
>> Fortigate1 (8) $ set alias Public
>> command parse error before 'alias'
>> Command fail. Return code -61
>> Fortigate1 (8) $ set mode dhcp
>> command parse error before 'mode'
What could cause this behavior? Is this expected (e.g. since the config is too long) or is there a workaround?
Many thanks for the support!
Best regards
Martin
Hi Martin,
How is your config data looking like? Are you injecting also a license for the FGT or only the custom data? What if you move the config system interface earlier in the config?
Regards,
Joeri
Hi @jvhoof ,
sorry for the late response - holidays and new year :) I hope you had a good start?
I moved the config system interface to the very beginning of the config. In the end the overlapping starts at the same position, but overlaps now with a different part of the config:
>> Fortigate1 (filters) $ edit 5
>> Fortigate1 (5) $ set category 6
>> Fortigate1 (5) $ set action block
>> Fortigate1 (5) $ next
>> Fortigate1 (filters) $ edit 6
>> Fortigate1 (6) $ set category 12
>> Fortigate1 (6) $ set action block
>> Fortigate1 (6) $ next
>> Fortigate1 (filters) $ edit 7
>> Fortigate1 (7) $ set category 59
>> Fortigate1 (7) $ set action block
>> Fortigate1 (7) $ next
>> Fortigate1 (filters) $ edit 8
>> Fortigate1 (8) $ sconfig system accprofile <---- At this point it overlaps
>> Unknown action 0
>> Fortigate1 (8) $ edit "super_admin_api_access"
>> Unknown action 0
>> Fortigate1 (8) $ set authgrp read
>> command parse error before 'authgrp'
>> Command fail. Return code -61
>> Fortigate1 (8) $ set sysgrp read
as given in the repo, we use the terraform "template_file" for generating the license file. So at the end we have the following condition:
%{ if type == "byol" }
--==AZURE==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="license"
${file(license_file)}
%{ endif }
--==AZURE==--
The license will be set via variable in tf-File:
data "template_file" "Connectivity_TEMPLATE_CONF_FGT1" {
template = file("fgt1.conf")
vars = {
license_file = "license-1.lic"
type = var.LICENSE_TYPE
environment = var.ENVIRONMENT
}
}
The value is give via auto.tfvars.json:
[...]
"LICENSE_TYPE": "payg"
}
So for this scenario, no license should be attached - which seems to work, since at the end of cloudinit we see the following:
>> Load VM metadata document
>> Requesting FortiCare license: FGTAZRYQW4-QAK02
I also attached the config, we are using (config system interface is already move to the beginning). Seems like the overlaping starts at line 685.
Thank you!
Best regards
Martin
Hi @jvhoof ,
I did some further tests and moved the DNS filters to the very end of the config. The issue came up again, but not in the dnsfilter section. Instead, I faced the error in another section, but again around line 670.
Might it be possible that there is some kind of timeout or similar which might cause a script longer than ~670 Lines not to work?
Thanks
Martin