Fixed: Problems with user defined types caused json serialization errors
PolarbearChimney opened this issue · 9 comments
What happened? Provide a clear and concise description of the bug, including deployment details.
I have a new GitHub repository where we have used the ALZ-Bicep Accelerator to configure and set up the repo.
The initial deployment went fine without any issues, and I can re-run the github actions from Main branch without any issues.
The problem is when we are doing a Pull Request. For example, when I remove an extra line-spacing in config\custom-parameters\managementGroups.parameters.all.json to trigger alz-bicep-1-core.yml, we get the following error when it runs "Logging and Sentinel Resource Group Deployment":
New-AzDeployment: /home/runner/work/github-repo/github-repo/pipeline-scripts/Deploy-ALZLoggingAndSentinelResourceGroup.ps1:30 Line | 30 | New-AzSubscriptionDeployment @inputObject | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | Additional content found in JSON reference object. A JSON reference | object should only have a $ref property. Path | 'parResourceLockConfig.defaultValue'. Error: Error: The process '/usr/bin/pwsh' failed with exit code 1
I first notices the problem when I was trying to create my own module.
I copied one of the ps1 files and yml file, reconfigured them to point to my own custom bicep module with my own custom parameter file based on the orchestration hubPeeredSpoke, I got the same error:
New-AzManagementGroupDeployment: /home/runner/work/github-repo/github-repo/pipeline-scripts/Deploy-LandingZones.ps1:29 Line | 29 | New-AzManagementGroupDeployment @inputObject | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | Additional content found in JSON reference object. A JSON reference | object should only have a $ref property. Path | 'parResourceGroupLock.defaultValue'. Error: Error: The process '/usr/bin/pwsh' failed with exit code 1
And I cant seem to figure out that the relation is with "parResourceGroupLock.defaultValue".
Here is some steps I took to troubleshoot:
- Run ALZ_Bicep_1_Core on Main branch (This works)
- Create a new branch based on main where everything is default and freshly configured
- Run ALZ_Bicep_1_Core on the new branch (then I get the same error regarding parResourceGroupLock.defaultValue)
I dont know what to do now to move on
Please provide the correlation id associated with your error or bug.
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
What was the expected outcome?
No response
Relevant log output
VERBOSE: Using Bicep v0.26.54
VERBOSE: Calling Bicep with arguments: build "/home/runner/work/github-repo/github-repo/upstream-releases/v0.17.2/infra-as-code/bicep/modules/resourceGroup/resourceGroup.bicep" --stdout
WARNING: /home/runner/work/github-repo/github-repo/upstream-releases/v0.17.2/infra-as-code/bicep/modules/resourceGroup/resourceGroupLock.bicep(1,1) : Info Bicep Linter Configuration: Custom bicepconfig.json file found (/home/runner/work/github-repo/github-repo/upstream-releases/v0.17.2/infra-as-code/bicep/bicepconfig.json).
/home/runner/work/github-repo/github-repo/upstream-releases/v0.17.2/infra-as-code/bicep/CRML/customerUsageAttribution/cuaIdSubscription.bicep(1,1) : Info Bicep Linter Configuration: Custom bicepconfig.json file found (/home/runner/work/github-repo/github-repo/upstream-releases/v0.17.2/infra-as-code/bicep/bicepconfig.json).
/home/runner/work/github-repo/github-repo/upstream-releases/v0.17.2/infra-as-code/bicep/modules/resourceGroup/resourceGroup.bicep(1,1) : Info Bicep Linter Configuration: Custom bicepconfig.json file found (/home/runner/work/github-repo/github-repo/upstream-releases/v0.17.2/infra-as-code/bicep/bicepconfig.json).
Getting the latest status of all resources...
New-AzDeployment: /home/runner/work/github-repo/github-repo/pipeline-scripts/Deploy-ALZLoggingAndSentinelResourceGroup.ps1:30
Line |
30 | New-AzSubscriptionDeployment @inputObject
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Additional content found in JSON reference object. A JSON reference
| object should only have a $ref property. Path
| 'parResourceLockConfig.defaultValue'.
Error: Error: The process '/usr/bin/pwsh' failed with exit code 1Check previous GitHub issues
- I have searched the issues for this item and found no duplicate
Code of Conduct
- I agree to follow this project's Code of Conduct
So further troubleshooting:
When I remove all parts related to locks in
upstream-releases\v0.17.2\infra-as-code\bicep\modules\resourceGroup\resourceGroup.bicep
And
config\custom-parameters\resourceGroupLoggingAndSentinel.parameters.all.json
The workflow moves on, and failes on the next step instead: "Logging and Sentinel Deployment" with the same error:
New-AzResourceGroupDeployment: /home/runner/work/github-repo/github-repo/pipeline-scripts/Deploy-ALZLoggingAndSentinel.ps1:30
Line |
30 | New-AzResourceGroupDeployment @inputObject
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Additional content found in JSON reference object. A JSON reference
| object should only have a $ref property. Path
| 'parGlobalResourceLock.defaultValue'.
Error: Error: The process '/usr/bin/pwsh' failed with exit code 1So I go to 'upstream-releases\v0.17.2\infra-as-code\bicep\modules\logging\logging.bicep'
and remove all traces of "parGlobalResourceLock", but keeping all the other locks.
This is logging.bicep at this point:
metadata name = 'ALZ Bicep - Logging Module'
metadata description = 'ALZ Bicep Module used to set up Logging'
type lockType = {
@description('Optional. Specify the name of lock.')
name: string?
@description('Optional. The lock settings of the service.')
kind: ('CanNotDelete' | 'ReadOnly' | 'None')
@description('Optional. Notes about this lock.')
notes: string?
}
@sys.description('Log Analytics Workspace name.')
param parLogAnalyticsWorkspaceName string = 'alz-log-analytics'
@sys.description('Log Analytics region name - Ensure the regions selected is a supported mapping as per: https://docs.microsoft.com/azure/automation/how-to/region-mappings.')
param parLogAnalyticsWorkspaceLocation string = resourceGroup().location
@allowed([
'CapacityReservation'
'Free'
'LACluster'
'PerGB2018'
'PerNode'
'Premium'
'Standalone'
'Standard'
])
@sys.description('Log Analytics Workspace sku name.')
param parLogAnalyticsWorkspaceSkuName string = 'PerGB2018'
@allowed([
100
200
300
400
500
1000
2000
5000
])
@sys.description('Log Analytics Workspace Capacity Reservation Level. Only used if parLogAnalyticsWorkspaceSkuName is set to CapacityReservation.')
param parLogAnalyticsWorkspaceCapacityReservationLevel int = 100
@minValue(30)
@maxValue(730)
@sys.description('Number of days of log retention for Log Analytics Workspace.')
param parLogAnalyticsWorkspaceLogRetentionInDays int = 365
@sys.description('''Resource Lock Configuration for Log Analytics Workspace.
- `kind` - The lock settings of the service which can be CanNotDelete, ReadOnly, or None.
- `notes` - Notes about this lock.
''')
param parLogAnalyticsWorkspaceLock lockType = {
kind: 'None'
notes: 'This lock was created by the ALZ Bicep Logging Module.'
}
@allowed([
'AgentHealthAssessment'
'AntiMalware'
'ChangeTracking'
'Security'
'SecurityInsights'
'ServiceMap'
'SQLAdvancedThreatProtection'
'SQLVulnerabilityAssessment'
'SQLAssessment'
'Updates'
'VMInsights'
])
@sys.description('Solutions that will be added to the Log Analytics Workspace.')
param parLogAnalyticsWorkspaceSolutions array = [
'AgentHealthAssessment'
'AntiMalware'
'ChangeTracking'
'Security'
'SecurityInsights'
'SQLAdvancedThreatProtection'
'SQLVulnerabilityAssessment'
'SQLAssessment'
'Updates'
'VMInsights'
]
@sys.description('''Resource Lock Configuration for Log Analytics Workspace Solutions.
- `kind` - The lock settings of the service which can be CanNotDelete, ReadOnly, or None.
- `notes` - Notes about this lock.
''')
param parLogAnalyticsWorkspaceSolutionsLock lockType = {
kind: 'None'
notes: 'This lock was created by the ALZ Bicep Logging Module.'
}
@sys.description('Log Analytics Workspace should be linked with the automation account.')
param parLogAnalyticsWorkspaceLinkAutomationAccount bool = true
@sys.description('Automation account name.')
param parAutomationAccountName string = 'alz-automation-account'
@sys.description('Automation Account region name. - Ensure the regions selected is a supported mapping as per: https://docs.microsoft.com/azure/automation/how-to/region-mappings.')
param parAutomationAccountLocation string = resourceGroup().location
@sys.description('Automation Account - use managed identity.')
param parAutomationAccountUseManagedIdentity bool = true
@sys.description('Automation Account - Public network access.')
param parAutomationAccountPublicNetworkAccess bool = true
@sys.description('''Resource Lock Configuration for Automation Account.
- `kind` - The lock settings of the service which can be CanNotDelete, ReadOnly, or None.
- `notes` - Notes about this lock.
''')
param parAutomationAccountLock lockType = {
kind: 'None'
notes: 'This lock was created by the ALZ Bicep Logging Module.'
}
@sys.description('Tags you would like to be applied to all resources in this module.')
param parTags object = {}
@sys.description('Tags you would like to be applied to Automation Account.')
param parAutomationAccountTags object = parTags
@sys.description('Tags you would like to be applied to Log Analytics Workspace.')
param parLogAnalyticsWorkspaceTags object = parTags
@sys.description('Set Parameter to true to use Sentinel Classic Pricing Tiers, following changes introduced in July 2023 as documented here: https://learn.microsoft.com/azure/sentinel/enroll-simplified-pricing-tier.')
param parUseSentinelClassicPricingTiers bool = false
@sys.description('Log Analytics LinkedService name for Automation Account.')
param parLogAnalyticsLinkedServiceAutomationAccountName string = 'Automation'
@sys.description('Set Parameter to true to Opt-out of deployment telemetry')
param parTelemetryOptOut bool = false
// Customer Usage Attribution Id
var varCuaid = 'f8087c67-cc41-46b2-994d-66e4b661860d'
resource resAutomationAccount 'Microsoft.Automation/automationAccounts@2022-08-08' = {
name: parAutomationAccountName
location: parAutomationAccountLocation
tags: parAutomationAccountTags
identity: parAutomationAccountUseManagedIdentity ? {
type: 'SystemAssigned'
} : null
properties: {
encryption: {
keySource: 'Microsoft.Automation'
}
publicNetworkAccess: parAutomationAccountPublicNetworkAccess
sku: {
name: 'Basic'
}
}
}
// Create a resource lock for the automation account if parGlobalResourceLock.kind != 'None' or if parAutomationAccountLock.kind != 'None'
resource resAutomationAccountLock 'Microsoft.Authorization/locks@2020-05-01' = if (parAutomationAccountLock.kind != 'None') {
scope: resAutomationAccount
name: parAutomationAccountLock.?name ?? '${resAutomationAccount.name}-lock'
properties: {
level: parAutomationAccountLock.kind
notes: parAutomationAccountLock.?notes
}
}
resource resLogAnalyticsWorkspace 'Microsoft.OperationalInsights/workspaces@2022-10-01' = {
name: parLogAnalyticsWorkspaceName
location: parLogAnalyticsWorkspaceLocation
tags: parLogAnalyticsWorkspaceTags
properties: {
sku: {
name: parLogAnalyticsWorkspaceSkuName
capacityReservationLevel: parLogAnalyticsWorkspaceSkuName == 'CapacityReservation' ? parLogAnalyticsWorkspaceCapacityReservationLevel : null
}
retentionInDays: parLogAnalyticsWorkspaceLogRetentionInDays
}
}
// Create a resource lock for the log analytics workspace if parGlobalResourceLock.kind != 'None' or if parLogAnalyticsWorkspaceLock.kind != 'None'
resource resLogAnalyticsWorkspaceLock 'Microsoft.Authorization/locks@2020-05-01' = if (parLogAnalyticsWorkspaceLock.kind != 'None') {
scope: resLogAnalyticsWorkspace
name: parLogAnalyticsWorkspaceLock.?name ?? '${resLogAnalyticsWorkspace.name}-lock'
properties: {
level: parLogAnalyticsWorkspaceLock.kind
notes: parLogAnalyticsWorkspaceLock.?notes
}
}
resource resLogAnalyticsWorkspaceSolutions 'Microsoft.OperationsManagement/solutions@2015-11-01-preview' = [for solution in parLogAnalyticsWorkspaceSolutions: {
name: '${solution}(${resLogAnalyticsWorkspace.name})'
location: parLogAnalyticsWorkspaceLocation
tags: parTags
properties: solution == 'SecurityInsights' ? {
workspaceResourceId: resLogAnalyticsWorkspace.id
sku: parUseSentinelClassicPricingTiers ? null : {
name: 'Unified'
}
} : {
workspaceResourceId: resLogAnalyticsWorkspace.id
}
plan: {
name: '${solution}(${resLogAnalyticsWorkspace.name})'
product: 'OMSGallery/${solution}'
publisher: 'Microsoft'
promotionCode: ''
}
}]
// Create a resource lock for each log analytics workspace solutions in parLogAnalyticsWorkspaceSolutions if parGlobalResourceLock.kind != 'None' or if parLogAnalyticsWorkspaceSolutionsLock.kind != 'None'
resource resLogAnalyticsWorkspaceSolutionsLock 'Microsoft.Authorization/locks@2020-05-01' = [for (solution, index) in parLogAnalyticsWorkspaceSolutions: if (parLogAnalyticsWorkspaceSolutionsLock.kind != 'None') {
scope: resLogAnalyticsWorkspaceSolutions[index]
name: parLogAnalyticsWorkspaceSolutionsLock.?name ?? '${resLogAnalyticsWorkspaceSolutions[index].name}-lock'
properties: {
level: parLogAnalyticsWorkspaceSolutionsLock.kind
notes: parLogAnalyticsWorkspaceSolutionsLock.?notes
}
}]
resource resLogAnalyticsLinkedServiceForAutomationAccount 'Microsoft.OperationalInsights/workspaces/linkedServices@2020-08-01' = if (parLogAnalyticsWorkspaceLinkAutomationAccount) {
parent: resLogAnalyticsWorkspace
name: parLogAnalyticsLinkedServiceAutomationAccountName
properties: {
resourceId: resAutomationAccount.id
}
}
// Optional Deployment for Customer Usage Attribution
module modCustomerUsageAttribution '../../CRML/customerUsageAttribution/cuaIdResourceGroup.bicep' = if (!parTelemetryOptOut) {
#disable-next-line no-loc-expr-outside-params //Only to ensure telemetry data is stored in same location as deployment. See https://github.com/Azure/ALZ-Bicep/wiki/FAQ#why-are-some-linter-rules-disabled-via-the-disable-next-line-bicep-function for more information
name: 'pid-${varCuaid}-${uniqueString(resourceGroup().location)}'
params: {}
}
output outLogAnalyticsWorkspaceName string = resLogAnalyticsWorkspace.name
output outLogAnalyticsWorkspaceId string = resLogAnalyticsWorkspace.id
output outLogAnalyticsCustomerId string = resLogAnalyticsWorkspace.properties.customerId
output outLogAnalyticsSolutions array = parLogAnalyticsWorkspaceSolutions
output outAutomationAccountName string = resAutomationAccount.name
output outAutomationAccountId string = resAutomationAccount.id
And now I get this error:
New-AzResourceGroupDeployment: /home/runner/work/github-repo/github-repo/pipeline-scripts/Deploy-ALZLoggingAndSentinel.ps1:30
Line |
30 | New-AzResourceGroupDeployment @inputObject
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Additional content found in JSON reference object. A JSON reference
| object should only have a $ref property. Path
| 'parLogAnalyticsWorkspaceLock.defaultValue'.
Error: Error: The process '/usr/bin/pwsh' failed with exit code 1So the common code for all this errors is this part from the bicep modules:
type lockType = {
@description('Optional. Specify the name of lock.')
name: string?
@description('Optional. The lock settings of the service.')
kind: ('CanNotDelete' | 'ReadOnly' | 'None')
@description('Optional. Notes about this lock.')
notes: string?
}
Hey @PolarbearChimney, I've been able to replicate your issue. However, it has me a bit stumped at the moment, as everything appears to be correctly structured for the user-defined type declaration and the associated parameter/parameter file.
It does appear to be impacting both Azure DevOps and GitHub, specifically related to the PR workflows/pipelines as you noted. I haven't been able to replicate with a normal deployment from any branches (including the same branches I am using to create the PRs) both locally and through Azure DevOps/GitHub with and/without what-if operation.
I will continue working on this tomorrow morning and let you know what I end up determining. Apologies for the issue, but thankfully your normal deployments not triggered from a PR should work.
cc: @jtracey93
@PolarbearChimney, still working through it, and have brought it up to teammates to get their thoughts as well.
Hey @PolarbearChimney, due to the help of @jtracey93, we ended up figuring out what the issue was. It is related to the issue noted here in the Azure Bicep repository.
Essentially, there is a bug with the Azure PowerShell Module version 11.3.1 where the default JSON serializer used to read Bicep output treats $ref properties as a JSON reference, whereas the behavior we actually want is to just preserve it in the serialized JSON.
We do specify within our workflows/pipelines to use the latest version of Az module. However, this "latest" version correlates to the latest version installed on the particular agent/runner which is 11.3.1 at this time. To get around this (until the agents have the updated Az version installed) you can explicitly reference a particular Az version for each PowerShell task/action as shown below:
We will also be planning on adding a variable to the .env file to be able to control the version without having to add each one individually.
Please let me know if you have any other concerns and/or issues and thanks for your patience!
I ran in to this issue as well and did not see mention of the solution in the docs or wiki. The Github workflows on the main branch have not been changed either.
Unless I am missing something (very possible), this bug means that anyone cloning this repo today is unable to use the Accelerator without sifting through closed Github issues. My humble recommendation is to update the docs with the patch you proposed above. I agree that this is a tricky bug, so hard coding the version into the GH action might not be a great solution for people to blindly git clone.
Hey @kardiojack, just got the information published in the new wiki page, Known Issues. Sorry for the wait!
Hello there!