[BUG] Pipeline task returns InvalidPassFailCriteria when yaml has more than 5 failureCriteria
Opened this issue · 2 comments
leandrodotec commented
Describe the bug
AzureLoadTest@1 - Azure Load Testing v1 task fails pipeline with error code: 'InvalidPassFailCriteria' when failureCriteria contains more than 5 items.
To Reproduce
Steps to reproduce the behavior:
pipeline.yml:
# variables are set before this, other details removed for brevity
stages:
- stage: LoadTest
displayName: Load test
jobs:
- job: Run
displayName: Runs a fixed load test in azure
steps:
- task: AzureLoadTest@1
displayName: Azure load test
inputs:
azureSubscription: $(serviceConnection)
loadTestResource: $(loadTestResource)
resourceGroup: $(loadTestResourceGroup)
loadTestConfigFile: src/HomePage/HomePage.yml
src/HomePage/HomePage.yml:
version: v0.1
testName: BHB_Home_Page
testPlan: HomePage.jmx
description: BHB Home page
engineInstances: 1
splitAllCSVs: False
failureCriteria:
- AppointmentType: percentage(error) > 1
- AppointmentType: p90(response_time_ms) > 1000
- PatientLocationById: percentage(error) > 1
- PatientLocationById: p90(response_time_ms) > 1000
- Provider: percentage(error) > 1
- Provider: p90(response_time_ms) > 1000
env:
- name: JM_RampTime
value: 30
- name: JM_EndRpsA
value: 100
- name: JM_EndRpsB
value: 100
- name: JM_HoldFor
value: 300
- name: JM_CoolDown
value: 10
- name: JM_PauseFor
value: 5
result from DevOps pipelines
Starting: Azure load test
==============================================================================
Task : Azure Load Testing
Description : Automate performance regression testing with Azure Load Testing
Version : 1.2.9
Author : Microsoft Corporation
Help : https://docs.microsoft.com/azure/load-testing/tutorial-cicd-azure-pipelines#azure-load-testing-task
==============================================================================
{ error:
{ code: 'InvalidPassFailCriteria',
message: 'Invalid pass fail criteria "Invalid PassFailCriteria" .',
target: null,
details: null } }
##[error]Error in creating test: bhb_home_page
Finishing: Azure load test
Expected behavior
Azure load testing task should allow create/update test with the same amount of test criteria as the tests created by portal.
Screenshots
Additional context
Add any other context about the problem here.
AB#1642808
Sachid26 commented
@leandrodotec ... Noted. we will update once we have rectified the issue.
vinhent commented
We are having the same issue