Build getting failed with for lower performance threshold
Opened this issue · 3 comments
I have configured the performance threshold to 60% and 80 % for fail and unstable builds respectively. But I am unsure what exactly the issue is but the build is getting failed with the following error.
Calculated average from previous reports: 36.29135802469136
Current report average: 13.8
Current average as percentage of total average: 38.02558171179752%
Metric: Overall Application Performance|Average Response Time (ms) reported performance compared to an average of 38.02558171179752% . Build status is: FAILURE
I didn't understand the reason why it is trying to get the average response compared with the past builds.
Hi,
The idea is to use AppDynamics to gather performance metrics, as you would with e.g. JMeter. So during your build you should run a load test and performance metrics are fetched from AppDynamics.
The plugin then compares the results from the current build, with results from previous builds. If performance degrades, the build will fail.
E.g. if previous the average response time was 5 seconds, but in the last build it is 7.5 seconds, performance degraded by more then 20% so you will get a warning / failing build (depending on how you configured the thresholds).
I don't know whether all calculations are correct, for that I would need more information about your current and previous results, to compare them.
BR, Miel
Please find the below log for reference which made my build marked as failure it configured to mark build to fail/mark unstable for above 60%. But for 38 % it marked the build as failure.
Building in workspace F:\jenkins\Installation\workspace\AppDynamics
[AppDynamics] $ cmd /c call C:\windows\TEMP\hudson6083960022101086958.bat
F:\jenkins\Installation\workspace\AppDynamics>F:\jenkins\apache-jmeter-3.2\bin\jmeter -Jjmeter.save.saveservice.output_format=xml -Jjmeter.save.saveservice.response_data=true -n -t C:\Users\Prasad\Desktop\JMetettest.jmx -l F:\jenkins\Installation\workspace\AppDynamics\Test.jtl
Creating summariser
Created the tree successfully using C:\Users\Prasad\Desktop\JMetettest.jmx
Starting the test @ Mon Apr 24 18:49:35 IST 2017 (1493039975652)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary = 10 in 00:00:10 = 1.0/s Avg: 1090 Min: 1063 Max: 1247 Err: 0 (0.00%)
Tidying up ... @ Mon Apr 24 18:49:46 IST 2017 (1493039986685)
... end of run
Performance: Recording JMeter reports 'F:\jenkins\Installation\workspace\AppDynamics/Test.jtl'
Performance: Parsing JMeter report file 'F:\jenkins\Installation\jobs\AppDynamics\builds\30\performance-reports\JMeter\Test.jtl'.
Performance: No threshold configured for making the test unstable
Performance: No threshold configured for making the test failure
Performance: File Test.jtl reported 4.628% of errors [SUCCESS]. Build status is: SUCCESS
Verify connection to AppDynamics REST interface ...
Connection successful, continue to fetch measurements from AppDynamics Controller ...
Ready building AppDynamics report
Verifying for improving or degrading performance, main metric: Overall Application Performance|Average Response Time (ms) where lower is better = false
Performance degradation greater or equal than 80% sets the build as unstable
Performance degradation greater or equal than 60% sets the build as failure
Number of old reports located for average: 27
Calculated average from previous reports: 36.29135802469136
Current report average: 13.8
Current average as percentage of total average: 38.02558171179752%
Metric: Overall Application Performance|Average Response Time (ms) reported performance compared to average of 38.02558171179752% . Build status is: FAILURE
Build step 'AppDynamics Performance Publisher' changed build result to FAILURE
Finished: FAILURE
Hello,
We are using AppDynamics plugin in Jenkins to monitor the server performance during a performance test kicked by jenkins as post action build.
However we want to always PASS the build now and not fail due to any threshold.
I tried all options but still "FAILS" the build based on some calculation.
Can you please help in selecting the correct configuration in Advanced section as the documentation says .
"Furthermore, some advanced settings are available. You can configure the error percentage thresholds which would make the project unstable or failed or leave them blank to disable the feature."
Appreciate your help in this.
Build step 'Execute HP tests using HP Performance Center' changed build result to UNSTABLE
Verify connection to AppDynamics REST interface ...
Connection successful, continue to fetch measurements from AppDynamics Controller ...
No result for metric: Overall Application Performance|
--> Check if metric exists and correctly formatted.
Ready building AppDynamics report
Verifying for improving or degrading performance, main metric: Overall Application Performance|Errors per Minute where lower is better = true
Performance degradation greater or equal than 100% sets the build as unstable
Performance degradation greater or equal than 100% sets the build as failure
Number of old reports located for average: 101
Calculated average from previous reports: 0.6151393104126105
Current report average: 0.0
Current average as percentage of total average: Infinity%
Metric: Overall Application Performance|Errors per Minute reported performance compared to average of Infinity% . Build status is: UNSTABLE
Recording test results
Finished: UNSTABLE