es.test.mode "info" is sending results that doesn't contain errors
hanaleung opened this issue · 10 comments
I'm testing on jmeter 4.0 and setting es.test.mode="info" is still sending messages to ElasticSearch even when there isn't an error.
I'm using ElasticSearch 6.2.4
Kibana Result without filter:
Filtering my messages based on ErrorCount:1:
Hey @hanaleung ,
I'll be glad to look into the issue. May I know which realease of the plugin you are currently using?
Thanks,
Version 2.2.5
And maybe this matters, i'm running 1 jmeter client that is distributing load to 4 jmeter servers to send to my test site.
Using version 2.2.5 myself,
I am unable to reproduce the issue on a single JMeter execution. I'll have to try on a remote execution later tonight.
Would it be possible for you to supply me with the JSON data of the samplers for which the request/response body should not appear? (don't forget to blur the information related to the app you are testing)
Thanks,
As defined in the documentation though:
Info mode should be used when the tests are being executed in a production context. This mode will only send the details of the responses upon failed requests. This allows you to be able to locate the errors when visualizing the tests through either Kibana/Grafana.
info : Only send the request/response information of failed requests (headers, body, etc.)
By "details" what we mean is that only the samplers which are unsuccessful will have their request/response body sent to ElasticSearch. It doesn't mean that only the errors will be sent to ElasticSearch.
It could however, be a good idea to introduce such a mode.
Thanks,
Hi,
i may be misinterpretting what is a failed request, but here is what i expect should be sent to ElasticSearch under "info" mode which i do see on elasticSearch and can identify via ErrorCount field.
{ "_index": "stockholm_capacity", "_type": "SampleResult", "_id": "a4bDt2MBDfEYHFPh-WVH", "_version": 1, "_score": null, "_source": { "EndTimeInMs": 1527796463910, "ContentType": "", "EndTime": "2018-05-31T15:54:23.910-0400", "IdleTime": 0, "ElapsedTime": "Jan 31, 2018 12:36:53 AM", "ErrorCount": 1, "Timestamp": "2018-05-31T15:54:23.910-0400", "URL": "_removed_", "Bytes": 1158, "SentBytes": 0, "ElapsedTimeInMs": -556, "AllThreads": 118, "DataType": "text", "ResponseTime": 0, "SampleCount": 1, "ConnectTime": 0, "RequestHeaders": "", "RequestBody": "_removed_", "StartTimeInMs": 1527796463910, "ResponseCode": "Non HTTP response code: java.net.URISyntaxException", "StartTime": "2018-05-31T15:54:23.910-0400", "ResponseMessage": "Non HTTP response message: Illegal character in path at index 104: _removed_", "ResponseHeaders": "", "AssertionResults": [ { "failure": true, "name": "json_assertion", "failureMessage": "Expected to find an object with property ['catalogGroupView'] in path $ but found 'java.lang.String'. This is not a json object according to the JsonProvider: 'com.jayway.jsonpath.spi.json.JsonSmartJsonProvider'." } ], "Latency": 0, "ResponseBody": "", "GrpThreads": 118, "BuildNumber": 0, "BodySize": 0, "ThreadName": "guest_shopper_api 1-5", "SampleLabel": "_removed_" }, "fields": { "EndTime": [ "2018-05-31T19:54:23.910Z" ], "StartTime": [ "2018-05-31T19:54:23.910Z" ], "Timestamp": [ "2018-05-31T19:54:23.910Z" ] }, "sort": [ 1527796463910 ] }
And then here is what I don't expect to be logged but is currently being logged anyway:
{ "_index": "stockholm_capacity", "_type": "SampleResult", "_id": "X5DTt2MBDfEYHFPhYpk4", "_version": 1, "_score": null, "_source": { "EndTimeInMs": 1527797473836, "ContentType": "application/json", "EndTime": "2018-05-31T16:11:13.836-0400", "IdleTime": 0, "ElapsedTime": "Jan 31, 2018 12:53:43 AM", "ErrorCount": 0, "Timestamp": "2018-05-31T16:11:13.765-0400", "URL": "_removed_", "Bytes": 1025, "SentBytes": 293, "ElapsedTimeInMs": -489, "AllThreads": 150, "DataType": "text", "ResponseTime": 71, "SampleCount": 1, "ConnectTime": 0, "StartTimeInMs": 1527797473765, "ResponseCode": "200", "StartTime": "2018-05-31T16:11:13.765-0400", "AssertionResults": [ { "failure": false, "name": "json_assertion", "failureMessage": "" } ], "Latency": 71, "GrpThreads": 150, "BuildNumber": 0, "BodySize": 727, "ThreadName": "guest_shopper_api 1-137", "SampleLabel": "_removed_" }, "fields": { "EndTime": [ "2018-05-31T20:11:13.836Z" ], "StartTime": [ "2018-05-31T20:11:13.765Z" ], "Timestamp": [ "2018-05-31T20:11:13.765Z" ] }, "sort": [ 1527797473765 ] }
I forget to mention, the HTTP Request Samplers are embedded within it's own Transaction Controller but is part of a overall parent Transaction controller under a Test Fragment.
When my test runs, the main Testplan will only call upon IncludeControllers to run it's samplers.
Hi again,
I'm afraid what you are asking for is a new mode as the info mode is acting as it should. I do understand what you want though.
Basically, what you would like is an "error" mode which would only send the samplers which are failing to the ElasticSearch engine. It makes sense and I can see times when such a functionality could be useful. I will get to it first thing in the morning tomorrow.
I will also make the documentation more clear as to what the info mode reports.
For your own understanding though. Basically, the "info" mode behaves just like the "View results tree" listener would in GUI mode - with the "error only" checkbox checked that is. This means that all the samplers' metrics are going to be "saved" but only the failing ones would have their request/response details logged.
Thank you for your comments!
It is much appreciated :)
Hey @hanaleung,
I released version 2.3.0 a few minutes ago : https://github.com/delirius325/jmeter-elasticsearch-backend-listener/releases/tag/2.3.0
Feel free to manually upgrade and use the "error" test mode. The "error" test mode will only send the samplers that failed to the ElasticSearch engine :) !
Waiting for the JAR file to be on Maven's central repository before pushing the new version to JMeter-Plugins
Closing the issue :)
Cheers,
Thanks for clearing up my misunderstanding and your prompt response.
it seems error mode is what i was looking for and i had thought info mode was that behavior but now i see the difference.
I'll pull the latest and I'll give the error mode a try
Cheers!
@hanaleung : You're very welcome, don't forget to star the repo so it gets noticed :) !
As a side note, PR created on Jmeter-plugins