anthonygauthier/jmeter-elasticsearch-backend-listener

Different timestamp issues

leonsery opened this issue · 9 comments

leonsery commented

Hi,
There is inconsistency between different timestamps:

  1. "Elapsed time" (in my case 01:32:05) is provided as a timestamp
    "ElapsedTime": "2019-01-05T01:32:05.000+0200", that is not usable!
    It should be provided in seconds (like response time, etc) - in my case it should be 5525
    In this case it will be possible to configure it in Grafana correctly

  2. "TestStartTime" is provided in UNIX Epoch format with milliseconds
    "TestStartTime": 1567672621311
    This is good :-), it can be used in Grafana URL "&from" parameter.
    See example:
    http://localhost:3000/d/Q0KrFWcWz/02-1-jmeter-tests?orgId=1&from=1567672621311&to=1567678146900

  3. "Timestamp" is provided in strict_date_hour_minute_second_millis format
    (yyyy-MM-dd'T'HH:mm:ss.SSS)
    "Timestamp": ["2019-09-05T10:09:06.900Z"]
    Please change it to the same UNIX Epoch format with milliseconds too, or add another field with such format.
    This will allow to use this field in grafana URL "&to" parameter (like in see example above).

Thank you in advance

Leon

leonsery commented

As well I tried to use the following header-based workarounds:

image

Unfortunately, although I see these headers in the load traffic, corresponding fields don't appears in the elastic index!!! I will open additional issue for this...

anthonygauthier commented

Unfortunately, although I see these headers in the load traffic, corresponding fields don't appears in the elastic index!!! I will open additional issue for this...

@leonsery If you want to use custom fields (without sending ALL headers to ElasticSearch). You need to use the following notation:

es.[...] not es-[...]

leonsery commented

Anthony, I'm a little confused. I need to have some custom variable that will updated on each request (to have elapsed time in grafana in format 'HH:mm:ss'). As I understand the only way to achieve this is to use headers in format es-[...], because es.[...] will use only first value (that listener gets during initialization). Where I wrong?

anthonygauthier commented

Oh sorry, I thought the screenshot was showing your ES listener's configuration. As for the time, I just pushed a fix for it. It will come out with version 2.6.8 - most likely out in a few hours.

leonsery commented

Thanks a lot!

anthonygauthier commented

PR to jmeter-plugins done here undera/jmeter-plugins#352
Otherwise, you can manually update your plugin by deleting v.2.6.8 from lib/ext and replacing it with https://github.com/delirius325/jmeter-elasticsearch-backend-listener/releases/tag/2.6.9

Thanks,

leonsery commented

Hi,
I installed new version. Now ElapsedTime is provided in UNIX EPOCH format, but it seems that value is not correct. After several seconds I have this value: "ElapsedTime":1546985180000
According to Epoch Converter (https://www.epochconverter.com/) this can be converted as:

GMT: Tuesday, January 8, 2019 10:06:20 PM
Your time zone: Wednesday, January 9, 2019 12:06:20 AM GMT+02:00
Relative: 8 months ago

I want to emphasize again - ElapsedTime should be provided in very simple format - seconds (or milliseconds) since starting of test.

anthonygauthier commented

Are you starting your tests from a CI or setting up JBuildNumber through your jmeter execution? If so, then that's probably why you're getting a weird timestamp:

public Date getElapsedTime(boolean forBuildComparison) {
        String sElapsed;
        //Calculate the elapsed time (Starting from midnight on a random day - enables us to compare of two loads over their duration)
        long start = JMeterContextService.getTestStartTime();
        long end = System.currentTimeMillis();
        long elapsed = (end - start);
        long minutes = (elapsed / 1000) / 60;
        long seconds = (elapsed / 1000) % 60;

        Calendar cal = Calendar.getInstance();
        cal.set(Calendar.HOUR_OF_DAY, 0); //If there is more than an hour of data, the number of minutes/seconds will increment this
        cal.set(Calendar.MINUTE, (int) minutes);
        cal.set(Calendar.SECOND, (int) seconds);

        if (forBuildComparison) {
            sElapsed = String.format("2017-01-01 %02d:%02d:%02d", cal.get(Calendar.HOUR_OF_DAY),
                    cal.get(Calendar.MINUTE), cal.get(Calendar.SECOND));
        } else {
            sElapsed = String.format("%s %02d:%02d:%02d",
                    DateTimeFormatter.ofPattern("yyyy-mm-dd").format(LocalDateTime.now()),
                    cal.get(Calendar.HOUR_OF_DAY), cal.get(Calendar.MINUTE), cal.get(Calendar.SECOND));
        }

        SimpleDateFormat formatter = new SimpleDateFormat("yyyy-mm-dd HH:mm:ss");
        try {
            return formatter.parse(sElapsed);
        } catch (ParseException e) {
            logger.error("Unexpected error occured computing elapsed date", e);
            return null;
        }
    }

Comparing two builds on Kibana required me to have a random date from midnight, otherwise the time series would get messed up for some reason. However, that was in 2017, it might have changed.

leonsery commented