Add more listeners

[silviuburceadev]

Hello!

Thank you for creating the DSL, it is a very nice tool! It would be even better with more listeners.

JMeter has more native listeners, which can be attached at HTTP sampler level, thread group level and test plan level. So far, DSL only supports 'View Results Tree'. I'd love to see the rest of them:

• Summary Report
• Aggregate Graph
• Graph Results
• Generate Summary Results
• Response Time Graph

In particular, I find Summary Report very neat. The reason is that we do not plan to use DSL to also run the plan, we just do it to generate the JMX files with HTTP samplers, grouped by thread groups, and show the results from JMeter UI to people who like stats & graphs. HTML report is nice but I find the listeners in JMeter a lot cleaner.

I might contribute, too, just need to figure out how/where the listeners are created in DSL.

[rabelenda]

Hello, thank you for asking this question and be willing to contribute too!.

Even though the use case is interesting, I think is too particular as to implement something specific in the DSL as to add built in DSL elements.

Have you considered using influxdb + grafana as shown in this section of user guide. That would allow you to have a centralized repository for reports, customize the reports in may ways, creating your own dashboards, with historic executions and you can even compare executions.

Another alternative might be using provided dashboard. The dashboard is intended for running locally and quickly visualizing results, but is not intended to be used in usual DSL execution while integrated in a CI/CD pipeline or running test at scale. We could add here some graph that might be interesting and still missing, and potentially add some option to select which graphs to include or not. Any in particular you think would be interesting on adding, considering already provided ones?

If you still need to have support for such listeners to use with JMeter GUI, one option is using wrappers. For example, for your scenario you could use something like:

JmeterEnvironment env = new JmeterEnvironment();
    testPlan(
        threadGroup(1, 1,
            httpSampler("https://localhost")
        ),
        testElement(new SummaryReport()),
        testElement(new StatGraphVisualizer()),
        testElement(new GraphVisualizer()),
        testElement(new SummariserGui()),
        testElement(new RespTimeGraphVisualizer())
    ).showInGui();

Another option can also be to implement your own DSL listeners (which you can reuse in different test plans and even different projects if you package them in jars), eg:

public static class DslSummaryReport extends BaseListener {

    public DslSummaryReport() {
      super("Summary Report", SummaryReport.class);
    }

    @Override
    protected TestElement buildTestElement() {
      // The only one that differs here is SummariserGui which uses Summariser test class
      return new ResultCollector();
    }

  }

  @Test
  public void performanceTest() throws Exception {
    testPlan(
        threadGroup(1, 1,
            httpSampler("https://localhost")
        ),
        new DslSummaryReport()
    ).showInGui();
  }

If you always add same elements to test plan, you might even implement a DSL element that when added adds all these elements pre configured. Eg:

public static class DslGuiListeners extends BaseListener {

    public DslGuiListeners() {
      super(null, null);
    }

    @Override
    protected TestElement buildTestElement() {
      // this is in fact not used since we overwrite buildTreeUnder
      return null;
    }

    @Override
    public HashTree buildTreeUnder(HashTree parent, BuildTreeContext context) {
      parent.add(buildSummaryReport());
      parent.add(buildAggregateGraph());
      parent.add(buildGraphResults());
      parent.add(buildGenerateSummaryResults());
      parent.add(buildViewResultsTree());
      return parent.add(buildResponseTimeGraph());
    }

    ...
    
    private TestElement buildResponseTimeGraph() {
      return configureTestElement(new ResultCollector(), "Response Time Graph", RespTimeGraphVisualizer.class);
    }

  }

  @Test
  public void performanceTest() throws Exception {
    testPlan(
        threadGroup(1, 1,
            httpSampler("https://localhost")
        ),
        new DslGuiListeners()
    ).showInGui();
  }

What do you think?

[silviuburceadev]

Thank you for the details. I would still like the DSL to be able to add in the generated JMX file everything that JMeter can offer visually, otherwise I'd have to patch the generated JMX file, either manually or as an extra step and I'm not looking forward to play with XML, if DSL can do it easily.

[rabelenda]

The DSL is mainly focused on running tests with it. saveAsJmx has been added just as a support/backdoor feature, as a temporary solution for migration or supporting some particular corner cases.

Even though I really value that you ask for this feature, and I encourage other users having similar use cases to speak up, I currently think this is a corner case that most users will not use.

As to keep the DSL simple for most of the users and use cases, we try to avoid adding apis that are only used in some particular scenarios.

In this scenario, I think that using one of provided solutions should help, using specific logic where is needed (making DSL usage a little more complex for "complex"/border case scenarios, and simple for simple scenarios) you achieve what you want without having to patch JMX file (provided solutions already generate JMX with required elements).

I do think though, that if you consider that we can improve provided live dashboard, as previously mentioned in one of the alternatives, by allowing to select some additional graphs, we could implement some changes to it to make it more flexible and general.

In this scenario, you wouldn't need to use JMeter GUI any longer (no need for a separate install, or having different ways for creating and running test plans, installing plugins, etc), better integrating with JMeter DSL use case, and we could provide additional features for most of jmeter dsl use cases and users.

To implement these changes (if they make sense to you), you could help us to identify which graphs (taking into consideration the info that you actually check on them) makes sense to add that are not yet covered by provided ones. For instance, Summary Report, Generate Summary Results and View Results Tree I think are not required, because summary report is already in dashboard, summary results is already provided in console output and DSL already provides a resultsTreeVisualizer.

[rabelenda]

I am closing this issue since no answer has been given. We can later re open it if you think we still need to review this.