Proposal: Add a group to commands for nicer command logging

Question

Proposal: Add a group to commands for nicer command logging

Opened this issue 7 years ago · 13 comments

I've been playing around with ways of defining reusable helpers (blog post coming) and have been using the Cypress log API and passing { log: false } to allow control of the log output of these helpers/commands. This has been useful, but there are some times that I want to dive into that abstraction (most likely on failure) to see what step of the helper failed.

I think a concept of a group would be useful here. Instead of passing { log: false } to every encapsulated command, having a group would help a lot more

Current behavior:

// This will work on https://github.com/cypress-io/cypress-example-todomvc
export const createTodo = (name) => {
  const log = Cypress.log({
    name: 'createTodo',
    message: name,
    consoleProps() {
      return {
        'Inserted Todo': name,
      }
    }
  })
  cy.get('.new-todo', { log: false }).type(`${name}{enter}`, { log: false })

  return cy
    .get('.todo-list li', { log: false })
    .contains('li', name.trim(), { log: false })
    .then(($el) => {
      log.set({ $el }).snapshot().end()
    })
}

// to use this helper:
createTodo('Learn the Cypress Log API')
  .then(console.log) // logs the created todo element from the TodoMVC Cypress example

Screenshot of the above:

Proposed behavior:

export const createTodo = (name) => {
  const group = Cypress.group({
    name: 'createTodo',
    message: name,
    consoleProps() {
      return {
        'Inserted Todo': name,
      }
    }
  })

  cy.get('.new-todo').type(`${name}{enter}`)

  return cy
    .get('.todo-list')
    .contains('li', name.trim())
    .then($el => { group.set({ $el }).snapshot().end() })

Matching the log API seems like a logical choice. In the UI it would show createTodo with the logging the same as the Current behavior example, but would have an arrow that would expand to the grouped commands. This would both simplify creating custom command output (don't have to pass { log: false } to everything) as well as make it easier to understand that's going on under the hood of a custom command. The group would be collapsed by default and expanded if a nested command failed.

Answer 1 · 2018-03-18T10:54:46.000Z

This proposal is conceptually similar to the Web Console log's group: https://developer.mozilla.org/en-US/docs/Web/API/Console/group

Answer 2 · 2018-05-24T08:56:48.000Z

I'm surprised this didn't get more votes yet.

I'd love to be able to group several commands into one, nicely labelled, group that is collapsed by default, and autoxpands on a failure.

Also note this could be useful when having a larger test (which we are encouraged to do) to group sections of a single test case for better readability.

Answer 3 · 2018-11-19T06:31:50.000Z

I was thinking about this more. Basically as test suites get larger, people look to ways to "macro" commands. There are a few ways to do this:

Cypress Custom Commands

// definition
Cypress.Commands.add('fillForm', (first, last) => {
  cy.get('#first').type(first)
  cy.get('#last').type(last)
}

// usage
cy.fillForm('John', 'Doe')

// log
GET  #first
TYPE John
GET  #last
TYPE Doe

Regular functions

// definition
function fillForm(first, last) {
  cy.get('#first').type(first)
  cy.get('#last').type(last)
}

// usage
fillForm('John', 'Doe')

// log
GET  #first
TYPE John
GET  #last
TYPE Doe

Functional composition (.then)

// definition
const fillForm = (first, last) => ($form) => {
  cy.wrap($form).find('#first').type(first)
  cy.wrap($form).find('#last').type(last)
}

// usage
cy
  .get('#form')
  .then(fillForm('John', 'Doe'))

// log
GET  #form
THEN
WRAP <form>
FIND #first
TYPE John
WRAP <form>
FIND #first
TYPE Doe

Functional composition (.pipe)

// definition
const fillForm = (first, last) => function fillForm($form) {
  cy.wrap($form).find('#first').type(first)
  cy.wrap($form).find('#last').type(last)
}

// usage (.pipe)
cy
  .get('#form')
  .pipe(fillForm('John', 'Doe'))

// log (.pipe)
GET  #form
PIPE fillForm
WRAP <form>
FIND #first
TYPE John
WRAP <form>
FIND #first
TYPE Doe

Page Objects

// definitions
class Page {
  fillForm(first, last) {
    cy.get('#first').type(first)
    cy.get('#last').type(last)
  }
}

// usage
const page = new Page()

page.fillForm('John', 'Doe')

// log
GET  #first
TYPE John
GET  #last
TYPE Doe

The function name fillForm isn't shown in the log output of all the examples except cypress-pipe. Even cypress-pipe is confusing, because I only see cy.get('#form').pipe(fillForm('John', 'Doe')) in my test, but a bunch more commands logged in the Command Log. Grouping would make this much easier to understand the relationship between my test code and the log.

The following could be modifications to get grouping to work:

Cypress Custom Commands

// definition
Cypress.Commands.add('fillForm', (first, last) => {
  cy.group('fillForm', () => {
    cy.get('#first').type(first)
    cy.get('#last').type(last)
  })
}

// usage
cy.fillForm('John', 'Doe')

// log
GROUP fillForm
  - GET  #first
  - TYPE John
  - GET  #last
  - TYPE Doe

Regular functions

// definition
import { group } from '@cypress/group' // Higher-order function/class decorator

const fillForm = group('fillForm', (first, last) => {
  cy.get('#first').type(first)
  cy.get('#last').type(last)
})

// usage
fillForm('John', 'Doe')

// log
GROUP fillForm
  - GET  #first
  - TYPE John
  - GET  #last
  - TYPE Doe

Functional composition (.then)

// definition
const fillForm = (first, last) => ($form) => {
  cy.group('fillForm', () => {
    cy.wrap($form).find('#first').type(first)
    cy.wrap($form).find('#last').type(last)
  })
}

// usage
cy
  .get('#form')
  .then(fillForm('John', 'Doe'))

// log
GET  #form
THEN
GROUP fillForm
  - WRAP <form>
  - FIND #first
  - TYPE John
  - WRAP <form>
  - FIND #first
  - TYPE Doe

Functional composition (.pipe) (no changes actually needed)

// definition
const fillForm = (first, last) => function fillForm($form) {
  cy.wrap($form).find('#first').type(first)
  cy.wrap($form).find('#last').type(last)
}

// usage
cy
  .get('#form')
  .pipe(fillForm('John', 'Doe'))

// log
GET  #form
PIPE fillForm
  - WRAP <form>
  - FIND #first
  - TYPE John
  - WRAP <form>
  - FIND #first
  - TYPE Doe

Page Objects

// definitions
import { group } from '@cypress/group' // Higher-order function/class decorator
class Page {
  @group('fillForm') // optional if name can't be inferred
  fillForm(first, last) {
    cy.get('#first').type(first)
    cy.get('#last').type(last)
  }
}

// usage
const page = new Page()

page.fillForm('John', 'Doe')

// log
GROUP fillForm
  - GET  #first
  - TYPE John
  - GET  #last
  - TYPE Doe

Answer 4 · 2019-07-08T19:15:15.000Z

I spent some time looking into this over the weekend. I don't think group can be added without changes to the Cypress runner code. The React UI renders a list, but will need to render a tree. cy.within would be easier to understand as well in the Command Log if it also created a group. Right now you can understand what is inside a cy.within while debugging by clicking on Commands inside a within by the debug output, but not part of the Video or screenshots.

Next step would be to evaluate how much code needs to change to support grouping. cy.within required changes to all commands within querying.coffee to know if querying APIs should target the root element or the withinSubject. Grouping doesn't seem to need that level of interaction with other commands, but does require cooperation with the UI.

Answer 5 · 2022-10-04T16:29:58.000Z

I too would love to be able to utilise grouping of logs for custom commands as some of the built in commands (e.g. cy.within) do.

It now looks like cy.within uses logGroup internally to allow commands executed inside the within block to be collapsed in the runner UI, but I assume logGroup is not exported or otherwise available to end users authoring tests?

Answer 6 · 2022-11-22T08:57:44.000Z

https://j1000.github.io/blog/2022/10/27/enhanced_cypress_logging.html

Answer 7 · 2023-06-16T13:14:49.000Z

It seems that the parameters outlined in that blog post aren't yet in the TypeScript definitions:

VSCode is complaining but Cypress is fine.

And while it's nice to be able to rather simply create groups this way, it would be nice to be able to close them as easily too.

Maybe an additional option for Cypress.log({ groupEnd: true, groupCollapse: true }) to be used in conjunction with groupEnd?

Answer 8 · 2023-07-24T09:34:06.000Z

see also please expose groupStart and groupEnd

Answer 9 · 2023-12-08T14:47:19.000Z

In our use case, it would be very nice to have this implemented. Because our test suites and tests are so long that we have to split them by it(). That’s the only way we can survive. But this has 2 main drawbacks:

tests don’t stop where they fail, the next it starts while we know it won’t succeed because of the previous failure,
our Cypress Cloud quota is exploded in 2 runs…

As Cypress Cloud is billed on an it() usage basis, it’s very expensive for us to run tests.

So we would love to have this implemented :-)

Answer 10 · 2024-02-09T22:55:35.000Z

As of now, we have implemented this workaround:

index.d.ts

/// <reference types="cypress" />

declare namespace Cypress {
    interface Chainable<Subject = any> {
        group(label: string, fn: () => void): void;
        skip(label: string, fn: () => void): void;
    }
}

commands.ts

Cypress.Commands.add("group", { prevSubject: false }, (label, fn) => {
    const log = Cypress.log({
        name: "group",
        displayName: label,
        message: [],
        autoEnd: true,
        // @ts-ignore
        groupStart: true
    });


    return cy
        .window({ log: false })
        .then(() => {
            fn();
        })
        .then(() => {
            // @ts-ignore
            log.endGroup();
        });
});

Cypress.Commands.add("skip", { prevSubject: false }, (label, fn) => {
    Cypress.log({
        name: "skip",
        displayName: `⏹ ${label}`,
        message: ["_skipped_"],
        autoEnd: true,
        // @ts-ignore
        groupStart: false,
    });
});

Usage:

describe('My describe', () => {
    it('My test should group', () => {
        cy.visit('/test.html')
        cy.get('h1').should('contain.text', 'Testing')


        cy.group("Label", () => {
            cy.get('h1').should('contain.text', 'Lorem')

            cy.skip("More deeply grouping", () => {
                cy.get('h1').should('contain.text', 'ipsum')
            })
        })


        cy.get('p').should('contain.text', 'Lorem ipsum')

    })
})

Answer 11 · 2024-02-12T13:30:18.000Z

As of now, we have implemented this workaround:
commands.ts

Cypress.Commands.add("group", { prevSubject: false }, (label, fn) => {
    const log = Cypress.log({
        name: "group",
        displayName: label,
        message: [],
        autoEnd: true,
        // @ts-ignore
        groupStart: true
    });


    return cy
        .window({ log: false })
        .then(() => {
            fn();
        })
        .then(() => {
            // @ts-ignore
            log.endGroup();
        });
});

Do you also have the problem that this spinner always shows?

And the group is not collapsed. Would be really helpful if group collapsing and the never-ending spinner could be "fixed" somehow. But also the group logging mechanism should not add to the overall execution time of the test. I'd like to add group logging on a per-UI-component basis (i.e. by default I want to hide the specifics of how to select a value from an Angular Material Dropdown). At least Cypress does not mess with the order of logs as Playwright does ;-) (inconsistent order of logs 🤔)

Answer 12 · 2024-02-12T13:34:02.000Z

@mirobo

Do you also have the problem that this spinner always shows?

No

And the group is not collapsed.

No, they are not collapsed, but I added a CSS rule to see them more easily.

Answer 13 · 2024-02-13T09:58:13.000Z

Do you also have the problem that this spinner always shows?

I have that with own version occasionally, but it's more often with 3rd party requests that just "died".

And the group is not collapsed.

I dug through the UI to get mine to do that, works most of the time, but with very fast tests it's possible that the UI updates faster than the code can keep up with, in which case groups to be closed stay open.

Cypress.Commands.add("group", { prevSubject: false }, (label, fn) => {
  const group_id = `grp_${new Date().getTime()}`;
  
  Cypress.log({
    id: group_id,
    displayName: label,
    message: [],
    // @ts-ignore
    groupStart: true,
  });
  
  const chain = fn(group_id);
  
  return cy.wait(1, { log: false })
    .then(() => {
      Cypress.log({ groupEnd: true, emitOnly: true });
      
      const loggedCommands = Array.from(window?.top?.document
        ?.querySelectorAll("ul.commands-container > li.command") || []) as HTMLElement[];
      // @ts-ignore
      if (loggedCommands.length > 0) {
        // "li.command" => .__reactInternalInstance*.return.key == group_id
        for (const command of loggedCommands.reverse()) {
          const internal = Object.keys(command).find(key => key.startsWith("__reactInternalInstance"));
          if (!internal) { continue; }
          if (command[internal].return.key == group_id) {
            // cy.log("command 1st child:", group_id, command.firstChild)
            const expander = Array.from(command.firstChild?.childNodes || [])
              .find(node => (node as HTMLElement).className == "command-expander-column")
              ?.firstChild as HTMLElement;
            if (!expander) {
              cy.log("couldn't find '> .command-expander-column'")
              continue;
            }
            if (expander.classList.contains("command-expander-is-open")) {
              // @ts-ignore
              expander.parentNode.click();
              break;
            };
          }
        }
      }
        
      return chain;
    })
});

The way it works is that it generates a unique group_id for each call and assigns it to Cypress.log({ id }), because this was the ONLY thing I could find that would manage to make it into the UI somehow and be findable. Disassembling React is always such a fun task.
Anyways, after I found out how to find the group_id set that way, I just had to dig through the list to find it, then back out to find the folding toggle and click it.

It's also likely that this might not work right away if you directly copy it into your setup, because this is just a part of my version.
The entire thing is at this point so complicated, because I've added various other features like (wonky) auto retry if something fails within the group and custom function call if something writes an error to the console, that I myself needed to take several looks over it again to even understand how the hell the damn thing worked. And I'm the one who built it...