Is it possible to run multiple test scenarios from the same file at once?

[emmaxtung]

Hello, I wanted to see if I could create one file for all the test scenarios which will be run with Node.js.

I tried doing this by having multiple scenario modules defined in one test scenario file, and then in the node test file I have designated different directories for each scenario's results and included all the run statements in one function.
However when I run the node test file, it only produces a result for the first scenario, followed by an error in the console that states "target is unknown to synthesizers". I have the node test file function set up like the example on this page.
I also tried splitting up the memlab run commands for each scenario in the node test file into separate asynchronous functions but then nothing runs successfully and I get an error of "unknown app: UNSET".

Is there a way to set up my tests so that I can have them all defined in one file and ran at once, with the results all going into separate directories? Thanks in advance

[JacksonGL]

Could you provide a specific coding example illustrating how you define all test cases into a single file?

[emmaxtung]

Sure - I kept all the scenarios pretty simple just for now until I get this working.

Scenario test file:

const scenario1 = {
  url: () => 'https://www.google.com/maps/@40.7381132,-73.9914544,17z?entry=ttu', 
  action: async (page) => {
    await page.click('button[aria-label="Restaurants"]') 
  },
  back: async (page) => {
    await page.click('button[aria-label="Clear search"]') 
  },
}

const scenario2 = {
  url: () => 'https://www.google.com/maps/@40.7381132,-73.9914544,17z?entry=ttu', 
  action: async (page) => {
    await page.click('button[aria-label="Hotels"]')
  },
  back: async (page) => {
    await page.click('button[aria-label="Clear search"]') 
  },
}

const scenario3 = {
  url: () => 'https://www.google.com/maps/@40.7381132,-73.9914544,17z?entry=ttu',
  action: async (page) => {
    await page.click('button[aria-label="Things to do"]') 
  },
  back: async (page) => {
    await page.click('button[aria-label="Clear search"]') 
  },
}

module.exports = { scenario1, scenario2, scenario3 };

Node test driver file:

const { run } = require('@memlab/api');
const { scenario1, scenario2, scenario3 } = require('./test-scenario.js');
const fs = require('fs-extra');


// Running all tests in one function - produces result for the first scenario, then an error of "target is unknown to synthesizers" after the first retainer trace
(async function () {
	const workDir = './results';
	const workDir2 = './results2'; 
	const workDir3 = './results3';

	fs.mkdirsSync(workDir);
	fs.mkdirsSync(workDir2);
	fs.mkdirsSync(workDir3);
	
	const result = await run({scenario1, workDir});
	const result2 = await run ({scenario2, workDir2});
	const result3 = await run ({scenario3, workDir3});
})();


/*
// Separating each test into separate functions - produces error "unknown app: UNSET"
(async function test1() {
	const workDir = './results';
	fs.mkdirsSync(workDir);
	const result = await run({scenario1, workDir});
})();


(async function test2() {
	const workDir2 = './results2'; 
	fs.mkdirsSync(workDir2);
	const result = await run ({scenario2, workDir2});
})();

(async function test3() {
	const workDir3 = './results3';
	fs.mkdirsSync(workDir3);
	const result = await run ({scenario3, workDir3});
})();
*/

[JacksonGL]

I can't reproduce the problem. Having multiple test scenarios in one file works on my machine.

The example files you provided doesn't really pass in the scenario and workDir argument to memlab API. Maybe that is the root cause:

const result = await run({scenario1, workDir});
const result2 = await run({scenario2, workDir2});
const result3 = await run({scenario3, workDir3});

It should be:

const result = await run({scenario: scenario1, workDir});
const result2 = await run({scenario: scenario2, workDir: workDir2});
const result3 = await run({scenario: scenario3, workDir: workDir3});

[emmaxtung]

That seemed to fix it for me, thank you! Totally overlooked the syntax for the arguments....