Get summary results (scores) inside a test
Arahort opened this issue · 5 comments
What would you like to see in the library?
Get summary results (scores) inside a test
Why?
I generate an html report file and see the result in the log. But I need data from the log inside the test to process it and send it to the database.
Any other information that may help understanding the need
Test:
it('Lighthouse: Desktop', function () {
cy.viewport(1920, 1080).then(()=>{
cy.visit(pageUrl).then(()=>{
const lighthouseScore = {
performance: 0,
accessibility: 0,
seo: 0,
'best-practices': 0,
pwa: 0,
}
const lighthouseConfig = {
extends: 'lighthouse:default',
formFactor: 'desktop',
screenEmulation: {disabled: true}
}
cy.lighthouse(lighthouseScore, lighthouseConfig).then(()=>{
cy.get(databaseResult).then(()=>{
databaseResult['lighthouse_report'] = 'cypress/lighthouse/'+fileName+'lighthouse-report.html'
cy.log(
'cypress/lighthouse/'+fileName+'log.txt'
)
}).then(()=>{
cy.log(
`Lighthouse: report ready`
)
})
})
})
})
})Plugins:
module.exports = (on, config) => {
function clearUrl(url){
url = url.replace("https://", '');
url = url.replace("http://", '');
url = url.replace("www.", '');
url = url.replaceAll("/", '_');
return url;
}
const fileName = clearUrl(config.env.PAGE_URL)
const lighthouseReportPath = config.projectRoot + '\\cypress\\lighthouse\\'+fileName+'lighthouse-report.html'
const lighthouseLogPath = config.projectRoot + '\\cypress\\lighthouse\\'+fileName+'log.txt'
/*cypress-audit*/
on("before:browser:launch", (browser = {}, launchOptions) => {
prepareAudit(launchOptions);
if (browser.name === 'chrome' && browser.isHeadless) {
launchOptions.args.push('--disable-gpu');
return launchOptions;
}
});
/*cypress-audit END*/
on("task", {
/*cypress-audit*/
lighthouse: lighthouse((lighthouseReport) => {
let lighthouseResult = JSON.stringify(lighthouseReport)
fs.writeFileSync(lighthouseLogPath, lighthouseResult, 'utf-8')
fs.writeFileSync(lighthouseReportPath, lighthouseReportGenerator.generateReport(lighthouseReport.lhr, 'html'))
console.log(lighthouseReport)
}),
pa11y: pa11y((pa11yReport) => {
console.log(pa11yReport)
})
});
/*cypress-audit END*/`fs.writeFileSync(lighthouseLogPath, lighthouseResult, 'utf-8') - I get a lot of megabytes .txt file in which there is a huge amount of data
fs.writeFileSync(lighthouseReportPath, lighthouseReportGenerator.generateReport(lighthouseReport.lhr, 'html')) - i get html report
console.log(lighthouseReport) - I get what I need, but only AFTER running all the tests in the log.txt, during the test I cannot get this data from the file
I need to get data from log.txt and add it in the test to databaseResult array
Did it like this:
Test
cy.lighthouse(lighthouseScore, lighthouseConfig).then((lighthouseScoreText)=>{
/*let lighthouseResultScore = JSON.stringify(lighthouseScoreText)*/
cy.get(databaseResult).then(()=>{
databaseResult['lighthouse_report'] = fileName+'lighthouse-report.html'
databaseResult['lighthouse_score'] = lighthouseResultScore
cy.log(
lighthouseResultScore
)
}).then(()=>{
cy.log(
`Lighthouse: Report ready`
)
})
})Plugin
lighthouse: lighthouse((lighthouseReport) => {
let lighthouseScoreText = ''
let lighthouseResult = lighthouseReport?.lhr?.categories
let lighthousePerformance = 'Performance: '+lighthouseResult?.performance?.score+'\n'
let lighthouseAccessibility = 'Accessibility: '+lighthouseResult?.accessibility?.score+'\n'
let lighthouseBestPractices = 'Best Practices: '+lighthouseResult?.['best-practices']?.score+'\n'
let lighthouseSEO = 'SEO: '+lighthouseResult?.seo?.score+'\n'
let lighthousePWA = 'PWA: '+lighthouseResult?.pwa?.score
lighthouseScoreText = lighthousePerformance + lighthouseAccessibility + lighthouseBestPractices + lighthouseSEO + lighthousePWA
fs.writeFileSync(lighthouseLogPath, lighthouseScoreText, 'utf-8')
fs.writeFileSync(lighthouseReportPath, lighthouseReportGenerator.generateReport(lighthouseReport.lhr, 'html'))
console.log(lighthouseReport)
return lighthouseScoreText
})But cy.log(lighthouseResultScore) is empty
The file receives data from fs.writeFileSync(lighthouseLogPath, lighthouseScoreText, 'utf-8')
Is what you want to have the report result for each cy.lighthouse run?
Closing since not active for a while, feel free to reopen if the need is still there
Hi @mfrachet yes, I would like to reopen this issue.
Yes, it would be very helpful to access lighthouse report results for each cy.lighthouse run, not just once globally.
One example for this, is that we need to run tests where we ensure that version A of some site has performance parity with version B. This may need to be replicated across several different sites or pages.
Example Test:
describe(`Ensure Perf Parity between A and B`, () => {
let thresholds;
let lighthouseConfig;
let resultsA = {};
before(() => {
thresholds = {...}
lighthouseConfig = {...};
});
it('Evaluates web performance of Version A', () => {
cy.visit('https://versionA/home');
cy.lighthouse(thresholds, lighthouseConfig).then((reportJSON) => {
resultsA = reportJSON;
});
});
it('Evaluates web performance of Version B', () => {
cy.visit('https://versionB/home');
cy.lighthouse(thresholds, lighthouseConfig).then((reportJSON) => {
expect(reportJSON.performance).to.be.atMost(resultsA.performance);
});
});
Since the task already has access to the report, it's just a matter of returning it to the caller of the cypress task. Is this a nontrivial change?
We could just pass the same set of thresholds to both A and B, but we do not know what thresholds should be set for A to begin with, and it is a changing target.
This seems also like a useful feature if you need to run lighthouse tests for multiple use cases in your test suite. Otherwise, the current system only seems to allow you to write one report globally.
Thank you