First Visual Change - detection bug
Closed this issue · 5 comments
I get very inconsistent detection from Lighthouse for : First Visual Change / Last Visual Change
The measurements have varied from 350ms to over 55,000ms, while in reality performance remains fairly constant, including visual inspection while Lighthouse is running. (I can see the page loading for less than 1 sec. and the tests continuing, but in the end-report there can be "measured" over 10 seconds for "First Visual Change")
It seems when I clear browser's cash and hosted app data, the measurement is more of what is expected - less 1 sec.
The app tested was: https://kinoseed.com/
Neat application! :)
How are you running lighthouse, via extension, CLI, or via node API? 55,000ms shouldn't be a possible value with the default implementation since the trace times out at 25s. Anecdotally, my couple of runs on that URL all ended up with 700ms-1100ms for first visual change, but any additional information on your environment would help here.
If you can give an example of the failure you're describing in the CLI with the --save-assets flag so we can take a look at the .trace.json file that'd be awesome, and we'd be able to diagnose.
I run it as an extension: Lighthouse Version 1.6.5
Chrome Version 57.0.2987.110 (64-bit)
Just ran it a couple of more times, and what is strange is that it reports:
- First meaningful paint: 345.4ms
which I think is accurate, and the site is visually rendered a few ms later, but ... at the same time it reports:
- Perceptual Speed Index: 4663 (target: 1,250)
.. 2 runs gave me more than 4 seconds
next 2 were fast (0.4sec)
I closed the tab, reloaded the page, and run it again:
First Visual Change: 5436ms
next run gave me: First Visual Change: 412ms
There are about 20-30 tabs opened (of different sites), other than that there is nothing special about the env.
the "measured" over 50,000ms happened coupe of days ago, although the whole report was generated for less than 10 seconds (so at least to me, it looks like the "measuring" is at fault, not the actual test)
a crash/failure was never indicated, only the "First Visual Change" gives false results at times.
and thanks :)
edit:
it can't be a decimal point floating (rounding) error, which only happens occasionally... can it?
instead of Perceptual Speed Index: 4663
it is much more reasonable to be 466.3
interesting. https://kinoseed.com/ loads really fast btw. nice work!
we'll investigate what's going on here.
It just happened again (the whole report was generated for about 10 seconds or less):
First meaningful paint: 314.3ms (target: 1,600ms)
100
Perceptual Speed Index: 49187 (target: 1,250)
0
First Visual Change: 49186ms
Last Visual Change: 49254ms
Here is the .json export:
kinoseed.com_2017-04-29_03-25-09.json.zip