callstack/reassure

[IMPROVEMENT] Precision clarification

Closed this issue · 1 comments

Summary

There's a bit of a misleading/wrong information in the documentation saying;

We measure React component render times with microsecond precision during performance measurements using React.Profiler

This may or may not be true depending on the test environment. You can see that's not true for the component tests if you check one of the latest PR reports, like this one:

Screenshot 2024-07-26 at 14 39 07

Tests have millisecond precision here. React Profiler uses performance.now if it exists in the environment and I believe it fallbacks to Date.now otherwise.

React Native test environment is basically node environment and it does have performance.now exposed already which is cool but it is actually a mock and is Date.now under the hood. You can simply test this by logging a performance.now call on you jest setup and see it logs timestamp which is not native Performance api does. Date.now has millisecond precision and it's the root of this issue.

I found this problem because we have both web and native reassure test setup. I realized that the web tests have sub-millisecond precision while native tests don't. Web env is based on jsdom and it does emulate browser environment which has a proper Performance api mock/support. So, React Profiler is able to properly profile the tests with higher precision on jsdom environment.

Same level of precision could be achieved for the native tests by mocking performance.now to the native node module from perf_hooks.

What can be done?

The library can easily expose a test setup call which will be called by the library users. That can automatically make the necessary mocking for the users. This is the explicit option. I think there are implicit ways to do the same thing since Reassure starts its own jest process. It's a matter of choice wether you want to communicate this with the users to not surprise them, potentially break their existing tests etc.

I think the microsecond precision statement in the docs should be changed. You don't end up with microsecond level precision even with the proposed change. I think the precision is around 100microsecond level, so it's probably better to replace the statement with something like sub-millisecond precision in that case.

Manual solution

We can also explain this to the users and let them take the action themselves.

Directly mocking the performance global object gave me a hard time for some reason, it resets itself. Couldn't debug it properly but here's my mock code that works until I or somebody else comes up with a better one.

// jest `native` setup file
import { performance as perf } from 'perf_hooks';

performance.now = jest.fn(() => perf.now());

@yusufyildirim sorry for late reply. This is really interesting. So what you are saying is that we can improve the granularity of React.Profiler info by mocking the performance.now function they way you showed it?

Is there a reliable way to detect if performance.now() is mocked to Date.now()?