Flakey tests

[tunetheweb]

These seem to fail most often (the first one the most). Not sure if we can add any more waits or the like to make them less flakey? I've opened a PR #599 to automate the reruns in GitHub Actions.

Chrome:

LCP 1

[0-3] Error in "onLCP().stops reporting after the document changes to hidden (reportAllChanges === false)"
Error: waitUntil condition timed out after 10000ms
at async beaconCountIs (file:///Users/runner/work/web-vitals/web-vitals/test/utils/beacons.js:26:3)
at async Context. (file:///Users/runner/work/web-vitals/web-vitals/test/e2e/onLCP-test.js:283:5)

INP 1

[0-2] AssertionError in "onINP().attribution.includes LoAF entries if the browser supports it"
AssertionError [ERR_ASSERTION]: The expression evaluated to a falsy value:
assert(inp1.attribution.longAnimationFrameEntries.length > 0)
at Context. (file:///Users/runner/work/web-vitals/web-vitals/test/e2e/onINP-test.js:843:7)

INP 2

[chrome 134.0.6998.118 mac #0-2] 1) onINP() reports a new interaction after bfcache restore
[chrome 134.0.6998.118 mac #0-2] Expected values to be strictly equal:

• actual - expected

• 'good'

• 'needs-improvement'
  actual expected

  'goodneeds-improvement'
  

[tunetheweb]

The first waits for readystate of interactive, but that doesn't mean the web-vitals.js script has loaded, nor executed. Which is necessary to ensure the document visibility change is monitored. Going to try changing that to complete in #599

[tunetheweb]

Actually this could also be the issue with test case 2 and 3, since the default Event Timing threshold is 104ms and the tests are often for 100ms. So if the interaction happens before web-vitals.js has loaded and executed, then it may not have reduced to it's default of 40ms and so may miss any 100ms events as not buffered. Changing to await for complete should make them less flakey.

[tunetheweb]

So I see there's logic to add __testImport calls to __readyPromises but I don't think that's enough for two reasons:

1. The script may not have executed before calling interactive to add these to __readyPromises
2. The __testImport awaits until __readyPromises is drained, but is not guaranteed to be the next to execute.
3. Even if all that works, I don't think not guaranteed that the rest of the inline module runs and web-vitals.js is loaded before the other awaits resolve.

Plus there's no real extra delay by wait for complete instead of interactive since, if I'm wrong about the above, then it basically has to wait for this anyway. Plus it's all local stuff so should be quick.

Saying, that. When I changed it from interactive to complete it was still flakey :-(