Skip to content

Commit 4dfa8fa

Browse files
authored
Add Measurement and Scoring details to README.md (#467)
Document in more detail how scoring and workload measurement works: - Add example timeline to explain which parts contribute to the score - Add basic formulas on how the test steps are combined into a final score
1 parent faf9fff commit 4dfa8fa

File tree

3 files changed

+308
-0
lines changed

3 files changed

+308
-0
lines changed

README.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,25 @@ benchmark, actual users should benefit. In order to achieve this, it should:
1414
- Be accessible to the public and useful to browser engineers. It should run in every modern
1515
browser by visiting a normal web page. It should run relatively quickly, while providing
1616
enough test coverage to be reflective of the real-world Web.
17+
18+
# Score and Test Measurement
19+
20+
Each test can contain several steps who contributed to the test duration. Note that the prepare step of a test is _unmeasured_ and thus does not contribute to the score. All following test
21+
steps are **measured** and summed up in the test step time.
22+
23+
<img src="./resources/measurement-timeline.svg" />
24+
25+
Each step consists of the following phases:
26+
27+
<dl>
28+
<dt>Sync:</dt>
29+
<dd>Time spent in synchronous JS execution.</dd>
30+
<dt>Async:</dt>
31+
<dd>Time spent in asynchronous JS like Promise or setTimeout tasks scheduled by the sync-step.</dd>
32+
</dl>
33+
34+
## Score
35+
36+
The final benchmark score is calculated based of the inverse of [geomean](https://en.wikipedia.org/wiki/Geometric_mean) of all tests. We average the score over multiple iterations with the [arithmetic mean](https://en.wikipedia.org/wiki/Arithmetic_mean). By using the geomean to combine the test durations we maintain the invariant that relative improvements are favoured equally amongst all tests, even though they can have vastly different durations.
37+
38+
<img src="./resources/score.svg" />

resources/measurement-timeline.svg

Lines changed: 88 additions & 0 deletions
Loading

resources/score.svg

Lines changed: 198 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)