Skip to content

Add complexity param #430

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 31 commits into
base: main
Choose a base branch
from

Conversation

camillobruni
Copy link
Contributor

@camillobruni camillobruni commented Oct 9, 2024

Depends on params.mjs changes in #469

Work in progress proposal for adding a variable complexity param.

  • complexity === 1.0: workloads run in default configuration
  • complexity > 1.0: ideally workloads run proportionally slower
  • complexity < 1.0: ideally workloads run proportionally fast

Background:
We're occasionally seeing issues with gc timing / optimisations heuristics with the current workloads. If we can dynamically change the duration / complexity of workloads it's a bit easier to assess how good these heuristics are and how much we're potentially overfitting.

Current Limitations:

  • DeveloperMode UI cannot reload tests.mjs causing stale BenchmarkTestStep titles
  • Only TodoMVC are currently scaling the workload size (other workloads are depend on whether we're fine with the general idea here)

@camillobruni camillobruni requested a review from julienw October 9, 2024 14:07
@camillobruni camillobruni marked this pull request as draft October 9, 2024 14:08
@camillobruni camillobruni requested a review from rniwa October 9, 2024 14:08
@camillobruni camillobruni changed the title WIP: Add complexity param Add complexity param Oct 9, 2024
@bgrins
Copy link
Contributor

bgrins commented Oct 10, 2024

Without looking closely at the code change, I think this is a neat idea and I could imagine us increasing the complexity of charts, text editing, etc if we move forward with it. Will be curious for feedback from @julienw.

Copy link
Contributor

@julienw julienw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea :-)

@@ -101,11 +102,21 @@ function createUIForSyncStepDelay() {
return label;
}

function createTimeRangeUI(labelText, initialValue, unit = "ms", min = 0, max = 1000) {
function createUIForComplexity() {
const { range, label } = createTimeRangeUI("Relative complexity", params.complexity, "x", 0, 10, 0.01);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that rather than a float, maybe this can be an integer between 1 and 100, with 50 being the default?

Then later you can divide this number by 50 to get the multiplier.
And you can cut down a lot of the changes.

But this is merely a suggestion, I'm also happy with what you did.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kinda prefer a normalized float value, aiming quite literaly at a complexity-factor, 2x => it should run roughly twice as slow.

@camillobruni
Copy link
Contributor Author

camillobruni commented Oct 11, 2024

  • Only create new Params object if there are URL search params
  • Added table to highlight non-standard params if there are is a default to the defaultParams object
Screenshot 2024-10-11 at 14 15 17

@camillobruni camillobruni marked this pull request as ready for review December 10, 2024 19:10
@camillobruni
Copy link
Contributor Author

I've now changed the complexity slider to have an exponential scale which makes it a bit more comfortable to use.

Copy link

netlify bot commented May 21, 2025

Deploy Preview for webkit-speedometer-preview ready!

Name Link
🔨 Latest commit 52d1154
🔍 Latest deploy log https://app.netlify.com/projects/webkit-speedometer-preview/deploys/683eb6474c69b600089c0846
😎 Deploy Preview https://deploy-preview-430--webkit-speedometer-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@camillobruni camillobruni requested review from julienw and rniwa May 21, 2025 12:41
Copy link
Contributor

@julienw julienw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

can we also add the complexity value to the json?

return label;
}

function createExpRangeUI(labelText, paramKey, unit = "ms", min = 0, max = 1000, step = 1) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was a bit afraid of the accessibility, but this looks mostly OK.
Only the accessibility name on the <label> itself is a bit off, but on the slider this looks fine, so I think we're pretty good. I'm not sure why the slider's label is fine but not the label's label...

@camillobruni
Copy link
Contributor Author

We should probably log all non-standard params to the json file.
let me do that in a separate PR. In our infra we use the json file to populate perf data.

@camillobruni camillobruni requested a review from julienw May 28, 2025 08:05
Copy link
Contributor

@julienw julienw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you, looks good to me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants