Skip to content

Efficient use of test262 #6432

Open
Open
@rhuanjl

Description

@rhuanjl

There are currently a few problems with using test262 efficiently on chakracore:

  1. ch does not support monotonicNow (see issue Test262 Agent feature request: $262.agent.monotonicNow() #5271)
  2. I'm unsure if the createRealm method works properly - some tests relating to it fail - though it could be that that's exposing underlying CC issues.
  3. Some spec violations may happen for different reasons in jitted code vs interpreted code, currently running test262 99% of the time it's all going to be in the interpreter.
  4. test262Harness seems the easiest way to run the thing https://github.com/bterlson/test262-harness with a couple of provisos:
    i. I've had to hack in support for running code as a module (it doesn't support this with CC - because it uses a flag that's only in master)
    ii. controlling what tests you run is awkward to do manually - almost feels like it could do with a separate script to manage it, I have an alternative js start file for it offline which allows you to specify the inputs as a JS object instead of command line args; not sure if there's a better way of doing this.
    iii. managing the output is also awkward, I've got an alternate reporter file that compares results before and after a change to CC by saving a baseline but it all feels a bit hacky
    iv. I note that to solve issue 3 above may take some more fundamental edits/may not be feasible in the framework of test262Harness

Solving points 1 and 2 involve working on CC, for the other points I think we could probably do with a CC specific test262 runner script of some kind that can be stored in the tools directory - perhaps written in python (per #6411), something that can:

  1. fetch the latest test262
  2. Optionally Serialise the harness code within test262 as bytecode for fast injection
  3. prepare all the tests
  4. run them - including capturing and using dynamic profile info for a re-run so that they can get run with the fulljit
  5. Store the output AND enable comparison between different versions of the output.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions