introduce --fuzz=<limit> to make tests pass after a certain quantity of fuzz testing has completed #22900
Labels
enhancement
Solving this issue will likely involve adding new logic or components to the codebase.
fuzzing
Milestone
Uh oh!
There was an error while loading. Please reload this page.
There should be two modes of fuzzing, infinite mode, and finite mode. This issue is for finite mode.
Introduce
--fuzz=<limit>
, where the limit can be one of these:1s
,2m
,3h
,4d
100
,100K
,100M
,100G
Time-based fuzzing makes all fuzz tests finish successfully after the specified interval has passed. The time is global; adding more fuzz tests and keeping the time the same means less time spent on each fuzz test.
Iteration count based fuzzing makes a given fuzz test finish successfully after that many different inputs have been tried. The number of iterations is per test, so adding more fuzz tests while keeping the same iteration count means it will take longer to finish successfully, since the new tests will be also given the iterations.
Open questions:
zig build
default to--fuzz=1K
or similar when fuzz tests are found? If this is set to a reasonably low value, it could be quite valuable in finding shallow bugs quickly. It also opens the door for reporting the seed rather than input and having developers reproduce the failure easily since the iteration count was low.The text was updated successfully, but these errors were encountered: