You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As the number of solvers has grown, updating benchmarking results has become cumbersome. Ideally, we would have single script to define the solvers and run the set of benchmarks.
Some identified issues:
Currently most solvers are setup in coreax/benchmark_util.py in the initialise_solvers() function. However benchmark/blobs_benchmark.py has separate code for this, which requires defining new methods in both places.
Make the save paths more robust to make sure benchmarking results and images are saved to the appropriate location (currently examples/benchmarking_images) regardless of where the script is run from.
Consider creating a script that contains basic configurations, runs all benchmarking and visualisation scripts, and updates the benchmark.rst documentation page.
Document the benchmarking process and conventions in CONTRIBUTING.md.
Refactor existing code to avoid local imports from other coreax packages.
Make sure the code adheres to the usual standards, e.g., for docstrings and typing.
Investigate optimisation possibilities, e.g., caching some parts of computation, parallelising test runs.
What's the new feature?
As the number of solvers has grown, updating benchmarking results has become cumbersome. Ideally, we would have single script to define the solvers and run the set of benchmarks.
Some identified issues:
coreax/benchmark_util.py
in theinitialise_solvers()
function. Howeverbenchmark/blobs_benchmark.py
has separate code for this, which requires defining new methods in both places.examples/benchmarking_images
) regardless of where the script is run from.benchmark.rst
documentation page.Some suggestions are given in #1000.
What value does this add?
Easier to benchmark new and existing coreset algorithms.
Is there an alternative you've considered?
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: