[DNM] Starkware Development

[JulianGCalderon]

A draft PR to track progress on the starkware-development branch.

[github-actions]

**Hyper Thereading Benchmark results**




hyperfine -r 2 -n "hyper_threading_main threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_main' -n "hyper_threading_pr threads: 1" 'RAYON_NUM_THREADS=1 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 1
  Time (mean ± σ):     26.280 s ±  0.018 s    [User: 25.532 s, System: 0.745 s]
  Range (min … max):   26.267 s … 26.293 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 1
  Time (mean ± σ):     25.721 s ±  0.101 s    [User: 24.869 s, System: 0.848 s]
  Range (min … max):   25.650 s … 25.792 s    2 runs
 
Summary
  hyper_threading_pr threads: 1 ran
    1.02 ± 0.00 times faster than hyper_threading_main threads: 1




hyperfine -r 2 -n "hyper_threading_main threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_main' -n "hyper_threading_pr threads: 2" 'RAYON_NUM_THREADS=2 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 2
  Time (mean ± σ):     14.635 s ±  0.009 s    [User: 25.638 s, System: 0.811 s]
  Range (min … max):   14.628 s … 14.642 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 2
  Time (mean ± σ):     13.949 s ±  0.018 s    [User: 24.848 s, System: 0.861 s]
  Range (min … max):   13.936 s … 13.961 s    2 runs
 
Summary
  hyper_threading_pr threads: 2 ran
    1.05 ± 0.00 times faster than hyper_threading_main threads: 2




hyperfine -r 2 -n "hyper_threading_main threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_main' -n "hyper_threading_pr threads: 4" 'RAYON_NUM_THREADS=4 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 4
  Time (mean ± σ):     10.955 s ±  0.233 s    [User: 38.393 s, System: 0.993 s]
  Range (min … max):   10.790 s … 11.120 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 4
  Time (mean ± σ):     10.491 s ±  0.017 s    [User: 36.982 s, System: 1.055 s]
  Range (min … max):   10.479 s … 10.503 s    2 runs
 
Summary
  hyper_threading_pr threads: 4 ran
    1.04 ± 0.02 times faster than hyper_threading_main threads: 4




hyperfine -r 2 -n "hyper_threading_main threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_main' -n "hyper_threading_pr threads: 6" 'RAYON_NUM_THREADS=6 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 6
  Time (mean ± σ):     10.845 s ±  0.143 s    [User: 38.808 s, System: 0.970 s]
  Range (min … max):   10.744 s … 10.946 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 6
  Time (mean ± σ):     10.155 s ±  0.101 s    [User: 37.122 s, System: 1.097 s]
  Range (min … max):   10.084 s … 10.226 s    2 runs
 
Summary
  hyper_threading_pr threads: 6 ran
    1.07 ± 0.02 times faster than hyper_threading_main threads: 6




hyperfine -r 2 -n "hyper_threading_main threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_main' -n "hyper_threading_pr threads: 8" 'RAYON_NUM_THREADS=8 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 8
  Time (mean ± σ):     10.701 s ±  0.138 s    [User: 39.234 s, System: 1.011 s]
  Range (min … max):   10.604 s … 10.799 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 8
  Time (mean ± σ):     10.221 s ±  0.166 s    [User: 37.478 s, System: 1.103 s]
  Range (min … max):   10.103 s … 10.338 s    2 runs
 
Summary
  hyper_threading_pr threads: 8 ran
    1.05 ± 0.02 times faster than hyper_threading_main threads: 8




hyperfine -r 2 -n "hyper_threading_main threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_main' -n "hyper_threading_pr threads: 16" 'RAYON_NUM_THREADS=16 ./hyper_threading_pr'
Benchmark 1: hyper_threading_main threads: 16
  Time (mean ± σ):     10.521 s ±  0.090 s    [User: 39.355 s, System: 1.090 s]
  Range (min … max):   10.458 s … 10.585 s    2 runs
 
Benchmark 2: hyper_threading_pr threads: 16
  Time (mean ± σ):     10.149 s ±  0.136 s    [User: 37.800 s, System: 1.150 s]
  Range (min … max):   10.052 s … 10.245 s    2 runs
 
Summary
  hyper_threading_pr threads: 16 ran
    1.04 ± 0.02 times faster than hyper_threading_main threads: 16

[github-actions]

Benchmark Results for unmodified programs 🚀

Command	Mean [s]	Min [s]	Max [s]	Relative
base big_factorial	2.126 ± 0.015	2.112	2.158	1.00
head big_factorial	2.162 ± 0.007	2.151	2.174	1.02 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base big_fibonacci	2.051 ± 0.012	2.036	2.074	1.00
head big_fibonacci	2.096 ± 0.016	2.078	2.135	1.02 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base blake2s_integration_benchmark	7.675 ± 0.333	7.490	8.533	1.00
head blake2s_integration_benchmark	7.698 ± 0.060	7.607	7.787	1.00 ± 0.04

Command	Mean [s]	Min [s]	Max [s]	Relative
base compare_arrays_200000	2.180 ± 0.007	2.168	2.192	1.00
head compare_arrays_200000	2.223 ± 0.008	2.212	2.241	1.02 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base dict_integration_benchmark	1.444 ± 0.005	1.438	1.456	1.00
head dict_integration_benchmark	1.447 ± 0.018	1.435	1.496	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base field_arithmetic_get_square_benchmark	1.212 ± 0.004	1.204	1.218	1.00
head field_arithmetic_get_square_benchmark	1.240 ± 0.010	1.232	1.266	1.02 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base integration_builtins	7.580 ± 0.020	7.542	7.611	1.00
head integration_builtins	7.718 ± 0.031	7.657	7.753	1.02 ± 0.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base keccak_integration_benchmark	7.876 ± 0.087	7.809	8.108	1.00
head keccak_integration_benchmark	7.880 ± 0.073	7.795	8.065	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base linear_search	2.169 ± 0.040	2.139	2.279	1.00
head linear_search	2.199 ± 0.022	2.181	2.251	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
base math_cmp_and_pow_integration_benchmark	1.523 ± 0.016	1.513	1.568	1.00
head math_cmp_and_pow_integration_benchmark	1.540 ± 0.024	1.529	1.608	1.01 ± 0.02

Command	Mean [s]	Min [s]	Max [s]	Relative
base math_integration_benchmark	1.467 ± 0.011	1.455	1.493	1.00
head math_integration_benchmark	1.468 ± 0.008	1.460	1.489	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base memory_integration_benchmark	1.223 ± 0.012	1.211	1.241	1.00
head memory_integration_benchmark	1.228 ± 0.003	1.223	1.231	1.00 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base operations_with_data_structures_benchmarks	1.584 ± 0.024	1.564	1.634	1.01 ± 0.02
head operations_with_data_structures_benchmarks	1.561 ± 0.015	1.549	1.601	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base pedersen	530.9 ± 1.0	529.6	533.0	1.00
head pedersen	537.0 ± 2.6	534.7	543.2	1.01 ± 0.01

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base poseidon_integration_benchmark	629.0 ± 7.2	621.7	642.6	1.01 ± 0.01
head poseidon_integration_benchmark	624.8 ± 2.9	621.3	629.1	1.00

Command	Mean [s]	Min [s]	Max [s]	Relative
base secp_integration_benchmark	1.879 ± 0.016	1.865	1.907	1.02 ± 0.01
head secp_integration_benchmark	1.846 ± 0.007	1.836	1.861	1.00

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
base set_integration_benchmark	625.5 ± 2.1	623.3	630.3	1.00
head set_integration_benchmark	633.8 ± 4.2	628.9	645.0	1.01 ± 0.01

Command	Mean [s]	Min [s]	Max [s]	Relative
base uint256_integration_benchmark	4.195 ± 0.013	4.176	4.218	1.00
head uint256_integration_benchmark	4.337 ± 0.032	4.309	4.422	1.03 ± 0.01

[codecov]

Codecov Report

Attention: Patch coverage is 94.25000% with 23 lines in your changes missing coverage. Please review.

Project coverage is 96.65%. Comparing base (0038711) to head (a5b3c9d).
Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
cairo-vm-cli/src/main.rs	63.63%	8 Missing ⚠️
vm/src/vm/runners/cairo_runner.rs	88.52%	7 Missing ⚠️
vm/src/vm/vm_core.rs	87.50%	7 Missing ⚠️
vm/src/cairo_run.rs	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2076      +/-   ##
==========================================
+ Coverage   96.55%   96.65%   +0.09%     
==========================================
  Files         102      104       +2     
  Lines       42872    43896    +1024     
==========================================
+ Hits        41394    42426    +1032     
+ Misses       1478     1470       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.