Skip to content

Commit 3f38f44

Browse files
Mark-SimulacrumMark-Simulacrum
authored
Merge pull request #1560 from pnkfelix/triage-2023-03-21
perf triage report for this week.
2 parents b98127d + c2c2e68 commit 3f38f44

File tree

1 file changed

+272
-0
lines changed

1 file changed

+272
-0
lines changed

‎triage/2023-03-21.md

Lines changed: 272 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,272 @@
1+
# 2023-03-21 Triage Log
2+
3+
A mixed week, with some nice wins, but also at least two PR's that were
4+
subsequently reverted, such as the upgrade to LLVM 16. We do want to
5+
note PR #108944, which cut down on crate metadata, binary sizes, and
6+
was an overall win on execution time for many benchmarks.
7+
8+
Triage done by **@pnkfelix**.
9+
Revision range: [00587489..ef03fda3](https://perf.rust-lang.org/?start=0058748944abb3282aba0e0a74823c6411703565&end=ef03fda339923e659d3d3ca3321de887316d2807&absolute=false&stat=instructions%3Au)
10+
11+
**Summary**:
12+
13+
| (instructions:u) | mean | range | count |
14+
|:----------------------------------:|:-----:|:---------------:|:-----:|
15+
| Regressions ❌ <br /> (primary) | 0.8% | [0.2%, 2.3%] | 31 |
16+
| Regressions ❌ <br /> (secondary) | 1.5% | [0.3%, 3.0%] | 40 |
17+
| Improvements ✅ <br /> (primary) | -1.1% | [-6.7%, -0.2%] | 101 |
18+
| Improvements ✅ <br /> (secondary) | -3.9% | [-16.8%, -0.5%] | 45 |
19+
| All ❌✅ (primary) | -0.6% | [-6.7%, 2.3%] | 132 |
20+
21+
22+
1 Regressions, 4 Improvements, 11 Mixed; 2 of them in rollups
23+
37 artifact comparisons made in total
24+
25+
#### Regressions
26+
27+
Rollup of 10 pull requests [#109206](https://github.com/rust-lang/rust/pull/109206) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=cd6c574af3886c41f34086d90df42c3da0144693&end=7ac4b82ddd596a218cac8cd6b88a91b54fcdcf13&stat=instructions:u)
28+
29+
| (instructions:u) | mean | range | count |
30+
|:----------------------------------:|:----:|:------------:|:-----:|
31+
| Regressions ❌ <br /> (primary) | 0.5% | [0.4%, 0.7%] | 2 |
32+
| Regressions ❌ <br /> (secondary) | 1.1% | [0.9%, 1.3%] | 4 |
33+
| Improvements ✅ <br /> (primary) | - | - | 0 |
34+
| Improvements ✅ <br /> (secondary) | - | - | 0 |
35+
| All ❌✅ (primary) | 0.5% | [0.4%, 0.7%] | 2 |
36+
37+
* as noted on PR, primary regressions are doc benchmarks, and secondary regressions are tt-muncher ... likely being noisy
38+
* marking as triaged.
39+
40+
#### Improvements
41+
42+
Remove identity_future indirection [#104833](https://github.com/rust-lang/rust/pull/104833) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=0058748944abb3282aba0e0a74823c6411703565&end=669e75163957f8f2408d515ce2da3516cb31f747&stat=instructions:u)
43+
44+
| (instructions:u) | mean | range | count |
45+
|:----------------------------------:|:-----:|:---------------:|:-----:|
46+
| Regressions ❌ <br /> (primary) | - | - | 0 |
47+
| Regressions ❌ <br /> (secondary) | - | - | 0 |
48+
| Improvements ✅ <br /> (primary) | -0.5% | [-0.5%, -0.5%] | 1 |
49+
| Improvements ✅ <br /> (secondary) | -3.9% | [-14.3%, -0.9%] | 19 |
50+
| All ❌✅ (primary) | -0.5% | [-0.5%, -0.5%] | 1 |
51+
52+
53+
fix ignore header in MSVC test [#108809](https://github.com/rust-lang/rust/pull/108809) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=18e305dfca6dfef07478378cb9a673fdba6ac3b3&end=cd6c574af3886c41f34086d90df42c3da0144693&stat=instructions:u)
54+
55+
| (instructions:u) | mean | range | count |
56+
|:----------------------------------:|:-----:|:--------------:|:-----:|
57+
| Regressions ❌ <br /> (primary) | - | - | 0 |
58+
| Regressions ❌ <br /> (secondary) | - | - | 0 |
59+
| Improvements ✅ <br /> (primary) | -0.3% | [-0.3%, -0.3%] | 1 |
60+
| Improvements ✅ <br /> (secondary) | -1.0% | [-1.3%, -0.9%] | 6 |
61+
| All ❌✅ (primary) | -0.3% | [-0.3%, -0.3%] | 1 |
62+
63+
64+
fast path for process_obligations [#108815](https://github.com/rust-lang/rust/pull/108815) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=85123d2504885e4670d80dab96f4f9629f2be98b&end=df61fcaec1fdd3b949a7721abfaf37a8ca6e144c&stat=instructions:u)
65+
66+
| (instructions:u) | mean | range | count |
67+
|:----------------------------------:|:------:|:---------------:|:-----:|
68+
| Regressions ❌ <br /> (primary) | 0.5% | [0.5%, 0.5%] | 1 |
69+
| Regressions ❌ <br /> (secondary) | 1.5% | [1.5%, 1.5%] | 1 |
70+
| Improvements ✅ <br /> (primary) | -2.6% | [-8.6%, -0.3%] | 9 |
71+
| Improvements ✅ <br /> (secondary) | -12.7% | [-21.9%, -7.8%] | 6 |
72+
| All ❌✅ (primary) | -2.3% | [-8.6%, 0.5%] | 10 |
73+
74+
75+
Update host compiler to LLVM 16 [#108802](https://github.com/rust-lang/rust/pull/108802) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=f177b7cb91acd9debd1ed974e6521edb2ab6b097&end=2d0a7def330ed90f416a3429dbb44c5ee2a914e5&stat=instructions:u)
76+
77+
| (instructions:u) | mean | range | count |
78+
|:----------------------------------:|:-----:|:--------------:|:-----:|
79+
| Regressions ❌ <br /> (primary) | - | - | 0 |
80+
| Regressions ❌ <br /> (secondary) | - | - | 0 |
81+
| Improvements ✅ <br /> (primary) | -0.7% | [-1.4%, -0.3%] | 38 |
82+
| Improvements ✅ <br /> (secondary) | -0.9% | [-1.7%, -0.6%] | 11 |
83+
| All ❌✅ (primary) | -0.7% | [-1.4%, -0.3%] | 38 |
84+
85+
86+
#### Mixed
87+
88+
Rollup of 9 pull requests [#109130](https://github.com/rust-lang/rust/pull/109130) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=2e7034ebf7f57066c260b680c5e9dfcf04ca4cd5&end=1716932743a7b3705cbf0c34db0c4e070ed1930d&stat=instructions:u)
89+
90+
| (instructions:u) | mean | range | count |
91+
|:----------------------------------:|:-----:|:--------------:|:-----:|
92+
| Regressions ❌ <br /> (primary) | 0.3% | [0.3%, 0.4%] | 6 |
93+
| Regressions ❌ <br /> (secondary) | 0.6% | [0.3%, 1.0%] | 6 |
94+
| Improvements ✅ <br /> (primary) | - | - | 0 |
95+
| Improvements ✅ <br /> (secondary) | -0.5% | [-0.5%, -0.3%] | 5 |
96+
| All ❌✅ (primary) | 0.3% | [0.3%, 0.4%] | 6 |
97+
98+
* the primary regressions are all to stm32f4-0.14.0 incremental (unchanged and patched:negate).
99+
* nothing in the rollup stands out as an obvious culprit. (I was briefly curious about #109101, but I think that indeed should only affect compilation of type-erroneous code, IIUC)
100+
* marking as triaged, because I do not think these results warrant deeper investigation at this time.
101+
102+
103+
remove obsolete givens from regionck [#107376](https://github.com/rust-lang/rust/pull/107376) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1716932743a7b3705cbf0c34db0c4e070ed1930d&end=e84e5ff04a647ce28540300244a26ba120642eea&stat=instructions:u)
104+
105+
| (instructions:u) | mean | range | count |
106+
|:----------------------------------:|:-----:|:--------------:|:-----:|
107+
| Regressions ❌ <br /> (primary) | 0.6% | [0.5%, 0.7%] | 3 |
108+
| Regressions ❌ <br /> (secondary) | - | - | 0 |
109+
| Improvements ✅ <br /> (primary) | - | - | 0 |
110+
| Improvements ✅ <br /> (secondary) | -0.4% | [-0.5%, -0.3%] | 3 |
111+
| All ❌✅ (primary) | 0.6% | [0.5%, 0.7%] | 3 |
112+
113+
* primary benchmark regressions were to bitmaps incr-full { check, debug, opt }
114+
* This was reverted in PR #109183; but the revert PR didn't register analogous reversal of the performance impact here.
115+
* The performance regression to bitmaps incr-full was swallowed by big gains registered by PR #109035
116+
* Marking as triaged.
117+
118+
Ensure ptr::read gets all the same LLVM load metadata that dereferencing does [#109035](https://github.com/rust-lang/rust/pull/109035) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=992d154f3a84cc8abcefcf6e6cf3698e4821b506&end=e4b9f86054c581d931f8bad0c87523c53e1e8e3f&stat=instructions:u)
119+
120+
| (instructions:u) | mean | range | count |
121+
|:----------------------------------:|:-----:|:--------------:|:-----:|
122+
| Regressions ❌ <br /> (primary) | 0.8% | [0.3%, 1.9%] | 7 |
123+
| Regressions ❌ <br /> (secondary) | 1.5% | [0.3%, 2.9%] | 5 |
124+
| Improvements ✅ <br /> (primary) | -0.8% | [-1.3%, -0.3%] | 14 |
125+
| Improvements ✅ <br /> (secondary) | -2.2% | [-3.7%, -0.4%] | 20 |
126+
| All ❌✅ (primary) | -0.2% | [-1.3%, 1.9%] | 21 |
127+
128+
* already triaged, this was a big broad win.
129+
130+
Implement checked Shl/Shr at MIR building. [#108282](https://github.com/rust-lang/rust/pull/108282) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=ab654863c3d50482f260cf862647f1fe0ff5e010&end=c90eb4825a9faca0d6317292a452859f00d5b786&stat=instructions:u)
131+
132+
| (instructions:u) | mean | range | count |
133+
|:----------------------------------:|:-----:|:--------------:|:-----:|
134+
| Regressions ❌ <br /> (primary) | - | - | 0 |
135+
| Regressions ❌ <br /> (secondary) | 1.3% | [1.3%, 1.3%] | 2 |
136+
| Improvements ✅ <br /> (primary) | - | - | 0 |
137+
| Improvements ✅ <br /> (secondary) | -0.9% | [-1.3%, -0.5%] | 8 |
138+
| All ❌✅ (primary) | - | - | 0 |
139+
140+
* already triaged
141+
142+
Flatten/inline format_args!() and (string and int) literal arguments into format_args!() [#106824](https://github.com/rust-lang/rust/pull/106824) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=7ac4b82ddd596a218cac8cd6b88a91b54fcdcf13&end=1203e0866e6c3659775efcb8aecad21dc13ef38b&stat=instructions:u)
143+
144+
| (instructions:u) | mean | range | count |
145+
|:----------------------------------:|:-----:|:--------------:|:-----:|
146+
| Regressions ❌ <br /> (primary) | - | - | 0 |
147+
| Regressions ❌ <br /> (secondary) | 0.7% | [0.3%, 1.1%] | 2 |
148+
| Improvements ✅ <br /> (primary) | -0.7% | [-1.3%, -0.3%] | 4 |
149+
| Improvements ✅ <br /> (secondary) | - | - | 0 |
150+
| All ❌✅ (primary) | -0.7% | [-1.3%, -0.3%] | 4 |
151+
152+
* already triaged
153+
154+
Replace ZST operands and debuginfo by constants. [#107270](https://github.com/rust-lang/rust/pull/107270) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1203e0866e6c3659775efcb8aecad21dc13ef38b&end=e386217dd996d293b3bde2285f6f5f4f502d7c17&stat=instructions:u)
155+
156+
| (instructions:u) | mean | range | count |
157+
|:----------------------------------:|:-----:|:--------------:|:-----:|
158+
| Regressions ❌ <br /> (primary) | 1.2% | [0.4%, 1.9%] | 4 |
159+
| Regressions ❌ <br /> (secondary) | 0.7% | [0.7%, 0.7%] | 1 |
160+
| Improvements ✅ <br /> (primary) | -0.5% | [-0.6%, -0.3%] | 3 |
161+
| Improvements ✅ <br /> (secondary) | - | - | 0 |
162+
| All ❌✅ (primary) | 0.5% | [-0.6%, 1.9%] | 7 |
163+
164+
* already triaged
165+
166+
Wrap the whole LocalInfo in ClearCrossCrate. [#108944](https://github.com/rust-lang/rust/pull/108944) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e386217dd996d293b3bde2285f6f5f4f502d7c17&end=511364e7874dba9649a264100407e4bffe7b5425&stat=instructions:u)
167+
168+
| (instructions:u) | mean | range | count |
169+
|:----------------------------------:|:-----:|:--------------:|:-----:|
170+
| Regressions ❌ <br /> (primary) | 1.6% | [0.3%, 2.4%] | 6 |
171+
| Regressions ❌ <br /> (secondary) | 1.7% | [1.0%, 2.9%] | 16 |
172+
| Improvements ✅ <br /> (primary) | -0.8% | [-1.3%, -0.5%] | 45 |
173+
| Improvements ✅ <br /> (secondary) | -0.8% | [-0.8%, -0.7%] | 5 |
174+
| All ❌✅ (primary) | -0.5% | [-1.3%, 2.4%] | 51 |
175+
176+
177+
* primary instruction count regression was to unicode normalization (5 variations in [1.5%, 2.4%] range). Slightly more muted for cycle count and wall time there.
178+
* as noted by @lqd and @nnethercote on PR comments: crate metadata + binary size improved all around.
179+
* overall this is a win
180+
* marking as triaged.
181+
182+
Upgrade to LLVM 16 [#107224](https://github.com/rust-lang/rust/pull/107224) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=2d0a7def330ed90f416a3429dbb44c5ee2a914e5&end=4a04d086cac54a41517d5657b59d5fe2caca2d71&stat=instructions:u)
183+
184+
| (instructions:u) | mean | range | count |
185+
|:----------------------------------:|:-----:|:--------------:|:-----:|
186+
| Regressions ❌ <br /> (primary) | 1.2% | [0.3%, 3.7%] | 63 |
187+
| Regressions ❌ <br /> (secondary) | 1.1% | [0.2%, 2.7%] | 25 |
188+
| Improvements ✅ <br /> (primary) | -1.1% | [-3.4%, -0.5%] | 49 |
189+
| Improvements ✅ <br /> (secondary) | -1.2% | [-4.3%, -0.3%] | 70 |
190+
| All ❌✅ (primary) | 0.2% | [-3.4%, 3.7%] | 112 |
191+
192+
* instruction count deltas are all over the place
193+
* cycle counts and wall-time seem to paint this PR in a much more positive light
194+
* cycle counts regressions: [ +0.67%, +2.92%] +1.88% 22 (9); improvements: [-10.51%, -1.01%] -2.38% 141 (30)
195+
* wall-time regressions: [ +0.51%, +3.75%] +2.04% 9 (6); improvements: [-10.11%, -0.75%] -2.60% 75 (25)
196+
* marking as triaged, at least with respect to performance impact.
197+
* doesn't really matter, since it was reverted in PR #109326 anyway
198+
199+
200+
Revert "Auto merge of #107224 - nikic:llvm-16, r=cuviper" [#109326](https://github.com/rust-lang/rust/pull/109326) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=4a04d086cac54a41517d5657b59d5fe2caca2d71&end=ab9bb3ea368b2412531a3e8c07ba73d1dd690134&stat=instructions:u)
201+
202+
| (instructions:u) | mean | range | count |
203+
|:----------------------------------:|:-----:|:--------------:|:-----:|
204+
| Regressions ❌ <br /> (primary) | 1.1% | [0.5%, 3.6%] | 51 |
205+
| Regressions ❌ <br /> (secondary) | 1.2% | [0.3%, 4.5%] | 67 |
206+
| Improvements ✅ <br /> (primary) | -1.2% | [-3.6%, -0.3%] | 64 |
207+
| Improvements ✅ <br /> (secondary) | -1.1% | [-2.4%, -0.2%] | 25 |
208+
| All ❌✅ (primary) | -0.2% | [-3.6%, 3.6%] | 115 |
209+
210+
* already triaged. also, see notes for #107224.
211+
212+
Optimize dep node backtrace and ignore fatal errors [#108524](https://github.com/rust-lang/rust/pull/108524) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=356c651e6d013fe9ca1d47da278ba208a95dbcf9&end=13b7aa4d7faafbed78d195c27f6868f6bd90c2f8&stat=instructions:u)
213+
214+
| (instructions:u) | mean | range | count |
215+
|:----------------------------------:|:-----:|:--------------:|:-----:|
216+
| Regressions ❌ <br /> (primary) | 0.4% | [0.4%, 0.4%] | 1 |
217+
| Regressions ❌ <br /> (secondary) | - | - | 0 |
218+
| Improvements ✅ <br /> (primary) | -0.4% | [-0.6%, -0.2%] | 9 |
219+
| Improvements ✅ <br /> (secondary) | -0.3% | [-0.3%, -0.3%] | 1 |
220+
| All ❌✅ (primary) | -0.4% | [-0.6%, 0.4%] | 10 |
221+
222+
* already triaged
223+
224+
Remove the assume(!is_null) from Vec::as_ptr [#106967](https://github.com/rust-lang/rust/pull/106967) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=84c47b8279b39e165dfebeb529eb6d92592e4f8d&end=ef03fda339923e659d3d3ca3321de887316d2807&stat=instructions:u)
225+
226+
| (instructions:u) | mean | range | count |
227+
|:----------------------------------:|:-----:|:--------------:|:-----:|
228+
| Regressions ❌ <br /> (primary) | 0.6% | [0.3%, 1.7%] | 16 |
229+
| Regressions ❌ <br /> (secondary) | 0.7% | [0.2%, 1.4%] | 15 |
230+
| Improvements ✅ <br /> (primary) | -0.8% | [-1.9%, -0.4%] | 4 |
231+
| Improvements ✅ <br /> (secondary) | - | - | 0 |
232+
| All ❌✅ (primary) | 0.3% | [-1.9%, 1.7%] | 20 |
233+
234+
235+
* results from final PR as it finally landed are a bit different (more broad in number of primary benchmarks effected) than the observations from trial performance run
236+
* we expected to see [~2.0%, 2.2%] opt-full regression to regex and serde-derive
237+
* we actually got a [~.7%, ~1.7%] regression to them instead; but a bunch of other primary benchmarks saw a slight dip (less than 0.7% of a regression for the new cases, many less than 0.4%)
238+
* the cycle-counts and wall-times indicate more of an effect on clap and webrender here, but the effect there shows potentially more benefit than harm
239+
* marking as triaged. Its too hard to make strong connections between the changes made here and actual costs/benefits.
240+
241+
#### Untriaged Pull Requests
242+
243+
- [#109206 Rollup of 10 pull requests](https://github.com/rust-lang/rust/pull/109206)
244+
- [#109130 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/109130)
245+
- [#109097 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/109097)
246+
- [#108944 Wrap the whole LocalInfo in ClearCrossCrate.](https://github.com/rust-lang/rust/pull/108944)
247+
- [#108934 Rollup of 8 pull requests](https://github.com/rust-lang/rust/pull/108934)
248+
- [#108872 Strengthen state tracking in const-prop](https://github.com/rust-lang/rust/pull/108872)
249+
- [#108830 Treat projections with infer as placeholder during fast reject in new solver](https://github.com/rust-lang/rust/pull/108830)
250+
- [#108587 Rollup of 10 pull requests](https://github.com/rust-lang/rust/pull/108587)
251+
- [#108025 rustdoc: add more tooltips to intra-doc links](https://github.com/rust-lang/rust/pull/108025)
252+
- [#107833 Factor query arena allocation out from query caches](https://github.com/rust-lang/rust/pull/107833)
253+
- [#107783 rustdoc: simplify DOM for `.item-table`](https://github.com/rust-lang/rust/pull/107783)
254+
- [#107765 rustc/rustdoc: Perform name resolver cleanups enabled by #94857](https://github.com/rust-lang/rust/pull/107765)
255+
- [#107672 Rollup of 3 pull requests](https://github.com/rust-lang/rust/pull/107672)
256+
- [#107408 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/107408)
257+
- [#107376 remove obsolete `givens` from regionck](https://github.com/rust-lang/rust/pull/107376)
258+
- [#107224 Upgrade to LLVM 16](https://github.com/rust-lang/rust/pull/107224)
259+
- [#107143 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/107143)
260+
- [#107103 Use new solver in `evaluate_obligation` query (when new solver is enabled)](https://github.com/rust-lang/rust/pull/107103)
261+
- [#107101 Filter param-env predicates for errors before calling `to_opt_poly_trait_pred`](https://github.com/rust-lang/rust/pull/107101)
262+
- [#106967 Remove the assume(!is_null) from Vec::as_ptr](https://github.com/rust-lang/rust/pull/106967)
263+
- [#106757 Rollup of 10 pull requests](https://github.com/rust-lang/rust/pull/106757)
264+
- [#105657 Guard ProjectionTy creation against passing the wrong number of substs](https://github.com/rust-lang/rust/pull/105657)
265+
- [#105472 Make encode_info_for_trait_item use queries instead of accessing the HIR](https://github.com/rust-lang/rust/pull/105472)
266+
- [#105426 Catch panics/unwinding in destruction of TLS values](https://github.com/rust-lang/rust/pull/105426)
267+
- [#105378 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/105378)
268+
- [#105323 Perform SimplifyLocals before ConstProp.](https://github.com/rust-lang/rust/pull/105323)
269+
- [#105147 Allow unsafe through inline const](https://github.com/rust-lang/rust/pull/105147)
270+
- [#104566 couple of clippy::perf fixes](https://github.com/rust-lang/rust/pull/104566)
271+
- [#104533 Clean up and harden various methods around trait substs](https://github.com/rust-lang/rust/pull/104533)
272+
- [#104017 Rollup of 5 pull requests](https://github.com/rust-lang/rust/pull/104017)

0 commit comments

Comments
 (0)