Incremental compilation produces runtime crash

[mbrandonw]

Describe the bug

It is possible to make a program that when compiled and run works correctly, but then if a small change is made and run again it will consume a large amount of memory and then crash at runtime.

Further, if the target is cleaned and run again it works fine, which makes me think it is a bug with incremental compilation.

This happens in Swift 5.5-5.7 and perhaps earlier.

Steps To Reproduce
Steps to reproduce the behavior:

1. Open the demo project: incremental-bug.zip
2. Navigate to Feature.swift
3. Run the application and observe that it completes successfully.
4. Uncomment the 3 lines in Feature.swift and run again.
5. Observe that the application consume ~1.5GB of memory and then crashes.
6. Clean the target (cmd+shift+K) and run again to observe that it completes successfully again.

Expected behavior
The application should not crash and should print the following to the console:

Sending .action1("Hello")
Running  action1("Hello")
Sending .action2
Running  action2

Environment (please fill out the following information)

• OS: macOS 12.4 (21F79)
• Xcode 13.2.1 (13C100) / Swift 5.5

[mbrandonw]

It's worth noting that I have found a few ways of working around this runtime crash:

• Adding another case to the enum with an associated type, such as case action3(Int).
• Putting all of Feature.swift's contents in IncrementBug.swift
• Extracting the Action enum to be defined at the file scope rather than the Feature struct.

In all of those situations the incremental compilation of the application somehow does not cause the runtime crash.

[CodaFi]

This is a fantastic issue, thank you for reducing the problem so well. The package manager is at least reporting that it's rebuilding Feature.swift during all of this which is good. There is a named reference to the Feature struct, but no direct reference to Feature.Action in IncrementalBug.swift and if that file is touched and rebuilt as well then this bug doesn't reproduce.

At first blush the bug is that Store(Feature()) should imply a potential member dependency on the exact witness to P.A supplied by Feature - which is Feature.Action. This is corroborated by a diff of the .os in the baseline and busted cases. The busted case sees the old layout of Feature.Action - which has trivial layout since it's just the single action2 case, even though the addition of action1 selects a completely different layout and makes it a non-trivial type with a retainable pointer inside for the String.

Working:

_$s15incremental_bug14IncrementalBugV4mainyyFZAA7FeatureV6ActionOAHcfU_:
00000000000000fc	sub	sp, sp, #0x40
0000000000000100	stp	x29, x30, [sp, #0x30]
0000000000000104	add	x29, sp, #0x30
0000000000000108	str	x8, [sp, #0x10]
000000000000010c	stur	xzr, [x29, #-0x10]
0000000000000110	stur	xzr, [x29, #-0x8]
0000000000000114	ldr	x8, [x0]
0000000000000118	str	x8, [sp, #0x8]
000000000000011c	ldr	x0, [x0, #0x8]
0000000000000120	str	x0, [sp, #0x18]
0000000000000124	stur	x8, [x29, #-0x10]
0000000000000128	stur	x0, [x29, #-0x8]
000000000000012c	bl	_swift_bridgeObjectRetain
0000000000000130	ldr	x10, [sp, #0x8]
0000000000000134	ldr	x9, [sp, #0x10]
0000000000000138	ldr	x8, [sp, #0x18]
000000000000013c	str	x10, [x9]
0000000000000140	str	x8, [x9, #0x8]
0000000000000144	ldp	x29, x30, [sp, #0x30]
0000000000000148	add	sp, sp, #0x40
000000000000014c	ret

Busted:

_$s15incremental_bug14IncrementalBugV4mainyyFZAA7FeatureV6ActionOAHcfU_:
00000000000000fc	ret

[CodaFi]

This reproduces all the way back in Xcode 13 albeit less spectacularly since it only eats a few megs before crashing trying to render the debug description of the zombie String enum payload.

[CodaFi]

I suspect this bug has been around for... basically since the incremental build was written. < Xcode 13 did not run its dependency tracking machinery past semantic analysis, and name lookups alone are not enough to actually net this dependency. We would (and still) need to record that the closure depends on the layout of Feature.Action, which is something only >= SILGen can do.

[mbrandonw]

FWIW we've had a few people run into this incremental compilation crash in our library: pointfreeco/swift-composable-architecture#1564.

[lukeredpath]

This issue is a constant source of frustration - we use the composable architecture and every time I change an associated value type (or add one) to one of our action enums, the app will crash and I will need to do a clean build to resolve it.