Skip to content

[Explicit Module Builds][Target Variant] Refactor build planning to allow for a second, separate emit-module task for the target variant #1856

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

artemcm
Copy link
Contributor

@artemcm artemcm commented Mar 28, 2025

This change allows builds that specify a variant triple and which must emit a target variant module to do so from within a single swiftc invocation when using Explicit Module Builds.

This is an extensive refactor of the build planning procedure with respect to Explicit Module Builds:

  • Remove ExplicitDependencyBuildPlanner global state from the Driver instance
  • Instantiate a ExplicitDependencyBuildPlanner instance at the beginning if build planning and propagate/thread this instance to all the required compilation task creation routines, all the way to addCommonFrontendOptions
  • For -target-variant builds which emit a variant triple module, instantiate a whole separate explicit build planner, including performing a secondary dependency scan using the variant triple. Using this information:
    • Schedule variant triple module pre-compile dependency tasks
    • Schedule a variant triple PCH compilation task, if necessary
    • Schedule the variant emit-module task
    • Schedule the variant textual module verification task, if necessary

@artemcm
Copy link
Contributor Author

artemcm commented Mar 28, 2025

@swift-ci test

@artemcm artemcm force-pushed the VariantEmitModule branch from 6d351de to 95c6c6d Compare April 1, 2025 21:12
@artemcm
Copy link
Contributor Author

artemcm commented Apr 1, 2025

@swift-ci test

@artemcm artemcm force-pushed the VariantEmitModule branch from 95c6c6d to e1c9db6 Compare April 1, 2025 22:21
@artemcm
Copy link
Contributor Author

artemcm commented Apr 1, 2025

@swift-ci test

@artemcm artemcm force-pushed the VariantEmitModule branch from e1c9db6 to 16688ae Compare April 1, 2025 22:38
@artemcm
Copy link
Contributor Author

artemcm commented Apr 1, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Apr 1, 2025

@swift-ci test Windows platform

1 similar comment
@artemcm
Copy link
Contributor Author

artemcm commented Apr 2, 2025

@swift-ci test Windows platform

@artemcm artemcm force-pushed the VariantEmitModule branch 2 times, most recently from 80d34fb to 5d7f59a Compare April 16, 2025 22:51
@artemcm
Copy link
Contributor Author

artemcm commented Apr 16, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Apr 16, 2025

@swift-ci test Windows platform

1 similar comment
@artemcm
Copy link
Contributor Author

artemcm commented Apr 16, 2025

@swift-ci test Windows platform

@artemcm artemcm force-pushed the VariantEmitModule branch from 5d7f59a to 94c621f Compare April 17, 2025 16:06
@artemcm
Copy link
Contributor Author

artemcm commented Apr 17, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Apr 18, 2025

@swift-ci test Windows platform

@artemcm artemcm force-pushed the VariantEmitModule branch from 94c621f to e47991f Compare April 18, 2025 22:40
@artemcm
Copy link
Contributor Author

artemcm commented Apr 18, 2025

@swift-ci test

@artemcm artemcm marked this pull request as ready for review April 18, 2025 22:41
@artemcm artemcm requested a review from etcwilde April 18, 2025 22:41
@artemcm
Copy link
Contributor Author

artemcm commented Apr 18, 2025

@swift-ci test Windows platform

@artemcm artemcm force-pushed the VariantEmitModule branch from e47991f to 6010b76 Compare April 22, 2025 17:49
@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test Windows platform

@artemcm artemcm force-pushed the VariantEmitModule branch from 6010b76 to bdf0ec6 Compare April 22, 2025 18:25
@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test
@swift-ci test Windows platform

@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test Windows platform

@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test

@artemcm artemcm force-pushed the VariantEmitModule branch from bdf0ec6 to ae75bc0 Compare April 22, 2025 18:35
@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test Windows platform

@artemcm artemcm force-pushed the VariantEmitModule branch from ae75bc0 to 8ea30fb Compare April 22, 2025 19:34
@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test Windows platform

@artemcm artemcm force-pushed the VariantEmitModule branch from 8ea30fb to 4049481 Compare April 22, 2025 20:01
@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test Windows platform

@artemcm artemcm force-pushed the VariantEmitModule branch from 4049481 to f8f26ff Compare April 22, 2025 20:17
@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test Linux platform

@artemcm artemcm force-pushed the VariantEmitModule branch 2 times, most recently from 8978aee to d1b112b Compare April 22, 2025 21:23
@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test Windows platform

1 similar comment
@artemcm
Copy link
Contributor Author

artemcm commented Apr 22, 2025

@swift-ci test Windows platform

@artemcm artemcm requested a review from owenv April 22, 2025 21:32
@artemcm
Copy link
Contributor Author

artemcm commented May 14, 2025

Ping for code review.

Copy link
Contributor

@cachemeifyoucan cachemeifyoucan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the inline comment, if you are not squashing, MergeModule is not deprecated. It is obsoleted and deleted.

if driver.parsedOptions.contains(.targetVariant),
driver.isFrontendArgSupported(.clangTargetVariant),
let targetVariantTripleStr = frontendTargetInfo.targetVariant?.triple {
var clangTargetVariantTriple: String? = nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indentation is really weird here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed up best I can. 👍🏼

let sdkPath = frontendTargetInfo.sdkPath?.path
let sdkInfo: DarwinSDKInfo? = sdkPath != nil ? getTargetSDKInfo(sdkPath: sdkPath!) : nil

// Pass down -clang-target.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make sure we have test for all -clang-target cases? I think still missing:

  • Only pass -clang-target
  • Not passing -clang-target-variant but specify -target-variant.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added testClangTargetOptionsExplicit.

@@ -47,6 +47,9 @@ public enum VirtualPath: Hashable {
/// Standard output
case standardOutput

/// ACTODO: Comment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix TODO

if !fileSystem.exists(path.parentDirectory) {
try fileSystem.createDirectory(path.parentDirectory, recursive: true)
}
return try VirtualPath.createBuildProductFileWithKnownContents(path, chainedHeader.content.data(using: .utf8)!).intern()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the force unwrap here.

@@ -15,7 +15,8 @@ extension Driver {
var commandLine: [Job.ArgTemplate] = swiftCompilerPrefixArgs.map { Job.ArgTemplate.flag($0) }
var inputs: [TypedVirtualPath] = []

try addCommonFrontendOptions(commandLine: &commandLine, inputs: &inputs, kind: .repl)
try addCommonFrontendOptions(commandLine: &commandLine, inputs: &inputs, kind: .repl,
explicitModulePlanner: nil)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just the default parameter.

artemcm added 4 commits May 15, 2025 12:28
…llow for a second, separte emit-module task for the target variant

This change allows builds that specify a variant triple and which must emit a target variant module to do so from within a single `swiftc` invocation when using Explicit Module Builds.

This is an extensive refactor of the build planning procedure with respect to Explicit Module Builds:
- Remove 'ExplicitDependencyBuildPlanner' global state from the 'Driver' instance
- Instantiate a 'ExplicitDependencyBuildPlanner' instance at the beginning if build planning and propagate/thread this instance to all the required compilation task creation routines, all the way to 'addCommonFrontendOptions'
- For '-target-variant' builds which emit a variant triple module, instantiate a whole separate explicit build planner, including performing a secondary dependency scan using the variant triple. Using this information:
  - Schedule variant triple module pre-compile dependency tasks
  - Schedule a variant triple PCH compilation task, if necessary
  - Schedule the variant emit-module task
  - Schedule the variant textual module verification task, if necessary
…th known contents

Similarly to a 'temporaryWithKnownContents', but at an explicitly-determined-by-the-driver 'AbsolutePath'. This is useful for when the driver is able to determine that build planning will produce an input to compilation which must be placed in a specific location. For example, the chained bridging header content and output path are determined during build planning (specifically, dependency scanning) and must be fed into compilation tasks as inputs. This new file type will allow the driver to compute the new file and its contents and then have them get written to disk when resolving input paths to said compilation tasks.
@artemcm artemcm force-pushed the VariantEmitModule branch from d1b112b to 91811bb Compare May 15, 2025 20:42
@artemcm
Copy link
Contributor Author

artemcm commented May 15, 2025

@swift-ci test

@artemcm
Copy link
Contributor Author

artemcm commented May 15, 2025

@swift-ci test Windows platform

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants