Skip to content

8357079: Fix Windows AArch64 DevKit Creation #25259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

8357079: Fix Windows AArch64 DevKit Creation #25259

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion make/devkit/createWindowsDevkit.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,11 @@ REDIST_SUBDIR="VC/Redist/MSVC/$REDIST_VERSION"
echo "Copying VC..."
rm -rf $DEVKIT_ROOT/VC
mkdir -p $DEVKIT_ROOT/VC/bin
cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostx64/arm64" $DEVKIT_ROOT/VC/bin/
if [ -d "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostarm64/arm64" ]; then
cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostarm64/arm64" $DEVKIT_ROOT/VC/bin/
else
cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostx64/arm64" $DEVKIT_ROOT/VC/bin/
fi
Comment on lines +159 to +163
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to make sense to have a single devkit since there seem to be a lot of overlap and not much additional code added to a cross-compiling devkit to also have native binaries.

In fact, I could that not just be done as easily with something like:

Suggested change
if [ -d "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostarm64/arm64" ]; then
cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostarm64/arm64" $DEVKIT_ROOT/VC/bin/
else
cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostx64/arm64" $DEVKIT_ROOT/VC/bin/
fi
if [ -d "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostarm64/arm64" ]; then
cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostarm64/arm64" $DEVKIT_ROOT/VC/bin/
fi
if [ -d "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostx64/arm64" ]; then
cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostx64/arm64" $DEVKIT_ROOT/VC/bin/
fi

Then of course we need to pick up the right set of compilers from the devkit, but I guess that would sort itself out if we know the build platform and are given the target platform as configure argument..?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking we could create the devkit based on the platform where the script is being run. For example, running createWindowsDevkit.sh on:

  • Windows x64 would produce:
    • Native x86 libraries
    • Native x64 libraries
    • Cross-compiled aarch64 libraries
  • Windows aarch64 would produce:
    • Native x86 libraries
    • Native x64 libraries
    • Native aarch64 libraries

From my analysis of your code suggestion, it looks like it might override the native aarch64 libraries and always set the cross-compiled ones. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with your general premise, that the devkit should be created for the platform the devkit script is run on.

When you say that you should produce "native" x64 libraries on aarch64, do you mean that they should be run using x64 emulation? Otherwise you can't really have native x64 on aarch64... And even if that works, would it not be better to have pure native aarch64 cross-compiling to x64 (given that Microsoft in fact ships such compilers).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you download the Visual Studio Build Tools on an ARM64 machine, it downloads the native binaries for the x64 architecture under Hostx64 and the native binaries for arm64 under Hostarm64. This devkit script does not generate any binaries; it simply copies the Build Tools from Visual Studio.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this confusing, but that's probably since I don't understand what Microsoft is shipping and where.

There are 4 relevant compilers:

  1. A compiler that runs on x64 and generates x64 code (x64 native compilation)
  2. A compiler that runs on x64 and generates aarch64 code (aarch64 cross-compilation on x64)
  3. A compiler that runs on aarch64 and generates aarch64 code (aarch64 native compilation)
  4. A compiler that runs on aarch64 and generates x64 code (x64 cross-compilation on aarch64)

For a devkit targeting x64, 1) and 2) should be included. For a devkit targeting aarch64, 3) (and ideally 4) as well) should be included.

Are you saying that if you download VS on aarch64, you will get all four compilers installed?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, if I download VS on aarch64 I get the four compilers installed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the case when downloading VS on x64 as well, that you get a binary that can only be run on aarch64 bundled too? Or is the x64 installation just a subset of the aarch64 installation?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you download VS on x64 you just get cross-compilers for aarch64 (I don't know why). The Hostarm64 directory is not created in x64 machines.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if we run the devkit script on an aarch64 machine, we can create a devkit that will work on both x64 and aarch64? But if we run it on x64, the devkit that we create will only work on x64? Have I got this right?

If so, I think the script should support this, and make it clear what it does. That is, if you run it on aarch64, you get a multi-platform devkit, but on x64 you get a x64-only devkit. I think the devkit script should mention this, ideally in an echo statement, but a comment in the source code is acceptable. Like:
You are generating a devkit for x64 only. To include aarch64 as well, run the script on windows/aarch64 and Generating devkit for x64 and aarch64, respectively. Or something like that.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was planning adding something like this:

# Detect host architecture to determine devkit platform support
# Note: The devkit always includes x86, x64, and aarch64 libraries and tools
# The difference is in toolchain capabilities:
# - On x64|AMD64 hosts: aarch64 tools are cross-compilation tools (Hostx64/arm64)
# - On aarch64|ARMv8 hosts: aarch64 tools are native tools (Hostarm64/arm64)
HOST_ARCH=`echo $PROCESSOR_IDENTIFIER`
case $HOST_ARCH in
    AMD64)
        echo "Running on x64 host - generating devkit with native x86/x64 tools and cross-compiled aarch64 tools."
        echo "For native aarch64 compilation tools, run this script on a Windows/aarch64 machine."
        SUPPORTED_PLATFORMS="x86, x64 (native) and aarch64 (cross-compiled)"
        ;;
    ARMv8)
        echo "Running on aarch64 host - generating devkit with native tools for all platforms (x86, x64, aarch64)."
        SUPPORTED_PLATFORMS="x86, x64, and aarch64 (all native)"
        ;;
    *)
        echo "Unknown host architecture: $HOST_ARCH"
        echo "Proceeding with devkit generation - toolchain capabilities may vary."
        SUPPORTED_PLATFORMS="x86, x64, and aarch64"
        ;;
esac

cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostx64/x64" $DEVKIT_ROOT/VC/bin/
cp -r "$VS_INSTALL_DIR/${VC_SUBDIR}/bin/Hostx86/x86" $DEVKIT_ROOT/VC/bin/
mkdir -p $DEVKIT_ROOT/VC/lib
Expand Down