cross build --target aarch64-pc-windows-msvc fails with error 53

[mominul]

Checklist

• I've looked through the issues and pull requests for similar reports

Describe your issue

I have built the ghcr.io/cross-rs/aarch64-pc-windows-msvc-cross image with cross-toolchains. Here is the output of the image build:

     Running `target/debug/xtask build-docker-image aarch64-pc-windows-msvc-cross --tag local`
[cross] note: Build aarch64-pc-windows-msvc-cross for x86_64-unknown-linux-gnu
[+] Building 5733.1s (22/22) FINISHED          docker:desktop-linux
 => [internal] load build definition from Dockerfile.aarch64-  0.0s
 => => transferring dockerfile: 1.11kB                         0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.  8.7s
 => [internal] load .dockerignore                              0.0s
 => => transferring context: 260B                              0.0s
 => ERROR importing cache manifest from ghcr.io/cross-rs/aarc  1.3s
 => [ 1/16] FROM docker.io/library/ubuntu:20.04@sha256:0b897  25.2s
 => => resolve docker.io/library/ubuntu:20.04@sha256:0b897358  0.0s
 => => sha256:5f5250218d28ad6612bf653eced4071 2.30kB / 2.30kB  0.0s
 => => sha256:9ea8908f47652b59b8055316d9c0 27.51MB / 27.51MB  24.4s
 => => sha256:0b897358ff6624825fb50d20ffb605a 1.13kB / 1.13kB  0.0s
 => => sha256:d86db849e59626d94f768c679aba441163c 424B / 424B  0.0s
 => => extracting sha256:9ea8908f47652b59b8055316d9c0e16b365e  0.6s
 => [internal] load build context                              0.0s
 => => transferring context: 10.87kB                           0.0s
 => [ 2/16] COPY common.sh lib.sh /                            0.1s
 => [ 3/16] RUN /common.sh                                   520.4s
 => [ 4/16] COPY cmake.sh /                                    0.0s
 => [ 5/16] RUN /cmake.sh                                    369.8s
 => [ 6/16] COPY xargo.sh /                                    0.0s 
 => [ 7/16] RUN /xargo.sh                                    371.7s 
 => [ 8/16] COPY cross-toolchains/docker/msvc-wine.sh /        0.0s 
 => [ 9/16] RUN /msvc-wine.sh                               3483.0s
 => [10/16] COPY wine.sh /                                     0.0s
 => [11/16] RUN /wine.sh                                     652.9s
 => [12/16] COPY cross-toolchains/docker/perl.sh /             0.0s
 => [13/16] RUN /perl.sh                                     278.4s
 => [14/16] COPY cross-toolchains/docker/msvc-wine-symlink.sh  0.0s
 => [15/16] RUN /msvc-wine-symlink.sh arm64                    0.3s
 => [16/16] COPY cross-toolchains/docker/msvc-windows-entry.s  0.0s
 => exporting to image                                        21.0s
 => => exporting layers                                       21.0s
 => => writing image sha256:ef7aa3b437a146390f4df69a91ca6d888  0.0s
 => => naming to ghcr.io/cross-rs/aarch64-pc-windows-msvc-cro  0.0s
------
 > importing cache manifest from ghcr.io/cross-rs/aarch64-pc-windows-msvc-cross:main:
------

Here is the screenshot from docker desktop:

and output from docker image:

REPOSITORY                                       TAG       IMAGE ID       CREATED        SIZE
ghcr.io/cross-rs/aarch64-pc-windows-msvc-cross   local     ef7aa3b437a1   11 hours ago   9.43GB

and the Cross.toml:

[target.aarch64-pc-windows-msvc]
image = "ghcr.io/cross-rs/aarch64-pc-windows-msvc-cross:local"

For the first time I used cross build --target aarch64-pc-windows-msvc it errored out with 125 and this output:

info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
info: latest update on 2024-05-02, rust version 1.78.0 (9b00956e5 2024-04-29)
info: downloading component 'cargo'
  8.0 MiB /   8.0 MiB (100 %)   1.7 MiB/s in  4s ETA:  0s
info: downloading component 'rust-std'
 24.3 MiB /  24.3 MiB (100 %)   1.9 MiB/s in 13s ETA:  0s
info: downloading component 'rustc'
 63.7 MiB /  63.7 MiB (100 %)   2.2 MiB/s in 34s ETA:  0s
info: installing component 'cargo'
info: installing component 'rust-std'
 24.3 MiB /  24.3 MiB (100 %)  18.7 MiB/s in  1s ETA:  0s
info: installing component 'rustc'
 63.7 MiB /  63.7 MiB (100 %)  20.3 MiB/s in  3s ETA:  0s

  stable-x86_64-unknown-linux-gnu installed - (error reading rustc version)

info: checking for self-update
info: downloading component 'rust-std' for 'aarch64-pc-windows-msvc'
info: installing component 'rust-std' for 'aarch64-pc-windows-msvc'
Unable to find image 'aarch64-pc-windows-msvc-cross:local' locally
docker: Error response from daemon: pull access denied for aarch64-pc-windows-msvc-cross, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

Afterwards I restarted the docker desktop, and now everytime I run cross build --target aarch64-pc-windows-msvc it gives no output but errors out with 53 error number and a target directory with just a CACHEDIR.TAG file.

What target(s) are you cross-compiling for?

other (specify in description)

Which operating system is the host (e.g computer cross is on) running?

• macOS
• Windows
• Linux / BSD
• other OS (specify in description)

What architecture is the host?

• x86_64 / AMD64
• arm32
• arm64 (including Mac M1)

What container engine is cross using?

• docker
• podman
• other container engine (specify in description)

cross version

cross 0.2.5 (19be834 2024-05-17)

Example

No response

Additional information / notes

No response

[Emilgardis]

The first problem you describe is an issue with compat between docker-desktop and buildx. There's an option in docker-desktop for macos that has caused issues previously for other people (although it has worked for me), "Use containerd for pulling and storing images"

Should be solved by building with CROSS_BUILD_OPTS="--output=type=docker".

For the second issue, I require some more information. Can you post the output together with adding -v to the command?

[mominul]

The first problem you describe is an issue with compat between docker-desktop and buildx. There's an option in docker-desktop for macos that has caused issues previously for other people (although it has worked for me), "Use containerd for pulling and storing images"

This was already disabled by default, did you meant to enable it?

Should be solved by building with CROSS_BUILD_OPTS="--output=type=docker".

For the second issue, I require some more information. Can you post the output together with adding -v to the command?

Here is the output of that command, I don't know why it is inquiring rustup about stable-x86_64-unknown-linux-gnu target.

mominul@Muhammads-MacBook-Air ~/s/tsf-example (main)> CROSS_BUILD_OPTS="--output=type=docker" cross build -v --target aarch64-pc-windows-msvc
+ cargo metadata --format-version 1 --filter-platform aarch64-pc-windows-msvc
+ rustc --print sysroot
+ /usr/local/bin/docker
+ /usr/local/bin/docker version -f '{{ .Server.Os }},,,{{ .Server.Arch }}'
+ /usr/local/bin/docker info -f {{.SecurityOptions}}
+ rustup toolchain list
+ rustup target list --toolchain stable-x86_64-unknown-linux-gnu
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu
+ /usr/local/bin/docker run --userns host --platform linux/amd64 -e 'XARGO_HOME=/Users/mominul/.xargo' -e 'CARGO_HOME=/Users/mominul/.cargo' -e 'CROSS_RUST_SYSROOT=/Users/mominul/.rustup/toolchains/stable-x86_64-unknown-linux-gnu' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e BROWSER -e CROSS_BUILD_OPTS -e TERM -e 'USER=mominul' -e 'CROSS_RUSTC_MAJOR_VERSION=1' -e 'CROSS_RUSTC_MINOR_VERSION=78' -e 'CROSS_RUSTC_PATCH_VERSION=0' --name cross-stable-x86_64-unknown-linux-gnu-da827-9b00956e5-aarch64-pc-windows-msvc-80a3f-1717764155689 --rm --user 501:20 -v /Users/mominul/.xargo:/Users/mominul/.xargo:z -v /Users/mominul/.cargo:/Users/mominul/.cargo:z -v /Users/mominul/.cargo/bin -v /Users/mominul/src/tsf-example:/Users/mominul/src/tsf-example:z -v /Users/mominul/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/Users/mominul/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:z,ro -v /Users/mominul/src/tsf-example/target:/target:z -w /Users/mominul/src/tsf-example -t ghcr.io/cross-rs/aarch64-pc-windows-msvc-cross:local sh -c 'PATH="$PATH":"/Users/mominul/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin" cargo build -v --target aarch64-pc-windows-msvc'
+ rustup component list --toolchain stable-x86_64-unknown-linux-gnu

Thanks for the fast reply!

[Emilgardis]

the toolchain x86_64-unknown-linux-gnu is what is used to compile for the chosen target.

note: I only suggested CROSS_BUILD_OPTS="--output=type=docker" for build-docker-image

But back to the issue.
There's literally no output? very strange, what happens if you do cross-util run -i -v --target aarch64-pc-windows-msvc -- bash

[mominul]

There's literally no output? very strange, what happens if you do cross-util run -i -v --target aarch64-pc-windows-msvc -- bash

It just prints these outputs and exits successfully. 😕

mominul@Muhammads-MacBook-Air ~/s/tsf-example (main)> cross-util run -i -v --target aarch64-pc-windows-msvc -- bash
+ /usr/local/bin/docker
+ /usr/local/bin/docker version -f '{{ .Server.Os }},,,{{ .Server.Arch }}'
+ /usr/local/bin/docker info -f {{.SecurityOptions}}
+ cargo metadata --format-version 1 --filter-platform aarch64-pc-windows-msvc
+ rustc --print sysroot
+ /usr/local/bin/docker
+ /usr/local/bin/docker version -f '{{ .Server.Os }},,,{{ .Server.Arch }}'
+ /usr/local/bin/docker info -f {{.SecurityOptions}}
+ /usr/local/bin/docker run --userns host --platform linux/amd64 -e 'XARGO_HOME=/Users/mominul/.xargo' -e 'CARGO_HOME=/Users/mominul/.cargo' -e 'CROSS_RUST_SYSROOT=/Users/mominul/.rustup/toolchains/stable-x86_64-unknown-linux-gnu' -e 'CARGO_TARGET_DIR=/target' -e 'CROSS_RUNNER=' -e BROWSER -e TERM -e 'USER=mominul' -e 'CROSS_RUSTC_MAJOR_VERSION=1' -e 'CROSS_RUSTC_MINOR_VERSION=0' -e 'CROSS_RUSTC_PATCH_VERSION=0' --name cross-stable-x86_64-unknown-linux-gnu-da827-9b00956e5-aarch64-pc-windows-msvc-80a3f-1717825740164 --rm --user 501:20 -v /Users/mominul/.xargo:/Users/mominul/.xargo:z -v /Users/mominul/.cargo:/Users/mominul/.cargo:z -v /Users/mominul/.cargo/bin -v /Users/mominul/src/tsf-example:/Users/mominul/src/tsf-example:z -v /Users/mominul/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:/Users/mominul/.rustup/toolchains/stable-x86_64-unknown-linux-gnu:z,ro -v /Users/mominul/src/tsf-example/target:/target:z -w /Users/mominul/src/tsf-example -t -i ghcr.io/cross-rs/aarch64-pc-windows-msvc-cross:local sh -c 'PATH="$PATH":"/Users/mominul/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin" sh -c bash'

note: I only suggested CROSS_BUILD_OPTS="--output=type=docker" for build-docker-image

Do I need to build the image again? 😢

Thanks!

[Emilgardis]

is your docker working?

try docker run -it ghcr.io/cross-rs/aarch64-pc-windows-msvc-cross:local and docker run -it ubuntu:20.04

[mominul]

docker run -it ghcr.io/cross-rs/aarch64-pc-windows-msvc-cross:local exits with the same 53 error no and gives a warning:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

While passing --platform linux/amd64 to the arguments eliminates the warning, it still errors with error no 53 without any output.

My docker is working as it can run the ubuntu:20.04 image interactively:

mominul@Muhammads-MacBook-Air ~ [53]> docker run -it ubuntu:20.04
Unable to find image 'ubuntu:20.04' locally
20.04: Pulling from library/ubuntu
f02209be4ee5: Pull complete 
Digest: sha256:0b897358ff6624825fb50d20ffb605ab0eaea77ced0adb8c6a4b756513dec6fc
Status: Downloaded newer image for ubuntu:20.04
root@acd2f0db3510:/# 

[camertron]

I'm also running into the exact same issue, same output and everything. Did you ever figure this out @mominul?

[mominul]

I'm also running into the exact same issue, same output and everything. Did you ever figure this out @mominul?

Unfortunately no, I have deleted the docker image and now using a VM to run Windows 😞

[MarijnS95]

Is this issue perhaps the reason why aarch64-pc-windows-msvc is not available by default in cargo-cross?

Even after installing the latest from-source version with cargo install cross --git https://github.com/cross-rs/cross (commit a6cffa0), cargo-cross says it doesn't have a docker image for this target yet (we're running this on a Windows x86_64 GitHub actions machine):

$ cross build --target=aarch64-pc-windows-msvc
[cross] warning: `cross` does not provide a Docker image for target aarch64-pc-windows-msvc, specify a custom image in `Cross.toml`.
[cross] error: Errors encountered before cross compilation, aborting.
[cross] note: Disable this with `CROSS_NO_WARNINGS=0`

EDIT: I don't see it listed in the published packages either: https://github.com/orgs/cross-rs/packages?tab=packages&q=aarch64

Is that why the OP is manually building the image?

EDIT2: This target doesn't seem to be noted in https://github.com/cross-rs/cross?tab=readme-ov-file#supported-targets, and only carries this off-hand note:

Additional Dockerfiles for other targets can be found in cross-toolchains. These include MSVC and Apple Darwin targets, which we cannot ship pre-built images of.

It seems that building manually is expected? Perhaps the warning should link to this, and the note should be more explicitly specified (perhaps by having some common targets spelled out in full)?

EDIT3: This also seems to be mentioned at cross-rs/cross-toolchains#32.

EDIT4: I misunderstood. As it turns out this is a Linux Docker image that is set up to cross-compile to MSVC, rather than running the tests natively (and emulating the architecture) on Windows: #1672 (comment)