Error compiling example crate on NixOS

[jfredett]

I'll start with two caveats:

1. I know very little about GPU programming, much less GPU programming in rust.
2. My setup involves a major confounding factor (nixos), which might be the reason this is failing.

---

I recently decided to start messing around with GPU programming in rust. Landed on the rust-gpu crate and found my way here while trying to get set up via this discussion. This looked like the easiest way to get something running so I went ahead and set up this test repo to try it out. The current state of that repo is:

1. A couple of initialization commits to set up the nix devshell w/ rust and such
2. Importing the shader template repo
3. Some clerical stuff (README, etc).

I then installed cargo-gpu via the instructions in the README, and attempted to cargo gpu build the test project, I got this error:

[2025-02-09T19:15:25Z ERROR cargo_gpu] No such file or directory (os error 2)
Error: No such file or directory (os error 2)

I poked around but didn't see anything obviously wrong, so I went ahead and reran with RUST_LOG=trace to get:

➜ RUST_LOG=trace cargo gpu build
[2025-02-09T20:21:59Z TRACE cargo_gpu] CLI args: [
        "/home/jfredett/code/rust-gpu-experiments/.devenv/state/cargo-install/bin/cargo-gpu",
        "build",
    ]
[2025-02-09T20:21:59Z DEBUG cargo_gpu::metadata] Querying Cargo metadata for "./Cargo.toml"
[2025-02-09T20:21:59Z DEBUG cargo_gpu::metadata] Matching shader crate path with manifest path: /home/jfredett/code/rust-gpu-experiments/Cargo.toml == /home/jfredett/code/rust-gpu-experiments/Cargo.toml?
[2025-02-09T20:21:59Z DEBUG cargo_gpu] building with final merged arguments: Build {
        install: Install {
            spirv_install: InstallArgs {
                dylib_path: "INTERNALLY_SET",
                shader_crate: "./",
                spirv_builder_source: None,
                spirv_builder_version: None,
                rust_toolchain: None,
                force_spirv_cli_rebuild: false,
                auto_install_rust_toolchain: false,
            },
        },
        build_args: BuildArgs {
            output_dir: "./",
            watch: false,
            no_default_features: false,
            features: [],
            target: "spirv-unknown-vulkan1.2",
            shader_target: "spirv-unknown-vulkan1.2",
            deny_warnings: false,
            debug: false,
            capability: [],
            extension: [],
            multimodule: false,
            spirv_metadata: None,
            relax_struct_store: false,
            relax_logical_pointer: false,
            relax_block_layout: false,
            uniform_buffer_standard_layout: false,
            scalar_block_layout: false,
            skip_block_layout: false,
            preserve_bindings: false,
        },
    }
[2025-02-09T20:21:59Z INFO  cargo_gpu::install] cache directory is '/home/jfredett/.cache/rust-gpu'
[2025-02-09T20:21:59Z DEBUG cargo_gpu::spirv_source] Running `cargo tree` on /home/jfredett/code/rust-gpu-experiments
[2025-02-09T20:21:59Z TRACE cargo_gpu::spirv_source]   found Some("spirv-std v0.9.0")
[2025-02-09T20:21:59Z TRACE cargo_gpu::spirv_source] parsing spirv-std source and version from def: 'spirv-std v0.9.0'
[2025-02-09T20:21:59Z DEBUG cargo_gpu::spirv_source] Parsed `rust-gpu` source and version: CratesIO("v0.9.0")
[2025-02-09T20:21:59Z DEBUG cargo_gpu::spirv_source] Not cloning `rust-gpu` repo (https://github.com/Rust-GPU/rust-gpu) as it already exists at /home/jfredett/.cache/rust-gpu/rust-gpu-repo/v0_9_0
[2025-02-09T20:21:59Z DEBUG cargo_gpu::spirv_source] Checking out `rust-gpu` repo at /home/jfredett/.cache/rust-gpu/rust-gpu-repo/v0_9_0 to v0.9.0
[2025-02-09T20:21:59Z DEBUG cargo_gpu::spirv_source] Getting `rust-gpu` version date from /home/jfredett/.cache/rust-gpu/rust-gpu-repo/v0_9_0
[2025-02-09T20:21:59Z DEBUG cargo_gpu::spirv_source] Parsed date for version v0.9.0: 2023-07-25
[2025-02-09T20:21:59Z DEBUG cargo_gpu::spirv_source] Parsing `rust-toolchain.toml` at "/home/jfredett/.cache/rust-gpu/rust-gpu-repo/v0_9_0" for the used toolchain
[2025-02-09T20:21:59Z DEBUG cargo_gpu::spirv_source] Parsed version, date and toolchain channel from shader-defined `rust-gpu`: CratesIO("v0.9.0"), 2023-07-25, nightly-2023-05-27
[2025-02-09T20:21:59Z ERROR cargo_gpu] No such file or directory (os error 2)
Error: No such file or directory (os error 2)

I then went poking around and did find that final debug line, but due to caveat #1 I haven't been particularly successful in figuring out which file it's looking for.

Assuming this example is expected to build out of the box, I'm guessing this is a nix related issue, but I'm hoping I can get some insight before I start chasing geese.

Thanks!

[tombh]

Hey there! Thanks for the detailed report ❤️ Yeah, that could be a better error, sorry. I think it might be because you don't have rustup installed? If that's not he cause, could you also add a RUST_BACKTRACE=1 (so RUST_BACKTRACE=1 RUST_LOG=trace cargo gpu build) to see if we can pin down the line where the error happens.

[jfredett]

Definitely was the lack of rustup. trace, no backtrace

[2025-02-09T22:06:33Z DEBUG cargo_gpu::spirv_source] Parsed version, date and toolchain channel from shader-defined `rust-gpu`: CratesIO("v0.9.0"), 2023-07-25, nightly-2023-05-27
[2025-02-09T22:06:33Z DEBUG cargo_gpu::spirv_cli] toolchain nightly-2023-05-27 is already installed
[2025-02-09T22:06:33Z DEBUG cargo_gpu::spirv_cli] asking for consent to install the required toolchain
🦀 Install toolchain components (rust-src, rustc-dev, llvm-tools) with `rustup` [y/n]: [2025-02-09T22:06:33Z TRACE mio::poll] registering event source with poller: token=Token(0), interests=READABLE
                                                                                                                                                                                                      [2025-02-09T22:06:33Z TRACE mio::poll] registering event source with poller: token=Token(1), interests=READABLE
                                                                                                                                                                                                                                                                                                                     🦀 Key(KeyEvent { code: Char('y'), modifiers: KeyModif
iers(0x0), kind: Press, state: KeyEventState(0x0) })
🦀 Installing toolchain components (rust-src, rustc-dev, llvm-tools) with `rustup`
info: downloading component 'rust-src'
info: installing component 'rust-src'
info: downloading component 'rustc-dev'
 85.8 MiB /  85.8 MiB (100 %)  10.5 MiB/s in  9s ETA:  0s
info: installing component 'rustc-dev'
 85.8 MiB /  85.8 MiB (100 %)   9.6 MiB/s in  8s ETA:  0s
info: downloading component 'llvm-tools'
 32.7 MiB /  32.7 MiB (100 %)   9.6 MiB/s in  7s ETA:  0s
info: installing component 'llvm-tools'
 32.7 MiB /  32.7 MiB (100 %)  10.4 MiB/s in  3s ETA:  0s
[2025-02-09T22:07:08Z DEBUG cargo_gpu::install] writing spirv-builder-cli source files into '/home/jfredett/.cache/rust-gpu/spirv-builder-cli/v0_9_0+nightly-2023-05-27'
[2025-02-09T22:07:08Z DEBUG cargo_gpu::spirv_source] Running `cargo tree` on /home/jfredett/code/rust-gpu-experiments
[2025-02-09T22:07:08Z TRACE cargo_gpu::spirv_source]   found Some("spirv-std v0.9.0")
[2025-02-09T22:07:08Z TRACE cargo_gpu::spirv_source] parsing spirv-std source and version from def: 'spirv-std v0.9.0'
[2025-02-09T22:07:08Z DEBUG cargo_gpu::spirv_source] Parsed `rust-gpu` source and version: CratesIO("v0.9.0")
[2025-02-09T22:07:08Z DEBUG cargo_gpu::spirv_source] Not cloning `rust-gpu` repo (https://github.com/Rust-GPU/rust-gpu) as it already exists at /home/jfredett/.cache/rust-gpu/rust-gpu-repo/v0_9_0
[2025-02-09T22:07:08Z DEBUG cargo_gpu::spirv_source] Checking out `rust-gpu` repo at /home/jfredett/.cache/rust-gpu/rust-gpu-repo/v0_9_0 to v0.9.0
[2025-02-09T22:07:08Z DEBUG cargo_gpu::spirv_source] Getting `rust-gpu` version date from /home/jfredett/.cache/rust-gpu/rust-gpu-repo/v0_9_0
[2025-02-09T22:07:08Z DEBUG cargo_gpu::spirv_source] Parsed date for version v0.9.0: 2023-07-25
[2025-02-09T22:07:08Z DEBUG cargo_gpu::spirv_source] Parsing `rust-toolchain.toml` at "/home/jfredett/.cache/rust-gpu/rust-gpu-repo/v0_9_0" for the used toolchain
[2025-02-09T22:07:08Z DEBUG cargo_gpu::spirv_source] Parsed version, date and toolchain channel from shader-defined `rust-gpu`: CratesIO("v0.9.0"), 2023-07-25, nightly-2023-05-27
[2025-02-09T22:07:08Z DEBUG cargo_gpu::install] writing Cargo.toml
[2025-02-09T22:07:08Z DEBUG cargo_gpu::install] writing Cargo.lock
[2025-02-09T22:07:08Z DEBUG cargo_gpu::install] writing src/main.rs
[2025-02-09T22:07:08Z DEBUG cargo_gpu::install] writing src/lib.rs
[2025-02-09T22:07:08Z DEBUG cargo_gpu::install] writing src/args.rs
🦀 Compiling shader-specific `spirv-builder-cli` for ./
[2025-02-09T22:07:08Z DEBUG cargo_gpu::install] building artifacts with `cd "/home/jfredett/.cache/rust-gpu/spirv-builder-cli/v0_9_0+nightly-2023-05-27" && "cargo" "+nightly-2023-05-27" "build" "--release" "--no-default-features" "--features" "spirv-builder-pre-cli"`
error: no such command: `+nightly-2023-05-27`

help: invoke `cargo` through `rustup` to handle `+toolchain` directives
[2025-02-09T22:07:08Z ERROR cargo_gpu] ...build error!
Error: ...build error!

This is definitely coming down to nix weirdness. Because my repo only has cargo & company available via the devshell provided by the flake, when you jump to the cached copy, it no longer has cargo or any of the rustup components. I suspect I may be SOL (at least on NixOS).

I assume that you require a specific version of nightly because of deep magic necessary for doing the Rust -> GPU magic? Wondering if I could just tell it to set the cache inside my repo in some ignored subdir then use the compiler stuff already in the devshell.

[tombh]

I'm very much open to making any changes needed to support Nix. I've been using and appreciating the Nix package manager for a couple of years, but never actually made a flake yet. I understand enough that rustup's habit of downloading random binaries is very un-Nix-like. So I'm wondering if maybe cargo gpu could have a new sub-command that instead of compiling the shader itself, outputs a custom flake specifically for the shader crate.

For now though, you can still certainly compile your shader! Just not with cargo gpu. You'd need to follow the guide in in the main rust-gpu repo. Probably the most important file to understand is this one: https://github.com/Rust-GPU/rust-gpu/blob/main/examples/runners/wgpu/builder/src/main.rs. The only downside to this approach is that the shader crate must use the very specific rust-toolchain.toml file required by the spirv-std dependency in your shader. This generally only becomes an actual problem (rather than a mere inconvenience) if your shader crate lives in a wider cargo workspace. Which your shader doesn't seem to be in yet.

Haha yes the deep magic is that custom codegen backends are not in stable Rust yet. rust-gpu has a backend that converts a significant subset of Rust to SPIR-V.

[jfredett]

Cool, I appreciate the pointers, I'll dig into those and see what I can get running. It should certainly be possible to pull specific versions of packages. My project uses the fenix rust flake, which does allow for specific versions to be loaded. I suppose you could have a flake that provides a specific rust toolchain as one package (e.g., packages.$system.spirv-toolchain) and then the flake would probably have to package some scripts to actually use them, or maybe a devshell.

Anyway -- I should probably go learn how to compile things the hard way first, if I figure out something solid wrt flakes I'm happy to ping this issue to save some hunting later.

Thanks again!

[tombh]

Great, I'll ping this issue too if I make any progress 🤓

[jfredett]

Hiyo.

I've made a bit of progress, sufficient to run into a different issue (#45), though I don't love my solution so much, as it ultimately bypasses the nix stuff entirely and just installs rustup. Since I can't get it totally compiling I can't be sure it will work, but since it's just the expected rustup environment, I think it should.

I did try (in the sistered flake.nix-fenix file) to have nix handle the install of the two rust versions, I wasn't able to figure out how to tell the build.rs to use the gpu_rustpkg based compiler when building the shader crate in the workspace, but the preferred_rustpkg for everything else, but this may be due to my inexperience writing build.rses.

I did see that rust-gpu does some checking of RUSTC, I suppose it could be possible to extend it to take an env var (e.g., GPU_RUSTC), but I don't know enough about the implications/preferred direction for the broader project, and that's also a little out of the scope I was aiming for.

tl;dr -- I think the rustup version should work, mod the mentioned issue, I'll probably see about trying the workaround suggested in there (manually running the version on PR #41) in this environment and see if that gets something compiling.

[tombh]

Nice progress!

I think it'd be worth opening a new dedicated issue to explore the potential of a faithful Nix flake that can build shaders. With the idea that cargo-gpu could add a subcommand that automatically created the flake when pointed at a given shader crate. You can help us with the Nix concepts and we can help you with the build.rs concepts.

And regarding the rustup flake, it sounds like it should just work with PR #41, let us know if there's a problem. Another option you may have to avoid the bug described in #45 is simply updating your version of spirv-std in the shader crate. Maybe even updating to the latest Git version, eg: spirv-std = { git = "https://github.com/Rust-GPU/rust-gpu", rev = "6e2c84d" }. Though note that some of us can't do that yet because latest rust-gpu has a regression where arrays in arrays won't compile Rust-GPU/rust-gpu#46

[jfredett]

Heyo, busy on a different thing for a bit, back to gpu stuff!

I got things kind of working, it was more difficult than just swapping the dependency. Full details in the experiment repo, but tl;dr -- bytemuck versions disagreed, but seem to work fine with a simple version bump in the spirt crate. rustix has an itoa dependency that didn't resolve, but the very bleeding edge of it doesn't require itoa anymore (only one function required it, so they reimplemented). Not sure if that change has landed in an official version yet though.

In any case, those changes were sufficient to get to the point where it tries to build, but I then run into actual Rust errors that aren't just dependency issues so my hamfisted version bumping isn't going to resolve them.

My total guess is I actually have a wrong version (whether too far ahead or too far behind, IDK) of spirt, since most of the errors have to do with unresolved imports/missing methods/missing fields/etc -- but I'm out of my depth at this point. The README has the full output.

EDIT:

Just decided to go hunting for one of the errors, and I see that in fc59398 of spirt, the control_node was renamed to node in a bunch of places, so probably I just need to roll my spirt back and re-apply the bytemuck bump to see what happens.

[tombh]

Hey hey, those logs are great! I love keeping journals too, it's always nice to see somebody else's thought process, the highs and lows, all very relatable 😅

Basically I don't think you should be trying to get cargo-gpu to work. Your efforts would be much more rewarded from investing in the build.rs approach.

What's more, is that my plan for supporting NixOS in cargo-gpu is that it wouldn't even try to do any building/compiling. It would merely spit out a customised flake, that then would do the shader compilation. So even if you did get cargo-gpu to work with your current approach, I'm not sure I'd want to integrate that into cargo-gpu, I think it would be too hard to maintain.

Sorry if that's disappointing to hear 😞

[jfredett]

Not at all, mostly I'm trying to get to the 'something-that-compiles' stage, I'm generally better at fixing broken/half-solutions than coming up with new ones from scratch.

Wrt the build.rs, I have mostly been trying 'transparent' stuff so far, I'm reasonably confident I could force it to work by changing stuff on the rust-gpu side, but I feel like it should be possible to do this from the 'outside' in a way that rust-gpu wouldn't really to know about. In principle it seems like if I can swap out RUSTC at the right time, and then swap back, it should 'just work'. So far I've tried (actually this was a while back) a proxy script that pretends to be RUSTC, my hope was to just detect when the script was being called on the shader crate, swap out, then swap back; that's not super straightforward to do -- at least not with my current level of understanding of the rust compilation process. I started down a path of -- essentially -- exception-catching, waiting for a rustc to fail, then trying to compile it again w/ the other version, but put it down when I started chasing the rustup-in-nixos approach.

I suppose another approach would be a flake which packaged the whole crate itself, though I think that's a little non-idiomatic for rust. Essentially it'd be a flake like:

packages.${system}.shader-crate = some-fenix-invocation-to-compile-the-package;

Then I think you'd need to tell the Cargo.toml to look for the specific compiled version provided by the flake, or just add another package for the containing crate, but at that point you're really just working all the way around cargo, and I think that might be pretty jarring.

Probably worth a try in any case.

What I'd expect in an ideal case is that I specify RUSTC_SHADERS and RUSTC, and the shader crate uses the RUSTC_SHADERS; but that violates my 'transparency' requirement. It would, however, basically preserve all the normal cargo commands and stuff, and provide the same experience to nix and non-nix users from a development perspective. For non-nix, they'd need to install RUSTC_SHADERS via rustup and set it somewhere, for nix, it'd be part of the flake.

[tombh]

Since rust-gpu's inception the only way of compiling shaders is that the entire workspace use the same specific version of nightly Rust. And that that version has to be specified in the workspace's root (or shader crate's root if not using a workspace). It's a pain, but that's how rust-gpu has been developed. And indeed is still developed and probably will continue to be developed until custom codegen backends are merged into stable Rust.

Whilst cargo-gpu overcomes this frustrating restriction from a user perspective, under the hood it is still honouring the single Rust version approach. So I think you're going to be really up against it by not following that. And also, I'm not sure that the effort would be worth it, because the only gain is being able to use a Rust version in the workspace that is different from the shader's Rust version. In other words, if you absolutely must use a different Rust version to the one required by the shader, it would likely be much easier to just keep the shader crate out of the workspace.

I think you're over-estimating the problem of multiple Rust versions. It really shouldn't be a blocker. At the end of the day it's nothing more than a minor inconvenience. It really shouldn't an obstacle to you compiling shader's with Nix. Have you managed to get a shader to compile with Nix where only the one version of Rust is used?

[jfredett]

Have you managed to get a shader to compile with Nix where only the one version of Rust is used?

Nope, because I was under a deeply mistaken impression of how things should work! :)

The way I understood it, the intent was to compile using a specific version to get the compiled artifact, then having the hosting application use whatever rust it liked, linking to the compiled result. In my head it was something akin to cross-compilation. You produce some binary blob that then gets hucked at the GPU by another program which views it as a black box.

If the intent is to have a single RUSTC across everything, that's actually more or less what my original flake did, but I was hung up trying to get it to compile each side independently!

I think you're over-estimating the problem...

This is on-brand for me. I'll see if I can whip up a flake w/ a fixed version, I suspect it will work without issue.

[jfredett]

I overestimated the ease as well.

I switched to a flake which only installs the compiler from the rust-toolchain of the version I was working against. I tried against both 0.9 and main, using code copied from the examples folder of rust-gpu at the same version. In the former, I got an issue with elsa wanting #[feature(inline_const)], attempting to force revert to 1.6 (down from 1.11) did not work as it appears to be using code from some >1.6 version (even though that was the specified dep in the example Cargo.toml).

Moving to main needed a different toolchain, that part worked fine, but it couldn't understand the #[spirv(...)] tag, which didn't make much sense to me.

I've already dropped way more time than I planned into this experiment. I've left my state dumped in that example repo, the flake there should give you a relatively minimal environment with the correct compiler for 0.9 (commented) or main (uncommented), but I don't plan to work on this anymore in the near future.

[tombh]

Thank you so much for your sharing your journey! If nothing else, I can just relate to it very much ❤️

Let's keep this issue open until we've got a good NixOS solution.