Skip to content

Possible regression causing linking of proc_macro #1374

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
snowp opened this issue Jun 2, 2022 · 10 comments · Fixed by #1500
Closed

Possible regression causing linking of proc_macro #1374

snowp opened this issue Jun 2, 2022 · 10 comments · Fixed by #1500
Assignees
@scentini

Comments

@snowp
Copy link

snowp commented Jun 2, 2022

We updated rules_rust from 0.4.0 to 28ac6b1 and ran into an issue where proc_macro is now statically linked into a test binary, causing

/home/runner/.cache/bazel/_bazel_runner/374864cd5aeb008ef6a614637fda772e/sandbox/linux-sandbox/630/execroot/__main__/bazel-out/k8-fastbuild/bin/src/wrapper/buffer_test.runfiles/__main__/src/wrapper/_buffer_test: error while loading shared libraries: libstd-05b39ac0cb4c5688.so: cannot open shared object file: No such file or directory

21eed19...28ac6b1

I made sure that this was not coming from the Rust version upgrade (tried bad sha + 1.60).

I don't have a simple reproducer, though I suspect it's coming from our usage of cxx, though we also make use of other proc_macro libraries such as tokio, thiserror, prost, etc.
@scentini
Copy link
Collaborator

scentini commented Jun 7, 2022

I managed to create a simple repro. This is not related to proc_macros; the culprit is #1333 and @krasimirgg and I are discussing the best way forward.

@scentini
Copy link
Collaborator

scentini commented Jun 9, 2022

@snowp could you check whether #1394 solves the problem for you?

@snowp
Copy link
Author

snowp commented Jun 9, 2022

Running against that branch I get

dyld[52718]: Library not loaded: @rpath/libstd-0dd069df8d91d43f.dylib
  Referenced from: /private/var/tmp/_bazel_snowp/c1591b76b8a2fcd15b6ff3cd34fdd7da/execroot/__main__/bazel-out/darwin_arm64-dbg/bin/src/logger/logger_test
  Reason: tried: '/usr/local/lib/libstd-0dd069df8d91d43f.dylib' (no such file), '/usr/lib/libstd-0dd069df8d91d43f.dylib' (no such file)

for a lot (not all) Rust targets when I apply repository_override to our project

@scentini scentini mentioned this issue Jun 10, 2022
Merged
@scentini
Copy link
Collaborator

Thanks! That's the same error, just on macos this time, which is a reminder that we should apply the solution to all platforms.

@snowp
Copy link
Author

snowp commented Jun 10, 2022

Tried this on Linux and there it seems to be no issues when running the branch, so sounds like this is the right fix and just needs to also be done for macos :)

@Reflejo
Copy link

Reflejo commented Jun 24, 2022

@scentini Were you able to make any progress with this issue? Thanks!

@scentini
Copy link
Collaborator

Sorry for the delay; I managed to get CI green in the unpolished (and incorrectly named) #1411. I want to try one more approach before I mark it ready for review. The PR will most likely be ready on Monday, as I'm going to be OOO for the next few days.

@UebelAndre
Copy link
Collaborator

@scentini hey, any updates here?

scentini added a commit to scentini/rules_rust that referenced this issue Aug 5, 2022
@scentini
Fix bazelbuild#1374
74ef87d
@scentini scentini mentioned this issue Aug 5, 2022
Merged
@scentini
Copy link
Collaborator

scentini commented Aug 5, 2022

Hey folks, I have #1500 up for review.

scentini added a commit that referenced this issue Aug 5, 2022
@scentini
Do not pass --Clink-arg=-l for libstd and libtest (#1500)
4e5fac5 
Fixes #1374

We skip adding `-Clink-arg=-l...` for libstd and libtest from the standard library, as these two libraries are present both as an `.rlib` and a `.so` format. On linux, Rustc adds a -Bdynamic to the linker command line before the libraries specified with `-Clink-arg`, which leads to us linking against the `.so`s but not putting the corresponding value to the runtime library search paths, which results in a "cannot open shared object file: No such file or directory" error at exectuion time. We can fix this by adding a `-Clink-arg=-Bstatic` on linux, but we don't have that option for macos. The proper solution for this issue would be to remove `libtest-{hash}.so` and `libstd-{hash}.so` from the toolchain. However, it is not enough to change the toolchain's `rust_std_{...}` filegroups here: https://github.com/bazelbuild/rules_rust/blob/a9d5d894ad801002d007b858efd154e503796b9f/rust/private/repository_utils.bzl#L144 because rustc manages to escape the sandbox and still finds them at linking time. We need to modify the repository rules to erase those files completely. This PR should be a good workaround until we get do to the proper thing though.

This PR also fixes the following issues for Windows:
* get_lib_name() didn't work properly on windows for `libc`
* `-Clink-arg` should point to the library name with extension included for windows.
@snowp
Copy link
Author

snowp commented Aug 12, 2022

After applying this fix I'm still seeing

dyld[51630]: Library not loaded: '@rpath/libtest-e6734f05acc07fba.dylib'
  Referenced from: '/private/var/tmp/_bazel_snowp/c1591b76b8a2fcd15b6ff3cd34fdd7da/execroot/__main__/bazel-out/darwin_arm64-fastbuild/bin/test/client/logger_runner'
  Reason: tried: '/usr/local/lib/libtest-e6734f05acc07fba.dylib' (no such file), '/usr/lib/libtest-e6734f05acc07fba.dylib' (no such file)

for one of our use cases on macos (linux is fine). Would you prefer me to open up a new issue to track this?

@snowp snowp mentioned this issue Sep 30, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@scentini scentini

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Do not pass --Clink-arg=-l for libstd and libtest scentini/rules_rust
4 participants
@Reflejo @snowp @scentini @UebelAndre