Dynamic libraries

[petrasvestartas]

Hi,

I would like to ask how scikit-build-core and nanobind handles dynamic libraries in cmake?

I have a 3rd party library that have dynamic libraries. Do you need to copy .dll/.dylib to wheel directory or it is managed automatically?

[LecrisUT]

No magical handling right now. The relevant PR is #1009, but the various auditwheel and such might be able to handle it. It is more of a gray area because the linkage is within the wheel, and the RPATH is not available without manual intervention in those cases.

Also the online documentation about it is here if you have any notes/improvements for it.

[petrasvestartas]

Hmm, I am trying to understand.

If a cmake project references dynamic libraries that is built with cmake itself, will the wheel folder contain those file? Or I would need to make a command to copy paste the libs next to nanobind built file?
( i am not using any dynamic library that i do not build myself )

Is there any simple example?

[LecrisUT]

TLDR: auditwheel and co. might do the job for you automagically, if you point LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/PATH appropriately.

It is a bit convoluted. First regarding what do the wheel repairs do, they detect what libraries are dependent on one another using the default library loading methods (RPATH, dlls in the same folder, system libraries, etc.), and then they copy the dependent libraries over (also rename the libraries to a random string) and patch in the dependencies so that they are preferred and loaded when requested (adding RPATH or calling os.add_dll_directory). As long as the dependencies are able to be resolved, the wheel repair tools will handle the dirty work for you. You can use LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/PATH to help it.

The issue with nanobind (and any other multi-library builds) is that after the installation, the dependency information is gone (RPATH is stripped, and dlls are being dlls), so the wheel repair tools do not have the knowledge of what each library is being linked to. For system libraries it can work more because those would be in the default library locations and are automatically picked up.

On our side though, we know during the build which library is dependent on which during the CMake build, so we can more reliably do the correction, which is what's being done in #1009.

Is there any simple example?

I don't have one at hand, but a simple 2 library hello-world project should show the issue. But you want an example of it working or not working?

[petrasvestartas]

I don't have one at hand, but a simple 2 library hello-world project should show the issue. But you want an example of it working or not working?

Yes just a simple hello world example that would work when I build using cibuildwheels
It would be amazing:)

I always start from this:
https://github.com/wjakob/nanobind_example

But never tried with dynamic libraries.