Future of wheel.macosx_libfile

[kunaljubce]

setuptools version

setuptools==78.1.0

Python version

Python 3.10

OS

macOS

Additional environment information

No response

Description

Getting the error below when trying uv sync. Seems to be happening due to this import.

As I see from the latest wheel release that the file, src/wheel/macosx_libfile.py, is no longer present. It was present in the previous wheel release

Error stack trace:

Traceback (most recent call last):
        File "<string>", line 11, in <module>
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/build_meta.py", line 435, in build_wheel
          return _build(['bdist_wheel'])
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/build_meta.py", line 426, in _build
          return self._build_with_temp_dir(
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/build_meta.py", line 407, in _build_with_temp_dir
          self.run_setup()
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 315, in <module>
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 186, in setup
          return run_commands(dist)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
          dist.run_commands()
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
          self.run_command(cmd)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
          super().run_command(command)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 407, in run
          impl_tag, abi_tag, plat_tag = self.get_tag()
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 321, in get_tag
          plat_name = get_platform(self.bdist_dir)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 63, in get_platform
          from wheel.macosx_libfile import calculate_macosx_platform_tag
      ModuleNotFoundError: No module named 'wheel.macosx_libfile'

Expected behavior

The imports should work for the latest setuptools release

How to Reproduce

Try to create a virtual environment with the latest setuptools==78.1.0.

Output

Traceback (most recent call last):
        File "<string>", line 11, in <module>
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/build_meta.py", line 435, in build_wheel
          return _build(['bdist_wheel'])
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/build_meta.py", line 426, in _build
          return self._build_with_temp_dir(
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/build_meta.py", line 407, in _build_with_temp_dir
          self.run_setup()
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/build_meta.py", line 320, in run_setup
          exec(code, locals())
        File "<string>", line 315, in <module>
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/__init__.py", line 117, in setup
          return distutils.core.setup(**attrs)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 186, in setup
          return run_commands(dist)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
          dist.run_commands()
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
          self.run_command(cmd)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/dist.py", line 1104, in run_command
          super().run_command(command)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
          cmd_obj.run()
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 407, in run
          impl_tag, abi_tag, plat_tag = self.get_tag()
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 321, in get_tag
          plat_name = get_platform(self.bdist_dir)
        File "/Users/a81045729/.cache/uv/builds-v0/.tmpEp6P7H/lib/python3.10/site-packages/setuptools/command/bdist_wheel.py", line 63, in get_platform
          from wheel.macosx_libfile import calculate_macosx_platform_tag
      ModuleNotFoundError: No module named 'wheel.macosx_libfile'

[abravalheri]

@agronholm, do you have any recommendations for this one?

(I have previously seen the wheel PR with the changes for bdist_wheel, and it seemed reasonable for me, but I haven't catched the change in the supporting module).

[agronholm]

I've yanked the latest wheel release until we can sort this out. Why is setuptools not importing this from its vendored copy of the module?

[agronholm]

The same problem was reported against wheel: pypa/wheel#659

[kunaljubce]

Thanks for the quick response and for yanking the release @agronholm -

[kunaljubce]

Closing the issue!

[abravalheri]

@agronholm, I apologize, I have previously seen the wheel PR with the changes for bdist_wheel, and it seemed reasonable for me, but I haven't catch the change in the supporting module.

I see 3 options going forward:

1. setuptools absorbs also the macosx_libfile module
2. A different foundational PyPA project absorbs it.
3. We create a separated project for it, e.g. pypa/macosx_libfile (I would still keep the API as "protected" for internal use of the PyPA projects only, e.g. via a private module macos_libfile._restricted¹)

Regarding 1, the module seems to be fairly complex and require advanced knowledge about macOS systems. I don't feel right now confident in transferring the maintenance of such module into the same project as setuptools (which is already high maintenance).
Regarding 2, I think it is unlikely other foundational projects are interested in absorbing this module.

So what makes more sense to me is option 3 to decouple the maintenance effort. It would be fair that setuptools maintainers would also added to that project (given setuptools is its main consumer), but hopefully we can also have other volunteers involved.

@jaraco, @agronholm any thoughts?

Footnotes

1. Not sure if we can have projects names starting with underscore in PyPI. ↩

[agronholm]

To me it seems like packaging should be where the modules ends up. It was contributed to wheel by someone else and I don't have the necessary expertise to maintain it myself either. But at least setuptools seems like the wrong place to expose it from.

But I repeat my question: why is setuptools not importing its vendored copy of the module?

[abravalheri]

But I repeat my question: why is setuptools not importing its vendored copy of the module?

This seems to be a limitation/intentional design of the simplified vendoring system in setuptools, introduced in #4457.

[agronholm]

Looks like this was introduced relatively recently, and after my migration work of bdist_wheel to setuptools. I had no clue this had been done, and I would've taken a different approach with wheel v0.46.0 had I known about it. In practice this means I'm again stuck with the wheel non-API which I had hoped to get rid of!

[abravalheri]

@pypa/packaging-committers any thoughts on the suggestions in #4935 (comment) and #4935 (comment)?

---

In practice this means I'm again stuck with the wheel non-API which I had hoped to get rid of!

@agronholm, let's see if we can find a solution.

[agronholm]

How are the other build back-ends dealing with macOS extension binaries?

[henryiii]

"protected" for internal use of the PyPA projects only

I might want to use it in scikit-build-core and/or meson-python. :)

Currently, I believe most of them don't analyze the binaries this way, but instead just base it on what a user requests.

[abravalheri]

I might want to use it in scikit-build-core and/or meson-python. :)

That should be fine. I think there might be still some benefit and keeping it reserved for a well know list of collaborating projects so that we can communicate changes effectively, instead of a publicly available API.

[jmroot]

But I repeat my question: why is setuptools not importing its vendored copy of the module?

The setuptools behaviour of only using the vendored dependencies if there is not an external version available almost seems like the worst of both worlds. The extra effort of managing vendored deps, and the increased installation size, without any of the certainty that things won't break.

[agronholm]

But I repeat my question: why is setuptools not importing its vendored copy of the module?

The setuptools behaviour of only using the vendored dependencies if there is not an external version available almost seems like the worst of both worlds. The extra effort of managing vendored deps, and the increased installation size, without any of the certainty that things won't break.

Yeah, it's a recipe for a disaster, as was just seen here.

[brettcannon]

@pypa/packaging-committers any thoughts on the suggestions in #4935 (comment) and #4935 (comment)?

I'm assuming the question is whether we would want to absorb macos_libfile into 'packaging', and my person answer would be "no" as it doesn't seem to tie into a fundamental PyPA spec.

[abravalheri]

Thanks @brettcannon.

I believe it is being used to calculate the correct platform tag for macOS, which is required by/related to the standard on the wheel format.

Does packaging offer an alternative way of doing that?

May be related:

pypa/packaging#330

pypa/packaging#288

---

If packaging is not interesting in absorbing the functionality, I think the best is to extract it into a separated lib.

[eli-schwartz]

I would be naturally inclined to assume the relevant stakeholders for this conceptual functionality, are the delocate project.

I can't directly see anywhere in the delocate documentation any suggestion that it can/does relabel a wheel with the correct platform tags according to minimum platform requirements. However auditwheel (the Linux equivalent of delocate) is certainly capable of doing precisely this for a Linux wheel -- that is, extract the minimum compatibility tag for glibc ELF binaries and convert it to a PyPA-endorsed "manylinux" platform tag.

So it's plausible that this should just be a feature request for delocate.

[brettcannon]

I believe it is being used to calculate the correct platform tag for macOS

... for a specific file, not the current platform that's running. (We calculate that using https://github.com/pypa/packaging/blob/e624d8edfaa28865de7b5a7da8bd59fd410e5331/src/packaging/tags.py#L408).

Does packaging offer an alternative way of doing that?

No, but that's because we don't try to calculate based on a file what wheel tags should be used for the resulting wheel file.

[m-aciek]

I would be naturally inclined to assume the relevant stakeholders for this conceptual functionality, are the delocate project.

cc @HexDecimal @matthew-brett (delocate maintainers)

[HexDecimal]

I can't directly see anywhere in the delocate documentation any suggestion that it can/does relabel a wheel with the correct platform tags according to minimum platform requirements.

Delocate does relabel a wheels platform tags. It currently uses the macholib package to fetch versioning info from the relevant MacOS library files. This works well so far, but I wouldn't call macholib well maintained.