Revert removals introduced in v78.0.0
#4911
Conversation
| @@ -0,0 +1,2 @@ | |||
| Postponed removals of deprecated dash-separated and uppercase fields in ``setup.cfg``. | |||
| All packages with deprecated configurations are advised to move before 2026. | |||
There was a problem hiding this comment.
Thank you! But when you eventually do restore this behavior, could we please get the option to opt out?
Some packages are simply old & abandoned, and there's not much that consumers can do when trying to install a third-party package (as the many comments on #4910 would indicate!)
There was a problem hiding this comment.
The option is to pin setuptools via the PIP_CONSTRAINT environment variable (or the equivalent in UV).
Some packages are simply old & abandoned, and there's not much that consumers can do when trying to install a third-party package (as the many comments on #4910 would indicate!)
Please note that this only affects packages that do not publish a wheel to PyPI or other package index. Unless it is a module with native binary extensions, the recommendation of publishing wheels is a bit old already.
All things considered this is likely a symptom of something bigger, more problematic, and the package you depend upon has been on borrowed time for a while. If you depend on packages that do not receive any support you might be subject to all sorts of risks including security risks. In that case there are a couple of things that can be done during the extended period:
- Organise among other users contributions (or even a fork) for maintenance.
- Fund the original developers for maintenance.
- Find a suitable replacement.
There was a problem hiding this comment.
That may well all be true. But why not show a loud warning for a while before removing totally? The reality is that we try to vet our packages carefully, but still woke up to Monday morning with lots of broken build processes.
There was a problem hiding this comment.
No, the issue does also affect all Linux distributions and downstream vendors that rebuild wheels from sources. Linux distributions like Debian, Fedora, and Gentoo do not use wheels. They take sdists and rebuild wheels.
There was a problem hiding this comment.
…and given the scale of this issue, it will mean we won't be able upgrade setuptools for months. Or to put it more precisely, once again the burden of fixing all the fallout will be on downstreams, who spend their whole days putting out fires for free.
There was a problem hiding this comment.
I completely sympathize with the goal of getting all dependencies to do the right thing and being able to simplify the setup tools code. But I agree with @danielcarcich. From a robustness principle, this seems very unfortunate.
While this is easy to fix in smaller repos, many large orgs deal with a ton of unfortunately old and unmaintained packages. Yes that's a fault of them (I'm in one), but here we are.
I hope to see this reverted and thanks to @abravalheri for starting this pull-request preemptively while having the discussion. (apologies is this conversation should be in #4910 instead)
There was a problem hiding this comment.
The option is to pin setuptools via the
PIP_CONSTRAINTenvironment variable (or the equivalent in UV).Some packages are simply old & abandoned, and there's not much that consumers can do when trying to install a third-party package (as the many comments on #4910 would indicate!)
Please note that this only affects packages that do not publish a
wheelto PyPI or other package index. Unless it is a module with native binary extensions, the recommendation of publishing wheels is a bit old already.All things considered this is likely a symptom of something bigger, more problematic, and the package you depend upon has been on borrowed time for a while. If you depend on packages that do not receive any support you might be subject to all sorts of risks including security risks. In that case there are a couple of things that can be done during the extended period:
- Organise among other users contributions (or even a fork) for maintenance.
- Fund the original developers for maintenance.
- Find a suitable replacement.
A build tool needs to aim to interact with as many versions of libraries from the ecosystem as possible, not just the latest versions that adhere to modern best practices. Thanks for all the hard work maintaining a core utility and for responding quickly to the response today.
There was a problem hiding this comment.
I say this as someone who has contributed a patch to a popular community package to remediate a CVE, and maintained an internal fork waiting for a new release. My PR was accepted, but no release yet for over a year. Ok, not that big of a contribution, but you can't accuse me of never having contributed.
It will be 1. undifferentiated heavy lifting and 2. utter chaos to maintain internal forks or coordinate community forks for every single transitive dependency in our codebase, especially as others also have the dependencies as well.
If Setuptools does this again in 2026, it will be a repeat of this. How many warnings went unheeded this time? What are the chances those warnings will be heeded in the year to come? It will take years, not months, to untangle.
If the plan is to try this again in 2026, the package to be forked will be setuptools.
There was a problem hiding this comment.
While the contributions of Setuptools maintainers are very appreciated, I think it's important to realize that the Setuptools API has become a load-bearing component of the distribution infrastructure of many legacy Python packages that can no longer be easily updated, yet are widely needed for the foreseeable future. As such, I think it's simply not possible to introduce a change like this one without unbearable levels of disruption. A more finessed approach (like selectively treating these errors as warnings based on signals like the age of the package release, etc.) is necessary. And also a rethink of how to restrict and compartmentalize the packaging API surface so build dependencies for old packages can be bundled with them for future issues like these. PEP 517 takes us most of the way to where we want to be, but more work is needed I think.
There was a problem hiding this comment.
If Setuptools does this again in 2026, it will be a repeat of this. How many warnings went unheeded this time? What are the chances those warnings will be heeded in the year to come? It will take years, not months, to untangle.
Concur. Yes, some projects will fix the issue in the meantime (and some did in response to this). But there are still a fair number of basically-unmaintained packages out there, and pushing this back by a year won't solve that.
If the plan is to try this again in 2026, the package to be forked will be setuptools.
Agree. I'm not saying that's how it should be, but that's a potential outcome if this still goes forward.
I know from personal experience how stressful and thankless a job it can be maintaining open source tools that hundreds of thousands or millions of devs depend on.
One of the key responsibilities from that is knowing that if you introduce a breaking change -- no matter HOW well you pre-communicate it -- you're potentially breaking tons of people, and some of them will simply settle on an old version and refuse to upgrade from then on. If enough of them are upset the project gets forked. The usual escape valve to prevent this is offering an override option to restore the previous behavior. It sucks, it can be maddening being so restricted in the ability to move beyond past technical/feature mistakes. But you learn to pick and choose your battles, and something like underscores-vs-hyphens usually isn't worth the fight.
TL;DR: "With great power comes great responsibility" is the mantra for critical open source projects -- writ large in all-caps bold.
A naïve question: Why couldn't it always allow |
|
Could you consider yanking the existing release on PyPi? |
This is a stepping stone, ideally we would like to remove the special handling and simplify the code, for now we left it because it is useful to display the error and prompt package developers to change. The funny part is that this helped to unveil a separated bug on all the packages that use In #4913 I am tracking this problem from setuptools side (devs should be informed if they provide invalid config and long term we should not allow that with the correct error, otherwise package authors think that they are providing important information to PyPI and in reality they are not). |
|
@abravalheri Resurfacing the same ask from before and other places. Can we just yank the problematic version from pypi for now? Let's please first unblock folks around the world and work on the fix in this PR separately in parallel! |
|
As fun as crashing an uncountable number of systems while tests are figured out, removing the file making everyone's lives miserable might be a good triage step. |
Failures are apparently unrelated but needs more investigation.
|
@abravalheri can you please yank the problematic release until you have resolved this revert? This has disrupted a MASSIVE amount of developers and yanking would instantly unblock them. |
|
|
|
I see 78.0 was yanked but 78.0.1 is still there. Can we yank that as well since that still contains the same breaking code? |
Not sure, I'll try |
|
78.0.2 is out: https://pypi.org/project/setuptools/78.0.2/ |
In `setuptools==78.0.1`, they started enforcing that no dependencies specify this legacy field (pypa/setuptools#4870). They ended up reverting in pypa/setuptools#4911, but presumably they'll turn this behavior back on in the future...
An SO on the possible origins of |
|
Please see this open issue Urgent: Governance and Change Management Review After Breaking Change in #4870 |
In `setuptools==78.0.1`, they started enforcing that no dependencies specify this legacy field (pypa/setuptools#4870). They ended up reverting in pypa/setuptools#4911, but presumably they'll turn this behavior back on in the future...
Summary of changes
Closes #4910
Pull Request Checklist
newsfragments/.(See documentation for details)