Skip to content

parallelize pip wheel #8708

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mmerickel opened this issue Aug 5, 2020 · 1 comment
Open

parallelize pip wheel #8708

mmerickel opened this issue Aug 5, 2020 · 1 comment
Labels
type: feature request Request for a new feature

Comments

@mmerickel
Copy link

What's the problem this feature will solve?
pip wheel -r requirements.txt may trigger building wheels for a lot of packages at once and currently every package is converted to a wheel sequentially which may take a lot of time. Especially packages that override the build step to run custom scripts.

Describe the solution you'd like
pip wheel should automatically build packages in parallel, or with a --parallel option on the CLI.

Additional context
I'm not clear in what scenarios would cause problems building packages in parallel into wheels. It seems most cases would not be an issue because packages can be built without being installed, indicating that they can be built in isolation.

@ghost ghost added the S: needs triage Issues/PRs that need to be triaged label Aug 5, 2020
@pradyunsg pradyunsg added the type: feature request Request for a new feature label Aug 12, 2020
@ghost ghost removed the S: needs triage Issues/PRs that need to be triaged label Aug 12, 2020
@pradyunsg
Copy link
Member

This has been brought up in #8187, but I'll keep this open as a separate feature request. There's a lot of related discussion in that issue, which is definitely worth a read if you're interested in this.

As far as I can tell, this would involve a fair amount of work, and it's not immediately clear what the implications of such a change might be. Given the (almost entirely) volunteer nature of developer time toward pip, I wouldn't recommend holding your breath in terms of how quickly this gets done. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature request Request for a new feature
Projects
None yet
Development

No branches or pull requests

2 participants