Skip to content

Run Flux LoRAs on Replicate #1395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
May 5, 2025

Conversation

zeke
Copy link
Contributor

@zeke zeke commented Apr 28, 2025

This PR updates the ReplicateTextToImageTask method to support running arbitrary LoRAs on Replicate.

How it works:

  1. Detect LoRAs by their tags before inference.
  2. Use https://replicate.com/black-forest-labs/flux-dev-lora for inference
  3. Use the requested LoRA as the weights_url input to the flux-dev-lora model

Companion PR

Testing

Remove the skip from describe.skip("InferenceClient" ... in InferenceClient.spec.ts

Then:

$ pnpx vitest run --config vitest.config.mts -t "Replicate"

@zeke zeke marked this pull request as ready for review April 28, 2025 18:34
Copy link
Contributor

@SBrandeis SBrandeis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice and simple!
For information, mapping lora models requires to create a different kind of "inference provider mapping" (tag-based).
See the (wip) documentation PR here for more info: huggingface/hub-docs#1717

Copy link
Contributor

@hanouticelina hanouticelina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@zeke
Copy link
Contributor Author

zeke commented May 1, 2025

Thanks for all the helpful reviews! Made another pass of updates. I think this is ready to go. 🚀

Copy link
Contributor

@SBrandeis SBrandeis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
I will let @hanouticelina do a final pass on this then let's merge!

Copy link
Contributor

@hanouticelina hanouticelina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you @zeke for the PR! let's first update the mapping as mentioned in this internal slack message before merging.

@zeke
Copy link
Contributor Author

zeke commented May 5, 2025

update the mapping

☝🏼 Done! replicate/huggingface-model-mappings#12

@hanouticelina hanouticelina merged commit 1f38698 into huggingface:main May 5, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants