Skip to content

feat: patchable init mirroring #1070

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 32 commits into from
May 9, 2025
Merged

feat: patchable init mirroring #1070

merged 32 commits into from
May 9, 2025

Conversation

dervoeti
Copy link
Member

@dervoeti dervoeti commented Apr 30, 2025

Fixes #1049

This adds the --mirrored flag to patchable init. It assumes that a mirror repository already exist, pushes the ref specified via --base to the mirror and configures the mirrorer repo as upstream in patchable.toml.
As discussed with @lfrancke, we want to fork the repos in the stackabletech org and use forks instead of clones (the latter decision was not a strong opinion, I decided to go with forks since they would facilitate back-contributions).

I created mirror repositories (forks in our case) for all our products and synced the tags for all versions we use. I updated all patchable.toml files to use the mirror repositories (with the exception of Omid 1.1.3-SNAPSHOT, which is a temporary version we don't ship).

I had to drop the usage of FETCH_HEAD, since it caused problems when running patchable init for a tag/branch that for some reason was already present in the local Git repo. The problem is that FETCH_HEAD is empty in that case, I think that's due to no changes being fetched.

@dervoeti dervoeti requested a review from nightkr April 30, 2025 12:52
@nightkr
Copy link
Member

nightkr commented Apr 30, 2025

As a nomenclature thing, I think I prefer calling this mirroring instead.

@dervoeti
Copy link
Member Author

As a nomenclature thing, I think I prefer calling this mirroring instead.

Had the same thought when opening the PR 🙂 Will refactor it.

@dervoeti dervoeti changed the title feat: patchable init forking feat: patchable init mirroring Apr 30, 2025
dervoeti and others added 7 commits May 5, 2025 17:47
Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
@dervoeti dervoeti force-pushed the feat/patchable-init-forking branch from 2bf81bd to f85102d Compare May 6, 2025 18:57
@dervoeti dervoeti force-pushed the feat/patchable-init-forking branch from f85102d to 694e5c7 Compare May 6, 2025 19:36
nightkr
nightkr previously approved these changes May 8, 2025
Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
@dervoeti
Copy link
Member Author

dervoeti commented May 8, 2025

@nightkr I would now go ahead and update all the patchable.toml files (create product-level configs and adapt the version-level configs). And update this PR to be compatible with the current main branch.

@nightkr
Copy link
Member

nightkr commented May 8, 2025

Sounds good to me.

@dervoeti dervoeti requested a review from nightkr May 8, 2025 14:45
@dervoeti
Copy link
Member Author

dervoeti commented May 9, 2025

@nightkr This is done now. Product-level configs are all there and I removed upstream from version-level configs and added mirror instead.

@nightkr
Copy link
Member

nightkr commented May 9, 2025

All versions seem to check out fine (ls **/stackable/patches | insert version { ls $in.name -s | filter { $in.type == "dir" } | get name } | insert product { $in.name | path dirname -n 2 } | flatten | each { cargo patchable checkout $in.product $in.version }).

nightkr
nightkr previously approved these changes May 9, 2025
@dervoeti
Copy link
Member Author

dervoeti commented May 9, 2025

@nightkr I noticed on more thing 🙁 Some builds would fail, because we use patchable checkout inside Docker to checkout the source before building the product. For this, the product config is needed.
At least for some images we only copy the specific version directory, which of course does not include the product config. I'll change that, so that the whole patches directory is copied into the image, not just the directory for the specific version.

@nightkr
Copy link
Member

nightkr commented May 9, 2025

Ahh, right. Good catch.

@dervoeti
Copy link
Member Author

dervoeti commented May 9, 2025

@nightkr Fixed in 6148608

@nightkr
Copy link
Member

nightkr commented May 9, 2025

Hm, that would mean losing caching for all versions when any version is modified, though. It should be enough to copy patches/$version and patches/patchable.toml.

@dervoeti
Copy link
Member Author

dervoeti commented May 9, 2025

@nightkr Reverted and fixed by copying only the product config

@dervoeti dervoeti added this pull request to the merge queue May 9, 2025
Merged via the queue into main with commit 037b97d May 9, 2025
3 checks passed
@dervoeti dervoeti deleted the feat/patchable-init-forking branch May 9, 2025 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Migrate all mirrored sources from Nexus to GitHub repositories
2 participants