Skip to content

Commit 037b97d

Browse files
dervoetinightkr
dervoeti
and
nightkr
authored
feat: patchable init mirroring (#1070)
* feat: support forking when initializing * fix: don't use fetchhead / update libgit / migrate patchable.toml files * chore: replace "forking" with "mirroring" * feat: keep original upstream / anonymous remote * refactor: move progress tracking setup into utils function * feat: always use tags in our mirror repos * Update rust/patchable/src/main.rs Co-authored-by: Natalie Klestrup Röijezon <[email protected]> * Update rust/patchable/src/main.rs Co-authored-by: Natalie Klestrup Röijezon <[email protected]> * Update rust/patchable/src/main.rs Co-authored-by: Natalie Klestrup Röijezon <[email protected]> * Update rust/patchable/src/main.rs Co-authored-by: Natalie Klestrup Röijezon <[email protected]> * Update rust/patchable/src/main.rs Co-authored-by: Natalie Klestrup Röijezon <[email protected]> * Update rust/patchable/src/main.rs Co-authored-by: Natalie Klestrup Röijezon <[email protected]> * fix: span behaviour / mirror flag * feat: separate product level configuration * refactor: separate product and version config * fix: adjust README * chore: formatting / unnecessary clone * fix: tracing * fix: readme adjustment * Make mirroring optional again * feat: ssh support * feat: remove url crate * docs: update README * Update rust/patchable/src/main.rs Co-authored-by: Natalie Klestrup Röijezon <[email protected]> * feat: product-level configs for patchable * docs: fixed README.md for markdownlint * chore: removed upstream from version configs / added missing product-level configs * fix: copy complete patches directories to include patchable product config * Revert "fix: copy complete patches directories to include patchable product config" This reverts commit 6148608. * fix: copy patchable product config as well --------- Co-authored-by: Natalie Klestrup Röijezon <[email protected]>
1 parent ab96aea commit 037b97d

File tree

68 files changed

+327
-83
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

68 files changed

+327
-83
lines changed

‎Cargo.lock

Lines changed: 4 additions & 4 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ resolver = "2"
44

55
[workspace.dependencies]
66
clap = { version = "4.5.27", features = ["derive"] }
7-
git2 = "0.20.0"
7+
git2 = "0.20.1"
88
serde = { version = "1.0.217", features = ["derive"] }
99
snafu = "0.8.5"
1010
tempfile = "3.16.0"

‎druid/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ EOF
3232
USER ${STACKABLE_USER_UID}
3333
WORKDIR /stackable
3434

35+
COPY --chown=${STACKABLE_USER_UID}:0 druid/stackable/patches/patchable.toml /stackable/src/druid/stackable/patches/patchable.toml
3536
COPY --chown=${STACKABLE_USER_UID}:0 druid/stackable/patches/${PRODUCT} /stackable/src/druid/stackable/patches/${PRODUCT}
3637

3738
# Cache mounts are owned by root by default

‎druid/stackable/patches/30.0.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/druid.git"
21
base = "09d36ee324747f1407705c27618b6d415c3fa8a9"
2+
mirror = "https://github.com/stackabletech/druid.git"

‎druid/stackable/patches/30.0.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/druid.git"
21
base = "a30af7a91d528e5c3a90356a5592abc7119191c6"
2+
mirror = "https://github.com/stackabletech/druid.git"

‎druid/stackable/patches/31.0.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/druid.git"
21
base = "520482cb9638e452b0553595b4f29bb397a63758"
2+
mirror = "https://github.com/stackabletech/druid.git"

‎druid/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/druid.git"
2+
default-mirror = "https://github.com/stackabletech/druid.git"

‎hadoop/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@ ln -s "/stackable/jmx/jmx_prometheus_javaagent-${JMX_EXPORTER}.jar" /stackable/j
5151
EOF
5252

5353
WORKDIR /build
54+
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/patches/patchable.toml /build/src/hadoop/stackable/patches/patchable.toml
5455
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/patches/${PRODUCT} /build/src/hadoop/stackable/patches/${PRODUCT}
5556
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/fuse_dfs_wrapper /build
5657
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/stackable/jmx /stackable/jmx

‎hadoop/stackable/patches/3.3.4/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hadoop.git"
21
base = "a585a73c3e02ac62350c136643a5e7f6095a3dbb"
2+
mirror = "https://github.com/stackabletech/hadoop.git"

‎hadoop/stackable/patches/3.3.6/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hadoop.git"
21
base = "1be78238728da9266a4f88195058f08fd012bf9c"
2+
mirror = "https://github.com/stackabletech/hadoop.git"

‎hadoop/stackable/patches/3.4.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hadoop.git"
21
base = "bd8b77f398f626bb7791783192ee7a5dfaeec760"
2+
mirror = "https://github.com/stackabletech/hadoop.git"

‎hadoop/stackable/patches/3.4.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hadoop.git"
21
base = "4d7825309348956336b8f06a08322b78422849b1"
2+
mirror = "https://github.com/stackabletech/hadoop.git"

‎hadoop/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/hadoop.git"
2+
default-mirror = "https://github.com/stackabletech/hadoop.git"

‎hbase/Dockerfile

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ COPY hbase/licenses /licenses
2626
USER ${STACKABLE_USER_UID}
2727
WORKDIR /stackable
2828

29+
COPY --chown=${STACKABLE_USER_UID}:0 hbase/stackable/patches/patchable.toml /stackable/src/hbase/stackable/patches/patchable.toml
2930
COPY --chown=${STACKABLE_USER_UID}:0 hbase/stackable/patches/${PRODUCT} /stackable/src/hbase/stackable/patches/${PRODUCT}
3031
COPY --chown=${STACKABLE_USER_UID}:0 hbase/stackable/jmx/config${JMX_EXPORTER} /stackable/jmx
3132

@@ -142,6 +143,7 @@ ARG DELETE_CACHES="true"
142143
# so that they are not expanded. Disabling ShellCheck rules in a Dockerfile
143144
# does not work, so please ignore the according warning (SC2016).
144145
COPY --chown=${STACKABLE_USER_UID}:0 hbase/stackable/bin/hbck2.env /stackable/bin/
146+
COPY --chown=${STACKABLE_USER_UID}:0 hbase/hbase-operator-tools/stackable/patches/patchable.toml /stackable/src/hbase-operator-tools/stackable/patches/patchable.toml
145147
COPY --chown=${STACKABLE_USER_UID}:0 hbase/hbase-operator-tools/stackable/patches/${HBASE_OPERATOR_TOOLS} /stackable/src/hbase-operator-tools/stackable/patches/${HBASE_OPERATOR_TOOLS}
146148
COPY --chown=${STACKABLE_USER_UID}:0 hbase/stackable/bin/hbase-entrypoint.sh /stackable/bin/
147149

@@ -241,6 +243,7 @@ ARG STACKABLE_USER_UID
241243
# This can be used to speed up builds when disk space is of no concern.
242244
ARG DELETE_CACHES="true"
243245

246+
COPY --chown=${STACKABLE_USER_UID}:0 hbase/phoenix/stackable/patches/patchable.toml /stackable/src/phoenix/stackable/patches/patchable.toml
244247
COPY --chown=${STACKABLE_USER_UID}:0 hbase/phoenix/stackable/patches/${PHOENIX} /stackable/src/phoenix/stackable/patches/${PHOENIX}
245248
USER ${STACKABLE_USER_UID}
246249
WORKDIR /stackable

‎hbase/hbase-operator-tools/stackable/patches/1.2.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hbase-operator-tools.git"
21
base = "478af00af79f82624264fd2bb447b97fecc8e790"
2+
mirror = "https://github.com/stackabletech/hbase-operator-tools.git"

‎hbase/hbase-operator-tools/stackable/patches/1.3.0-fd5a5fb/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hbase-operator-tools.git"
21
base = "fd5a5fb90755949a90c502c76de8313130403fa3"
2+
mirror = "https://github.com/stackabletech/hbase-operator-tools.git"

‎hbase/hbase-operator-tools/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/hbase-operator-tools.git"
2+
default-mirror = "https://github.com/stackabletech/hbase-operator-tools.git"

‎hbase/phoenix/stackable/patches/5.2.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/phoenix.git"
21
base = "b738d66cb5863b759bb98eaa417b3b5731d41f95"
2+
mirror = "https://github.com/stackabletech/phoenix.git"

‎hbase/phoenix/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/phoenix.git"
2+
default-mirror = "https://github.com/stackabletech/phoenix.git"

‎hbase/stackable/patches/2.4.18/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hbase.git"
21
base = "a1767f4d76859c0068720a6c1e5cb78282ebfe1e"
2+
mirror = "https://github.com/stackabletech/hbase.git"

‎hbase/stackable/patches/2.6.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hbase.git"
21
base = "de99f8754135ea69adc39da48d2bc2b2710a5366"
2+
mirror = "https://github.com/stackabletech/hbase.git"

‎hbase/stackable/patches/2.6.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hbase.git"
21
base = "7ed50b4dd742269a78875fb32112215f831284ff"
2+
mirror = "https://github.com/stackabletech/hbase.git"

‎hbase/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/hbase.git"
2+
default-mirror = "https://github.com/stackabletech/hbase.git"

‎hive/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ ARG STACKABLE_USER_UID
2323
ARG DELETE_CACHES="true"
2424

2525
# Copy patches into the builder
26+
COPY --chown=${STACKABLE_USER_UID}:0 hive/stackable/patches/patchable.toml /stackable/src/hive/stackable/patches/patchable.toml
2627
COPY --chown=${STACKABLE_USER_UID}:0 hive/stackable/patches/${PRODUCT} /stackable/src/hive/stackable/patches/${PRODUCT}
2728
# It is useful to see which version of Hadoop is used at a glance
2829
# Therefore the use of the full name here

‎hive/stackable/patches/3.1.3/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hive.git"
21
base = "4df4d75bf1e16fe0af75aad0b4179c34c07fc975"
2+
mirror = "https://github.com/stackabletech/hive.git"

‎hive/stackable/patches/4.0.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hive.git"
21
base = "183f8cb41d3dbed961ffd27999876468ff06690c"
2+
mirror = "https://github.com/stackabletech/hive.git"

‎hive/stackable/patches/4.0.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/hive.git"
21
base = "3af4517eb8cfd9407ad34ed78a0b48b57dfaa264"
2+
mirror = "https://github.com/stackabletech/hive.git"

‎hive/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/hive.git"
2+
default-mirror = "https://github.com/stackabletech/hive.git"

‎kafka/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ USER ${STACKABLE_USER_UID}
1515
WORKDIR /stackable
1616

1717
COPY --chown=${STACKABLE_USER_UID}:0 kafka/stackable/jmx/ /stackable/jmx/
18+
COPY --chown=${STACKABLE_USER_UID}:0 kafka/stackable/patches/patchable.toml /stackable/src/kafka/stackable/patches/patchable.toml
1819
COPY --chown=${STACKABLE_USER_UID}:0 kafka/stackable/patches/${PRODUCT} /stackable/src/kafka/stackable/patches/${PRODUCT}
1920

2021
RUN <<EOF

‎kafka/stackable/patches/3.7.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/kafka.git"
21
base = "e2494e6ffb89f8288ed2aeb9b5596c755210bffd"
2+
mirror = "https://github.com/stackabletech/kafka.git"

‎kafka/stackable/patches/3.7.2/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/kafka.git"
21
base = "79a8f2b5f44f9d5a6867190d1dfc463d08d60b82"
2+
mirror = "https://github.com/stackabletech/kafka.git"

‎kafka/stackable/patches/3.8.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/kafka.git"
21
base = "771b9576b00ecf5b64ab6e8bedf04156fbdb5cd6"
2+
mirror = "https://github.com/stackabletech/kafka.git"

‎kafka/stackable/patches/3.9.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/kafka.git"
21
base = "84caaa6e9da06435411510a81fa321d4f99c351f"
2+
mirror = "https://github.com/stackabletech/kafka.git"

‎kafka/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/kafka.git"
2+
default-mirror = "https://github.com/stackabletech/kafka.git"

‎nifi/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ EOF
3030
USER ${STACKABLE_USER_UID}
3131
WORKDIR /stackable
3232

33+
COPY --chown=${STACKABLE_USER_UID}:0 nifi/stackable/patches/patchable.toml /stackable/src/nifi/stackable/patches/patchable.toml
3334
COPY --chown=${STACKABLE_USER_UID}:0 nifi/stackable/patches/${PRODUCT} /stackable/src/nifi/stackable/patches/${PRODUCT}
3435

3536
RUN <<EOF

‎nifi/stackable/patches/1.27.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/nifi.git"
21
base = "e0c4461d90bd4f6e5f2b81765bcff5cd97ed3e18"
2+
mirror = "https://github.com/stackabletech/nifi.git"

‎nifi/stackable/patches/1.28.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/nifi.git"
21
base = "883338fe28883733417d10f6ffa9319e75f5ea06"
2+
mirror = "https://github.com/stackabletech/nifi.git"

‎nifi/stackable/patches/2.2.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/nifi.git"
21
base = "b33ffac8aa10992482f7fa54e6cfccc46a5e8e27"
2+
mirror = "https://github.com/stackabletech/nifi.git"

‎nifi/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/nifi.git"
2+
default-mirror = "https://github.com/stackabletech/nifi.git"

‎omid/Dockerfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ EOF
2121
USER ${STACKABLE_USER_UID}
2222
WORKDIR /stackable
2323

24+
COPY --chown=${STACKABLE_USER_UID}:0 omid/stackable/patches/patchable.toml /stackable/src/omid/stackable/patches/patchable.toml
2425
COPY --chown=${STACKABLE_USER_UID}:0 omid/stackable/patches/${PRODUCT} /stackable/src/omid/stackable/patches/${PRODUCT}
2526

2627
RUN --mount=type=cache,id=maven-omid-${PRODUCT},uid=${STACKABLE_USER_UID},target=/stackable/.m2/repository <<EOF

‎omid/stackable/patches/1.1.0/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/phoenix-omid.git"
21
base = "3b9e16b7537adbc90a7403507fb8aabd8d1fab0c"
2+
mirror = "https://github.com/stackabletech/phoenix-omid.git"

‎omid/stackable/patches/1.1.1/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/phoenix-omid.git"
21
base = "cd546d58d93f380fec9bf65dbfa618f53493f662"
2+
mirror = "https://github.com/stackabletech/phoenix-omid.git"

‎omid/stackable/patches/1.1.2/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/phoenix-omid.git"
21
base = "88812c9e127063f3b3016262f81ea3e8b48ec157"
2+
mirror = "https://github.com/stackabletech/phoenix-omid.git"

‎omid/stackable/patches/1.1.3-SNAPSHOT/patchable.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
1-
upstream = "https://github.com/apache/phoenix-omid.git"
21
base = "c3e4da626fdb27060fd139a809e057965e52d163"
2+
mirror = "https://github.com/stackabletech/phoenix-omid.git"

‎omid/stackable/patches/patchable.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
upstream = "https://github.com/apache/phoenix-omid.git"
2+
default-mirror = "https://github.com/stackabletech/phoenix-omid.git"

‎rust/patchable/README.md

Lines changed: 25 additions & 5 deletions

0 commit comments

Comments
 (0)