From 2e9fc40c5a136103ac41a6ffe7c9baf91a85283f Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Mon, 5 Sep 2022 12:42:35 +0200 Subject: [PATCH 01/18] Document upgrade 22.06 -> 22.09 --- modules/ROOT/pages/release_notes.adoc | 205 ++++++++++++++++++++++++++ 1 file changed, 205 insertions(+) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 69abcdf43..68cc06f8d 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -3,6 +3,205 @@ The Stackable platform consists of multiple operators that work together. Periodically a platform release is made, including all components of the platform at a specific version. +== Release 22.09 +This is our second release of the Stackable Data Platform. +It contains lots of new features and bugfixes. +The main features focus on OpenShift support and security. + +=== New platform features +The following new major platform features were added: + +OpenShift compatibility:: +The following operators are now OpenShift compatible: + +* https://github.com/stackabletech/airflow-operator/pull/127[airflow] +* https://github.com/stackabletech/hbase-operator/pull/232[hbase] +* https://github.com/stackabletech/spark-k8s-operator/pull/126[spark-k8s] + +Support for internal and external TLS:: +The following operators support operating the products at a maximal level of transport security by using TLS certificates to secure internal and external communication: + +* https://github.com/stackabletech/trino-operator/pull/244[trino] +* https://github.com/stackabletech/zookeeper-operator/pull/479[zookeeper] + +LDAP authentication:: +Use a central LDAP server to manage all of your user identities in a single place. +The following operators added support for LDAP authentication: + +* https://github.com/stackabletech/airflow-operator/pull/133[airflow] +* https://github.com/stackabletech/nifi-operator/pull/303[nifi] +* https://github.com/stackabletech/superset-operator/pull/180[superset] + +=== stackablectl + +`stackablectl` now supports deploying ready-to-use demos, which give and end-to-end demonstration of the usage of the Stackable data platform. +The xref:stackablectl::quickstart.adoc[quickstart guide] shows how to get started with `stackablectl`. Here you can see the xref:stackablectl::demos/index.adoc[available demos]. + +=== Supported Kubernetes versions +This release supports the following Kubernetes versions: + +* `1.24` +* `1.23` +* `1.22` + +Support for `1.21` was dropped. + +=== Upgrade from 22.06 +==== Using stackablectl +You can list the available releases as follows + +[source,console] +---- +$ stackablectl release list +RELEASE RELEASE DATE DESCRIPTION +22.06 2022-06-30 First official release of the Stackable Data Platform +22.09 2022-09-09 Second release focusing on OpenShift support and security +---- + +To uninstall the `22.06` release run + +[source,console] +---- +$ stackablectl release uninstall 22.06 +[INFO ] Uninstalling release 22.06 +[INFO ] Uninstalling airflow operator +[INFO ] Uninstalling commons operator +# ... +---- + +To install the `22.09` release run + +[source,console] +---- +$ stackablectl release install 22.09 +[INFO ] Installing release 22.09 +[INFO ] Installing airflow operator in version 0.5.0 +[INFO ] Installing commons operator in version 0.3.0 +[INFO ] Installing druid operator in version 0.7.0 +[INFO ] Installing hbase operator in version 0.4.0 +[INFO ] Installing hdfs operator in version 0.5.0 +[INFO ] Installing hive operator in version 0.7.0 +[INFO ] Installing kafka operator in version 0.7.0 +[INFO ] Installing nifi operator in version 0.7.0 +[INFO ] Installing opa operator in version 0.10.0 +[INFO ] Installing secret operator in version 0.6.0 +[INFO ] Installing spark-k8s operator in version 0.5.0 +[INFO ] Installing superset operator in version 0.6.0 +[INFO ] Installing trino operator in version 0.5.0 +[INFO ] Installing zookeeper operator in version 0.11.0 +# ... +---- + +==== Using helm +Use `helm list` to list the currently installed operators. + +You can use the following command to install all of the operators that are part of the release 22.06: + +[source,console] +---- +$ helm uninstall airflow-operator commons-operator druid-operator hbase-operator hdfs-operator hive-operator kafka-operator nifi-operator opa-operator secret-operator spark-k8s-operator superset-operator trino-operator zookeeper-operator +release "airflow-operator" uninstalled +release "commons-operator" uninstalled +# ... +---- + +To install the release 22.09 run + +[source,console] +---- +$ helm repo add stackable https://repo.stackable.tech/repository/helm-stable/ +$ helm install --wait airflow-operator stackable/airflow-operator --version 0.5.0 +$ helm install --wait commons-operator stackable/commons-operator --version 0.3.0 +$ helm install --wait druid-operator stackable/druid-operator --version 0.7.0 +$ helm install --wait hbase-operator stackable/hbase-operator --version 0.4.0 +$ helm install --wait hdfs-operator stackable/hdfs-operator --version 0.5.0 +$ helm install --wait hive-operator stackable/hive-operator --version 0.6.0 +$ helm install --wait kafka-operator stackable/kafka-operator --version 0.6.0 +$ helm install --wait nifi-operator stackable/nifi-operator --version 0.6.0 +$ helm install --wait opa-operator stackable/opa-operator --version 0.10.0 +$ helm install --wait secret-operator stackable/secret-operator --version 0.6.0 +$ helm install --wait spark-k8s-operator stackable/spark-k8s-operator --version 0.5.0 +$ helm install --wait superset-operator stackable/superset-operator --version 0.6.0 +$ helm install --wait trino-operator stackable/trino-operator --version 0.5.0 +$ helm install --wait zookeeper-operator stackable/zookeeper-operator --version 0.11.0 +---- + +==== Breaking changes + +You will need to adapt your existing CRDs to the following breaking changes: + +==== druid-operator +1. HDFS deep storage is now configurable via HDFS discovery config map instead of an url to a HDFS name node (https://github.com/stackabletech/druid-operator/pull/262[#262]). +Instead of + +[source,yaml] +---- + deepStorage: + hdfs: + storageDirectory: hdfs://druid-hdfs-namenode-default-0:8020/data +---- + +use + +[source,yaml] +---- + deepStorage: + hdfs: + configMapName: druid-hdfs + directory: /druid +---- + +==== trino-operator +1. TrinoCatalogs now have their own CRD object and get referenced by the TrinoCluster (https://github.com/stackabletech/trino-operator/pull/263[#263]). Instead of + +[source,yaml] +---- +apiVersion: trino.stackable.tech/v1alpha1 +kind: TrinoCluster +# ... +spec: + hiveConfigMapName: hive + s3: + inline: + host: minio + port: 9000 + accessStyle: Path + credentials: + secretClass: s3-credentials + # ... +---- + +use + +[source,yaml] +---- +apiVersion: trino.stackable.tech/v1alpha1 +kind: TrinoCluster +# ... +spec: + catalogLabelSelector: + trino: simple-trino + # ... +--- +apiVersion: trino.stackable.tech/v1alpha1 +kind: TrinoCatalog +metadata: + name: hive + labels: + trino: trino +spec: + connector: + hive: + metastore: + configMap: hive + s3: + inline: + host: minio + port: 9000 + accessStyle: Path + credentials: + secretClass: s3-credentials +---- == Release 22.06 @@ -98,3 +297,9 @@ Read up on the xref:operators:supported_versions.adoc[supported versions] for ea * xref:commons-operator::index.adoc[] (0.2.0) * xref:secret-operator::index.adoc[] (0.5.0) +=== Supported Kubernetes versions +This release supports the following Kubernetes versions: + +* `1.23` +* `1.22` +* `1.21` From be357bc54d4f0b8d5b238f1dea5953eb89a85c4b Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Mon, 5 Sep 2022 13:33:29 +0200 Subject: [PATCH 02/18] Apply suggestions from code review Co-authored-by: Andrew Kenworthy --- modules/ROOT/pages/release_notes.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 68cc06f8d..123f79524 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -34,7 +34,7 @@ The following operators added support for LDAP authentication: === stackablectl -`stackablectl` now supports deploying ready-to-use demos, which give and end-to-end demonstration of the usage of the Stackable data platform. +`stackablectl` now supports deploying ready-to-use demos, which give an end-to-end demonstration of the usage of the Stackable data platform. The xref:stackablectl::quickstart.adoc[quickstart guide] shows how to get started with `stackablectl`. Here you can see the xref:stackablectl::demos/index.adoc[available demos]. === Supported Kubernetes versions @@ -131,7 +131,7 @@ $ helm install --wait zookeeper-operator stackable/zookeeper-operator --version You will need to adapt your existing CRDs to the following breaking changes: ==== druid-operator -1. HDFS deep storage is now configurable via HDFS discovery config map instead of an url to a HDFS name node (https://github.com/stackabletech/druid-operator/pull/262[#262]). +1. HDFS deep storage is now configurable via the HDFS discovery config map instead of a url to a HDFS name node (https://github.com/stackabletech/druid-operator/pull/262[#262]). Instead of [source,yaml] From 1456020198765373f23db35d00914d7fe661295d Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Mon, 5 Sep 2022 13:34:09 +0200 Subject: [PATCH 03/18] Apply suggestions from code review Co-authored-by: Andrew Kenworthy --- modules/ROOT/pages/release_notes.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 123f79524..893ec2be9 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -115,9 +115,9 @@ $ helm install --wait commons-operator stackable/commons-operator --version 0.3. $ helm install --wait druid-operator stackable/druid-operator --version 0.7.0 $ helm install --wait hbase-operator stackable/hbase-operator --version 0.4.0 $ helm install --wait hdfs-operator stackable/hdfs-operator --version 0.5.0 -$ helm install --wait hive-operator stackable/hive-operator --version 0.6.0 -$ helm install --wait kafka-operator stackable/kafka-operator --version 0.6.0 -$ helm install --wait nifi-operator stackable/nifi-operator --version 0.6.0 +$ helm install --wait hive-operator stackable/hive-operator --version 0.7.0 +$ helm install --wait kafka-operator stackable/kafka-operator --version 0.7.0 +$ helm install --wait nifi-operator stackable/nifi-operator --version 0.7.0 $ helm install --wait opa-operator stackable/opa-operator --version 0.10.0 $ helm install --wait secret-operator stackable/secret-operator --version 0.6.0 $ helm install --wait spark-k8s-operator stackable/spark-k8s-operator --version 0.5.0 From 2c02ac9fb52045106719aa1f3faff1e6bcda3873 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Wed, 7 Sep 2022 11:13:49 +0200 Subject: [PATCH 04/18] Update secret-operator version --- modules/ROOT/pages/release_notes.adoc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 893ec2be9..84a864826 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -55,7 +55,7 @@ You can list the available releases as follows $ stackablectl release list RELEASE RELEASE DATE DESCRIPTION 22.06 2022-06-30 First official release of the Stackable Data Platform -22.09 2022-09-09 Second release focusing on OpenShift support and security +22.09 2022-09-09 Second release focusing on security and OpenShift support ---- To uninstall the `22.06` release run @@ -84,7 +84,7 @@ $ stackablectl release install 22.09 [INFO ] Installing kafka operator in version 0.7.0 [INFO ] Installing nifi operator in version 0.7.0 [INFO ] Installing opa operator in version 0.10.0 -[INFO ] Installing secret operator in version 0.6.0 +[INFO ] Installing secret operator in version 0.5.0 [INFO ] Installing spark-k8s operator in version 0.5.0 [INFO ] Installing superset operator in version 0.6.0 [INFO ] Installing trino operator in version 0.5.0 @@ -119,7 +119,7 @@ $ helm install --wait hive-operator stackable/hive-operator --version 0.7.0 $ helm install --wait kafka-operator stackable/kafka-operator --version 0.7.0 $ helm install --wait nifi-operator stackable/nifi-operator --version 0.7.0 $ helm install --wait opa-operator stackable/opa-operator --version 0.10.0 -$ helm install --wait secret-operator stackable/secret-operator --version 0.6.0 +$ helm install --wait secret-operator stackable/secret-operator --version 0.5.0 $ helm install --wait spark-k8s-operator stackable/spark-k8s-operator --version 0.5.0 $ helm install --wait superset-operator stackable/superset-operator --version 0.6.0 $ helm install --wait trino-operator stackable/trino-operator --version 0.5.0 From 9191a3d2096c26a62395047749460fadc6323071 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Wed, 7 Sep 2022 13:27:45 +0200 Subject: [PATCH 05/18] Update modules/ROOT/pages/release_notes.adoc Co-authored-by: Andrew Kenworthy --- modules/ROOT/pages/release_notes.adoc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 84a864826..2cb91dc0d 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -16,8 +16,10 @@ The following operators are now OpenShift compatible: * https://github.com/stackabletech/airflow-operator/pull/127[airflow] * https://github.com/stackabletech/hbase-operator/pull/232[hbase] +* https://github.com/stackabletech/hdfs-operator/pull/225[hdfs] * https://github.com/stackabletech/spark-k8s-operator/pull/126[spark-k8s] + Support for internal and external TLS:: The following operators support operating the products at a maximal level of transport security by using TLS certificates to secure internal and external communication: From 0577677b248f0ddf328d678f7dddee10922120d0 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Wed, 7 Sep 2022 15:03:29 +0200 Subject: [PATCH 06/18] fix intent --- modules/ROOT/pages/release_notes.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 2cb91dc0d..01445a708 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -198,11 +198,11 @@ spec: configMap: hive s3: inline: - host: minio - port: 9000 - accessStyle: Path - credentials: - secretClass: s3-credentials + host: minio + port: 9000 + accessStyle: Path + credentials: + secretClass: s3-credentials ---- == Release 22.06 From 4206a9d0d961a40d828f83673d491062f40a2545 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Wed, 7 Sep 2022 15:04:13 +0200 Subject: [PATCH 07/18] Update release_notes.adoc --- modules/ROOT/pages/release_notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 01445a708..7b0959dc4 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -182,7 +182,7 @@ kind: TrinoCluster # ... spec: catalogLabelSelector: - trino: simple-trino + trino: trino # ... --- apiVersion: trino.stackable.tech/v1alpha1 From 88162bb58841e992a8461153233b519f7002998c Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Thu, 8 Sep 2022 08:21:47 +0200 Subject: [PATCH 08/18] Update release_notes.adoc --- modules/ROOT/pages/release_notes.adoc | 1 + 1 file changed, 1 insertion(+) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 7b0959dc4..7d21911e6 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -24,6 +24,7 @@ Support for internal and external TLS:: The following operators support operating the products at a maximal level of transport security by using TLS certificates to secure internal and external communication: * https://github.com/stackabletech/trino-operator/pull/244[trino] +* https://github.com/stackabletech/kafka-operator/pull/442[kafka] * https://github.com/stackabletech/zookeeper-operator/pull/479[zookeeper] LDAP authentication:: From 26374747a2c08281bb1c69cd93f507f97f657b3e Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Thu, 8 Sep 2022 15:50:58 +0200 Subject: [PATCH 09/18] Update release_notes.adoc --- modules/ROOT/pages/release_notes.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 7d21911e6..838282dbf 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -90,7 +90,7 @@ $ stackablectl release install 22.09 [INFO ] Installing secret operator in version 0.5.0 [INFO ] Installing spark-k8s operator in version 0.5.0 [INFO ] Installing superset operator in version 0.6.0 -[INFO ] Installing trino operator in version 0.5.0 +[INFO ] Installing trino operator in version 0.6.0 [INFO ] Installing zookeeper operator in version 0.11.0 # ... ---- @@ -125,7 +125,7 @@ $ helm install --wait opa-operator stackable/opa-operator --version 0.10.0 $ helm install --wait secret-operator stackable/secret-operator --version 0.5.0 $ helm install --wait spark-k8s-operator stackable/spark-k8s-operator --version 0.5.0 $ helm install --wait superset-operator stackable/superset-operator --version 0.6.0 -$ helm install --wait trino-operator stackable/trino-operator --version 0.5.0 +$ helm install --wait trino-operator stackable/trino-operator --version 0.6.0 $ helm install --wait zookeeper-operator stackable/zookeeper-operator --version 0.11.0 ---- From 496ab45f6bea558c6071f459e84c7db2d29f0417 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Fri, 9 Sep 2022 14:39:42 +0200 Subject: [PATCH 10/18] Pretty product names --- modules/ROOT/pages/release_notes.adoc | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 838282dbf..6ef6b8c89 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -14,26 +14,26 @@ The following new major platform features were added: OpenShift compatibility:: The following operators are now OpenShift compatible: -* https://github.com/stackabletech/airflow-operator/pull/127[airflow] -* https://github.com/stackabletech/hbase-operator/pull/232[hbase] -* https://github.com/stackabletech/hdfs-operator/pull/225[hdfs] -* https://github.com/stackabletech/spark-k8s-operator/pull/126[spark-k8s] +* https://github.com/stackabletech/airflow-operator/pull/127[Apache Airflow] +* https://github.com/stackabletech/hbase-operator/pull/232[Apache HBase] +* https://github.com/stackabletech/hdfs-operator/pull/225[Apache HDFS] +* https://github.com/stackabletech/spark-k8s-operator/pull/126[Apache Spark on K8s] Support for internal and external TLS:: The following operators support operating the products at a maximal level of transport security by using TLS certificates to secure internal and external communication: -* https://github.com/stackabletech/trino-operator/pull/244[trino] -* https://github.com/stackabletech/kafka-operator/pull/442[kafka] +* https://github.com/stackabletech/trino-operator/pull/244[Trino] +* https://github.com/stackabletech/kafka-operator/pull/442[Apache Kafka] * https://github.com/stackabletech/zookeeper-operator/pull/479[zookeeper] LDAP authentication:: Use a central LDAP server to manage all of your user identities in a single place. The following operators added support for LDAP authentication: -* https://github.com/stackabletech/airflow-operator/pull/133[airflow] -* https://github.com/stackabletech/nifi-operator/pull/303[nifi] -* https://github.com/stackabletech/superset-operator/pull/180[superset] +* https://github.com/stackabletech/airflow-operator/pull/133[Apache Airflow] +* https://github.com/stackabletech/nifi-operator/pull/303[Apache NifFi] +* https://github.com/stackabletech/superset-operator/pull/180[Apache Superset] === stackablectl From febe18ee1067ae37b360f3825da7bfa85941eaf4 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Fri, 9 Sep 2022 14:47:36 +0200 Subject: [PATCH 11/18] Add section on uninstalling crds --- modules/ROOT/pages/release_notes.adoc | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 6ef6b8c89..3472e4c55 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -72,6 +72,14 @@ $ stackablectl release uninstall 22.06 # ... ---- +Afterwards you will need to remove all the CustomResourceDefinitions installed by the Stackable Platform. +Please note that this will remove all your installed CustomResources, so make sure to save them. + +[source,console] +---- +$ kubectl get crd -o name | grep '\.stackable\.tech$' | xargs kubectl delete +---- + To install the `22.09` release run [source,console] From b7685d85300bbd58a062ccf0d072010d21767828 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Fri, 9 Sep 2022 15:11:24 +0200 Subject: [PATCH 12/18] Update modules/ROOT/pages/release_notes.adoc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sönke Liebau --- modules/ROOT/pages/release_notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 3472e4c55..44556147e 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -32,7 +32,7 @@ Use a central LDAP server to manage all of your user identities in a single plac The following operators added support for LDAP authentication: * https://github.com/stackabletech/airflow-operator/pull/133[Apache Airflow] -* https://github.com/stackabletech/nifi-operator/pull/303[Apache NifFi] +* https://github.com/stackabletech/nifi-operator/pull/303[Apache NiFi] * https://github.com/stackabletech/superset-operator/pull/180[Apache Superset] === stackablectl From e39143764297f3ba2e03b8c714400d91ddae6565 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Fri, 9 Sep 2022 15:11:31 +0200 Subject: [PATCH 13/18] Update modules/ROOT/pages/release_notes.adoc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Sönke Liebau --- modules/ROOT/pages/release_notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 44556147e..ec771326f 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -25,7 +25,7 @@ The following operators support operating the products at a maximal level of tra * https://github.com/stackabletech/trino-operator/pull/244[Trino] * https://github.com/stackabletech/kafka-operator/pull/442[Apache Kafka] -* https://github.com/stackabletech/zookeeper-operator/pull/479[zookeeper] +* https://github.com/stackabletech/zookeeper-operator/pull/479[Apache ZooKeeper] LDAP authentication:: Use a central LDAP server to manage all of your user identities in a single place. From 27782bfa7878426666fc36495d83f1be8f924e45 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Mon, 12 Sep 2022 09:59:40 +0200 Subject: [PATCH 14/18] Apply new CRDs instead of deleting the old ones --- modules/ROOT/pages/release_notes.adoc | 61 ++++++++++++++++++++++++--- 1 file changed, 56 insertions(+), 5 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index ec771326f..acaf50d3b 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -57,8 +57,8 @@ You can list the available releases as follows ---- $ stackablectl release list RELEASE RELEASE DATE DESCRIPTION -22.06 2022-06-30 First official release of the Stackable Data Platform 22.09 2022-09-09 Second release focusing on security and OpenShift support +22.06 2022-06-30 First official release of the Stackable Data Platform ---- To uninstall the `22.06` release run @@ -72,12 +72,25 @@ $ stackablectl release uninstall 22.06 # ... ---- -Afterwards you will need to remove all the CustomResourceDefinitions installed by the Stackable Platform. -Please note that this will remove all your installed CustomResources, so make sure to save them. +Afterwards you will need to update the CustomResourceDefinitions (CRDs) installed by the Stackable Platform. +The reason is, that helm will uninstall the operators but not the CRDs. [source,console] ---- -$ kubectl get crd -o name | grep '\.stackable\.tech$' | xargs kubectl delete +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/airflow-operator/0.5.0/deploy/helm/airflow-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/commons-operator/0.3.0/deploy/helm/commons-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/druid-operator/0.7.0/deploy/helm/druid-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hbase-operator/0.4.0/deploy/helm/hbase-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hdfs-operator/0.5.0/deploy/helm/hdfs-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hive-operator/0.7.0/deploy/helm/hive-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/kafka-operator/0.7.0/deploy/helm/kafka-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/nifi-operator/0.7.0/deploy/helm/nifi-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/opa-operator/0.10.0/deploy/helm/opa-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/secret-operator/0.5.0/deploy/helm/secret-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/spark-k8s-operator/0.5.0/deploy/helm/spark-k8s-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/superset-operator/0.6.0/deploy/helm/superset-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/trino-operator/0.6.0/deploy/helm/trino-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/zookeeper-operator/0.11.0/deploy/helm/zookeeper-operator/crds/crds.yaml ---- To install the `22.09` release run @@ -116,11 +129,33 @@ release "commons-operator" uninstalled # ... ---- +Afterwards you will need to update the CustomResourceDefinitions (CRDs) installed by the Stackable Platform. +The reason is, that helm will uninstall the operators but not the CRDs. + +[source,console] +---- +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/airflow-operator/0.5.0/deploy/helm/airflow-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/commons-operator/0.3.0/deploy/helm/commons-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/druid-operator/0.7.0/deploy/helm/druid-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hbase-operator/0.4.0/deploy/helm/hbase-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hdfs-operator/0.5.0/deploy/helm/hdfs-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hive-operator/0.7.0/deploy/helm/hive-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/kafka-operator/0.7.0/deploy/helm/kafka-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/nifi-operator/0.7.0/deploy/helm/nifi-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/opa-operator/0.10.0/deploy/helm/opa-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/secret-operator/0.5.0/deploy/helm/secret-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/spark-k8s-operator/0.5.0/deploy/helm/spark-k8s-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/superset-operator/0.6.0/deploy/helm/superset-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/trino-operator/0.6.0/deploy/helm/trino-operator/crds/crds.yaml +$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/zookeeper-operator/0.11.0/deploy/helm/zookeeper-operator/crds/crds.yaml +---- + To install the release 22.09 run [source,console] ---- $ helm repo add stackable https://repo.stackable.tech/repository/helm-stable/ +$ helm repo update stackable $ helm install --wait airflow-operator stackable/airflow-operator --version 0.5.0 $ helm install --wait commons-operator stackable/commons-operator --version 0.3.0 $ helm install --wait druid-operator stackable/druid-operator --version 0.7.0 @@ -162,8 +197,24 @@ use directory: /druid ---- +==== kafka-operator +1. Add TLS encryption and authentication support for internal and client communications. This is breaking for clients because the cluster is secured per default, which results in a client port change (https://github.com/stackabletech/kafka-operator/pull/442[#442]). +If you don't want to use TLS to secure your Kafka cluster you can restore the old behavior by using the `tls` attribute as follows: + +[source,yaml] +---- +apiVersion: kafka.stackable.tech/v1alpha1 +kind: KafkaCluster +# ... +spec: + config: + tls: null + # ... +---- + ==== trino-operator -1. TrinoCatalogs now have their own CRD object and get referenced by the TrinoCluster (https://github.com/stackabletech/trino-operator/pull/263[#263]). Instead of +1. TrinoCatalogs now have their own CRD object and get referenced by the TrinoCluster (https://github.com/stackabletech/trino-operator/pull/263[#263]). +Instead of [source,yaml] ---- From e48231b070b4772bb233ced80806c2b15aead9b1 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Mon, 12 Sep 2022 16:32:57 +0200 Subject: [PATCH 15/18] Update modules/ROOT/pages/release_notes.adoc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Teo Klestrup Röijezon --- modules/ROOT/pages/release_notes.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index acaf50d3b..7fa83b8db 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -37,7 +37,7 @@ The following operators added support for LDAP authentication: === stackablectl -`stackablectl` now supports deploying ready-to-use demos, which give an end-to-end demonstration of the usage of the Stackable data platform. +`stackablectl` now supports deploying ready-to-use demos, which give an end-to-end demonstration of the usage of the Stackable Data Platform. The xref:stackablectl::quickstart.adoc[quickstart guide] shows how to get started with `stackablectl`. Here you can see the xref:stackablectl::demos/index.adoc[available demos]. === Supported Kubernetes versions From 34dc9b81a184ebd4ffdb29a6cce0efc351d13402 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Tue, 13 Sep 2022 11:46:11 +0200 Subject: [PATCH 16/18] Make kubectl command single liner --- modules/ROOT/pages/release_notes.adoc | 29 ++++++++++++++------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 7fa83b8db..189ee0ac4 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -134,20 +134,21 @@ The reason is, that helm will uninstall the operators but not the CRDs. [source,console] ---- -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/airflow-operator/0.5.0/deploy/helm/airflow-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/commons-operator/0.3.0/deploy/helm/commons-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/druid-operator/0.7.0/deploy/helm/druid-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hbase-operator/0.4.0/deploy/helm/hbase-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hdfs-operator/0.5.0/deploy/helm/hdfs-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/hive-operator/0.7.0/deploy/helm/hive-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/kafka-operator/0.7.0/deploy/helm/kafka-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/nifi-operator/0.7.0/deploy/helm/nifi-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/opa-operator/0.10.0/deploy/helm/opa-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/secret-operator/0.5.0/deploy/helm/secret-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/spark-k8s-operator/0.5.0/deploy/helm/spark-k8s-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/superset-operator/0.6.0/deploy/helm/superset-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/trino-operator/0.6.0/deploy/helm/trino-operator/crds/crds.yaml -$ kubectl apply -f https://raw.githubusercontent.com/stackabletech/zookeeper-operator/0.11.0/deploy/helm/zookeeper-operator/crds/crds.yaml +$ kubectl apply \ + -f https://raw.githubusercontent.com/stackabletech/airflow-operator/0.5.0/deploy/helm/airflow-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/commons-operator/0.3.0/deploy/helm/commons-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/druid-operator/0.7.0/deploy/helm/druid-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/hbase-operator/0.4.0/deploy/helm/hbase-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/hdfs-operator/0.5.0/deploy/helm/hdfs-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/hive-operator/0.7.0/deploy/helm/hive-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/kafka-operator/0.7.0/deploy/helm/kafka-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/nifi-operator/0.7.0/deploy/helm/nifi-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/opa-operator/0.10.0/deploy/helm/opa-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/secret-operator/0.5.0/deploy/helm/secret-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/spark-k8s-operator/0.5.0/deploy/helm/spark-k8s-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/superset-operator/0.6.0/deploy/helm/superset-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/trino-operator/0.6.0/deploy/helm/trino-operator/crds/crds.yaml \ + -f https://raw.githubusercontent.com/stackabletech/zookeeper-operator/0.11.0/deploy/helm/zookeeper-operator/crds/crds.yaml ---- To install the release 22.09 run From 6c8c20e92178330dcb306c519804eadfca6012e2 Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Tue, 13 Sep 2022 13:34:23 +0200 Subject: [PATCH 17/18] Update section on OpenShift --- modules/ROOT/pages/release_notes.adoc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 189ee0ac4..1fc733975 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -12,7 +12,9 @@ The main features focus on OpenShift support and security. The following new major platform features were added: OpenShift compatibility:: -The following operators are now OpenShift compatible: +As a first step towards certification of our own SDP operators we are implementing stricter security rules so that compatibility is satisfied. +This work will be iterative in nature but similar security features - mainly the fact that product Pods must run with a custom `ServiceAccount` and `SecurityContextConstraint` - will be added to other operators over time. +With this release the following operators have been improved with the previously mentioned features: * https://github.com/stackabletech/airflow-operator/pull/127[Apache Airflow] * https://github.com/stackabletech/hbase-operator/pull/232[Apache HBase] From 23a1fa18d46ca5f36faa40ba9fa8b16a7b94d0ed Mon Sep 17 00:00:00 2001 From: Sebastian Bernauer Date: Tue, 13 Sep 2022 16:07:57 +0200 Subject: [PATCH 18/18] oepnshift section --- modules/ROOT/pages/release_notes.adoc | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/modules/ROOT/pages/release_notes.adoc b/modules/ROOT/pages/release_notes.adoc index 1fc733975..7489ab686 100644 --- a/modules/ROOT/pages/release_notes.adoc +++ b/modules/ROOT/pages/release_notes.adoc @@ -12,16 +12,14 @@ The main features focus on OpenShift support and security. The following new major platform features were added: OpenShift compatibility:: -As a first step towards certification of our own SDP operators we are implementing stricter security rules so that compatibility is satisfied. -This work will be iterative in nature but similar security features - mainly the fact that product Pods must run with a custom `ServiceAccount` and `SecurityContextConstraint` - will be added to other operators over time. -With this release the following operators have been improved with the previously mentioned features: +We have made continued progress towards OpenShift compability, and the following operators can now be previewed on OpenShift. +Further improvements are expected in future releases, but no stability or compatibility guarantees are currently made for OpenShift clusters. * https://github.com/stackabletech/airflow-operator/pull/127[Apache Airflow] * https://github.com/stackabletech/hbase-operator/pull/232[Apache HBase] * https://github.com/stackabletech/hdfs-operator/pull/225[Apache HDFS] * https://github.com/stackabletech/spark-k8s-operator/pull/126[Apache Spark on K8s] - Support for internal and external TLS:: The following operators support operating the products at a maximal level of transport security by using TLS certificates to secure internal and external communication: