docs: merge Threeport user docs into Qleet docs (#35)

* docs: merge Threeport user docs into Qleet docs

Signed-off-by: Rich Lander <[email protected]>

* docs: add getting started guide

Signed-off-by: Rich Lander <[email protected]>

* docs: fix typos

Signed-off-by: Rich Lander <[email protected]>

---------

Signed-off-by: Rich Lander <[email protected]>

Author: lander2k2

.gitignore

@@ -1,2 +1,3 @@
 site/
-.cache/
+.cache/
+threeport-merge/bin

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "threeport-user-docs"]
+	path = threeport-user-docs
+	url = [email protected]:threeport/user-docs.git

Makefile

@@ -11,7 +11,7 @@ help:
 
 #deps: @ Install dependencies
 deps:
-	@pip install mkdocs mkdocs-material mkdocs-material-extensions mike  
+	@pip install mkdocs mkdocs-material mkdocs-material-extensions mike
 
 #run: @ Run mkdocs
 run:
@@ -33,3 +33,12 @@ release:
 #version: @ Print current version(tag)
 version:
 	@echo $(shell git describe --tags --abbrev=0)
+
+#build-threeport-merge: @ Build theeport-merge binary
+build-threeport-merge:
+	go build -o threeport-merge/bin/threeport-merge ./threeport-merge
+
+#threeport-merge: @ Build and run threeport-merge to update qleet user docs with threeport user docs
+threeport-merge: build-threeport-merge
+	./threeport-merge/bin/threeport-merge -config-file threeport-merge/merge-config.yaml
+

README.md

@@ -38,3 +38,64 @@ stop    - Stop mkdocs
 release - Create and push a new tag
 version - Print current version(tag)
 ```
+
+## Merge Threeport Docs
+
+In order to have the open source Threeport user docs nested in the Qleet docs we
+do the following:
+
+* Keep the upstream Threeport docs as a git submodule of Qleet docs
+* Use a merge tool to add the Threeport docs into the Qleet docs.  The source
+  code for this tool is in the `threeport-merge` directory.
+
+The `threeport-merge` tool does the following:
+
+* Copies all Threeport markdown docs into the `docs/threeport` directory.
+* Copies all images from Threeport docs into the `docs/img/threeport` directory.
+* Updates all instances of `tptctl` in markdown docs with `qleetctl`.
+* Updates all paths to images in markdown docs to point to its new home in Qleet
+  docs.
+* Updates the `.nav` in Qleet's `mkdocs.yml` config to include the Threeport
+  docs `.nav` with updated paths to markdown docs.
+* Removes any files found in the threeport-merge config file under `.exclude`.
+
+### threeport-merge Config
+
+Sample config:
+
+```yaml
+exclude:
+  - docs/threeport/guides/getting-started.md
+  - docs/threeport/guides/install-tptctl.md
+  - docs/threeport/guides/install-threeport-local.md
+  - docs/threeport/guides/install-threeport-aws.md
+  - docs/threeport/guides/deploy-workload-local.md
+```
+
+If there are any documents in the Threeport user docs that aren't appropriate in
+Qleet user docs and should be excluded, add the path to the `.exclude` list and
+they will be omitted.  The path provided must be the path to the Threeport
+document in the Qleet docs where it is found post-merge, i.e. the path to the
+document in the `docs/threeport` directory after `threeport-merge` is run.
+
+### Run Merge
+
+If there are changes in Threeport docs that need to be pulled into the Threeport
+user-docs submodule, first pull those changes.
+
+```bash
+git submodule update --remote
+```
+
+Then run `threeport-merge` with a make target.
+
+```bash
+make threeport-merge
+```
+
+This make target will:
+
+* Build threeport-merge from source.
+* Run threeport-merge and reference the `threeport-merge/merge-config.yaml`
+  config.
+

docs/faq.md

@@ -0,0 +1,27 @@
+# Frequently Asked Questions
+
+## Should I use separate Threeport control planes for dev, staging and prod?
+
+In most cases, we don't recommend this.  It makes promoting changes through the
+tiers to production more troublesome.  Managing each tier through a single
+control plane provides a smoother experience.
+
+If your organization has different departments or lines of business that you'd
+like to segregate and that don't overlap in the workloads they manage, separate
+Threeport control planes makes more sense in that case.
+
+## Can I add Extensions to my Qleet-managed Threeport control plane?
+
+For security reasons, we cannot allow our users to run Threeport extensions that
+they build on Qleet.  However, we can build extensions for you and install them
+on your existing control plane/s.  Shoot us an email at <[email protected]> if
+you want to explore that option.
+
+Your other option is to use Qleet Enterprise.  This product involves running the
+entire Qleet service on your AWS account with full support from Qleet.  In that
+case, you are free to internally develop Threeport extensions and add them to
+your control plane.  To find out more, email us at <[email protected]>.
+
+More information about Threeport extensions can be found in our [SDK
+Introduction document](../threeport/sdk/sdk-intro).
+

docs/getting-started.md

@@ -0,0 +1,14 @@
+# Getting Started
+
+Follow these steps to get up and running with Qleet:
+
+1. First, [install qleetctl](/guides/install-qleetctl), the Qleet command line tool.
+1. Next, create a new account with Qleet.  (Coming soon.)
+1. Register and configure your AWS account in Qleet using our [Add AWS Account
+   guide](/guides/add-aws-account).
+1. Create a Kubernetes Runtime in your AWS account for your workloads by
+   following the [Remote Kubernetes Runtime
+   guide](/threeport/kubernetes-runtime/remote-kubernetes-runtime).
+1. Optional: Add another user on your team by following the [Add User
+   guide](/guides/add-user).
+

docs/guides/add-aws-account.md

@@ -0,0 +1,37 @@
+# Add AWS Account
+
+## Prerequisites
+
+If you haven't already, [install qleetctl](/guides/install-qleetctl), the Qleet
+command line tool.
+
+## Add Account
+
+The following steps register and configure your AWS account in Qleet.  Once
+registered, you can spin up Kubernetes Runtimes and Workloads in your AWS account
+using Qleet.
+
+1. Set user environment variables by creating a file called `config-env-var`
+
+    ```bash
+    export region=<region>          # your default AWS region
+    export email=<email>            # your user email
+    export profile=<profile>        # your AWS profile name
+    export accountName=<account>    # your Qleet account name
+
+    # provided by Qleet
+    export controlPlaneName=dev
+    ```
+
+
+1. Register your account and configure AWS:
+
+    ```bash
+    qleetctl config aws-account \
+        --aws-account-name default-account \
+        --aws-region $region \
+        --aws-profile $profile \
+        --aws-account-id 983530947477 \
+        --external-runtime-manager-role-name resource-manager-threeport-$controlPlaneName-$accountName
+    ```
+

docs/guides/add-user.md

@@ -0,0 +1,83 @@
+# Add User to Qleet Account
+
+Follow these steps to add a new user to your Qleet account.
+
+## Prerequisites
+
+If you haven't already, [install qleetctl](/guides/install-qleetctl), the Qleet
+command line tool.
+
+## Add User
+
+1. Set user password by creating a file called `/tmp/qleet-password-env`:
+
+    ```bash
+    echo "export password=<password>" > /tmp/qleet-password-env
+    ```
+
+1. Source password by running `source` on the above file
+
+    ```bash
+    source /tmp/qleet-password-env
+    ```
+
+1. Set user environment variables by creating a file called `config-env-var`
+
+    ```bash
+    export region=<region>     # your default AWS region
+    export email=<email>       # your user email
+
+    # provided by Qleet
+    export controlPlaneName=dev
+    export accountName=pos-tech
+    ```
+
+1. Source config by running `source` on the above file
+
+    ```bash
+    source config-env-var
+    ```
+
+1. Register Qleet user
+
+    ```bash
+    qleetctl register user --account $accountName --username $email --password $password
+    ```
+
+1. Check your email (and potentially spam folder) for an email from `[email protected]` that looks like this
+
+    ```bash
+    Hi, please verify your account by using the following code: 894367, URL: http://localhost:31500/kratos/self-service/verification?code=894367&flow=b7182149-5f2a-40e2-b7e9-be59ce608171
+    ```
+
+1. Add second user to account.  Second user repeats steps to register above for onboarding.
+
+    ```jsx
+    qleetctl invite user --userid <userEmail>
+    ```
+
+1. Update your environment variables with the `code` and `flow` values from the email
+
+    ```bash
+    export code=894367
+    export flowId=b7182149-5f2a-40e2-b7e9-be59ce608171
+    ```
+
+1. Verify Qleet account
+
+    ```bash
+    qleetctl verify user --account $accountName --code $code --flow $flowId
+    ```
+
+1. Log in to Qleet account
+
+    ```bash
+    qleetctl login user --account $accountName --username $email --password $password
+    ```
+
+1. Set current control plane instance
+
+    ```bash
+    qleetctl config current-control-plane --name=$controlPlaneName
+    ```
+

docs/guides/install.md renamed to docs/guides/install-qleetctl.md

@@ -4,7 +4,7 @@ Use the following instructions to install the `qleetctl` command line tool for
 using Qleet.
 
 1. Visit the [releases page](https://github.com/qleet/resources/releases) on
-   github and download:
+   GitHub and download:
 
     1. `checksums.txt`
 

docs/img/threeport/APIDrivenDelivery.png

@@ -1,32 +1,26 @@
 # Qleet
 
-Qleet is a managed [Threeport](https://threeport.io/) offering.
-
-This site provides user documentation specific to the Qleet platform.  Since it
-provides fully managed Threeport control planes, the [Threeport
-Documentation](https://docs.threeport.io/) is still applicable and should be
-referenced for using it.
-
-### Note
-
-The open source Threeport project provides the `tptctl` command line tool for
-interacting with Threeport.  Qleet uses an extension of `tptctl` called `qleetctl`
-which includes all the functionality of `tptctl` with added features specific to
-the Qleet platform.
-
-Qleet users should always use `qleetctl` instead of `tptctl`.  Any command examples
-in the Threeport docs that provide commands for `tptctl`, just substitute qleetctl
-and use the same subcommands.
-
-For example...
-
-```bash
-tptctl create workload -c my-workload.yaml`
-```
-
-simply becomes...
-
-```bash
-qleetctl create workload -c my-workload.yaml
-```
+Welcome to Qleet, the managed [Threeport](https://threeport.io/) service
+provided by the creators of the Threeport project.
+
+Qleet provides all the functionality of open source Threeport, plus the
+following:
+
+* Managed Threeport Control Planes: Qleet users do not have to install, upgrade
+  or manage Threeport control planes.  The Qleet service provides them for its
+  users.
+* Role Based Access Control (RBAC): Open source Threeport does not implement access
+  control - all users are super-users.  Qleet users can apply RBAC rules to
+  users on their account to provide appropriate access based on roles.
+* 3rd Party Identity Provider Integration (Coming Soon): Qleet currently offers
+  secure authentication to Qleet services and will soon offer integrations with
+  3rd party services to authenticate users.
+* Graphical User Interface (Coming Soon): Qleet currently offers command line
+  access to the system with the `qleetctl` CLI tool.  Soon, we will release a
+  portal so Qleet users can view and manage their resources with a GUI as well.
+
+This site provides user documentation specific to the Qleet platform.  It
+includes documentation for using the managed Threeport control planes.  If you
+are using open source Threeport, see the [Threeport
+Documentation](https://docs.threeport.io/).
 

docs/img/threeport/DecentralizedComputingSolution.png

@@ -0,0 +1,66 @@
+# Kubernetes Federation
+
+This document describes the Threeport approach to managing a fleet of Kubernetes
+clusters.
+
+There have been many attempts at federating Kubernetes using Kubernetes itself,
+i.e. Kubernetes Operators that install and keep an inventory of clusters as well
+as manage multi-cluster app deployments.  Kubernetes was designed to be a
+datacenter-level abstraction and it performs this function very well.  It was
+not designed to be a global software fleet abstraction and it has inherent
+scaling and availability constraints that prevent it from providing the ideal
+solution to this concern.
+
+A global federation layer must be highly scalable and have geo-redundant
+availability.  A control plane for all your software deployments must have the
+appropriate capacity and resilience for the task.
+
+## Kubernetes Controllers
+
+[Kubernetes controllers](https://kubernetes.io/docs/concepts/architecture/controller/)
+are not horizontally scalable.  When deployed in a highly
+available configuration, only one controller is active at any given time and
+they use leader election to determine which of a set of identical controllers
+manage operations at any given time.  In many use-cases, many thousands of
+clusters must be managed coherently, not to mention the software in those
+clusters.  This is a tremendous amount of state reconcilation to be performed by
+controllers that do not share load across multiple instances.
+
+## Kubernetes Datastore
+
+Kubernetes uses [etcd](https://etcd.io/) which is an excellent distributed
+key-value store.  It has served Kubernetes very well in its purpose.  However,
+etcd works best in a single region.  Tuning for the increased latency of
+cross-region etcd clusters is possible, but treacherous.  Furthermore, it is not
+a relational database which means if you need transactional capabilities that
+allow a database to make changes to multiple objects with ACID guarantees, etcd
+is not the best choice.
+
+![Federating Kubernetes with Kubernetes](../../../img/threeport/KubernetesFederationWithKubernetes.png)
+
+## Threeport Controllers
+
+Threeport controllers inherit a lot of design principles from Kubernetes.  They
+are level-triggered state reconciliation workloads that operate on a non-terminating
+loop to ensure your desired state is realized in the system.  One thing that
+Threeport controllers add is horizontal scalability.  Any number of Threeport
+Controllers can operate simultaneously to manage the same set of object types.
+They use NATS Jetstream to broker notifications to help achieve this.  In
+Threeport, the message broker helps ensure a notification of a particular change
+is delivered to just one of a set of identical Threeport controllers.  Threeport
+controllers use the message broker to place distributed locks on specific objects while they are
+being reconciled so that race conditions between controllers don't develop in making
+changes to the system.
+
+## Threeport Datastore
+
+Threeport uses CockroachDB, a purpose-built geo-redundant relational database.  The
+geo-redundancy is essential for a purpose that is this critical.  And the
+transactional capabilities allow changes to multiple related objects to happen
+safely.  When you are dealing with remote clusters and the workloads therein,
+changes that affect multiple objects are common.  Being able to apply a change
+to all the affected objects _or_ none at all if a problem occurs, is an
+important guarantee to have for stability.
+
+![Federating Kubernetes with Threeport](../../../img/threeport/KubernetesFederationWithThreeport.png)
+