Skip to content

Commit aa19427

Browse files
emosbaughemosbaugh
committed
run e2e tests on cmx
1 parent f1b4d63 commit aa19427

38 files changed

+455
-227
lines changed

‎.github/actions/e2e/action.yml

+23
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,15 @@ inputs:
4343
version-specifier:
4444
description: 'the git sha or tag used to generate application version strings'
4545
required: true
46+
cmx-replicated-api-token:
47+
description: 'the replicated api token to use for cmx e2e tests'
48+
required: false
49+
replicatedvm-ssh-user:
50+
description: 'the user to use for ssh to the replicated vm'
51+
required: false
52+
replicatedvm-ssh-key:
53+
description: 'the ssh key to use for ssh to the replicated vm'
54+
required: false
4655
upgrade-target-ec-version:
4756
description: 'the embedded cluster version to expect after upgrades complete'
4857
required: false # this is only set by post-release testing
@@ -91,6 +100,20 @@ runs:
91100
with:
92101
go-version-file: go.mod
93102
cache-dependency-path: "**/*.sum"
103+
- name: Install Replicated CLI
104+
shell: bash
105+
run: |
106+
curl -o install.sh -sSL https://raw.githubusercontent.com/replicatedhq/replicated/master/install.sh
107+
sudo bash ./install.sh
108+
- name: Output CMX environment variables
109+
shell: bash
110+
if: ${{ inputs.cmx-replicated-api-token != '' && inputs.replicatedvm-ssh-key != '' && inputs.replicatedvm-ssh-user != '' }}
111+
run: |
112+
echo "${{ inputs.replicatedvm-ssh-key }}" | base64 --decode > /tmp/replicatedvm-ssh-key
113+
chmod 400 /tmp/replicatedvm-ssh-key
114+
echo "CMX_REPLICATED_API_TOKEN=${{ inputs.cmx-replicated-api-token }}" >> $GITHUB_ENV
115+
echo "REPLICATEDVM_SSH_USER=${{ inputs.replicatedvm-ssh-user }}" >> $GITHUB_ENV
116+
echo "REPLICATEDVM_SSH_KEY=/tmp/replicatedvm-ssh-key" >> $GITHUB_ENV
94117
- name: E2E
95118
shell: bash
96119
run: |

‎.github/workflows/ci.yaml

+58-55
Original file line numberDiff line numberDiff line change
@@ -632,36 +632,36 @@ jobs:
632632
matrix:
633633
test:
634634
- TestPreflights
635-
- TestPreflightsNoexec
636-
- TestMaterialize
637-
- TestHostPreflightCustomSpec
638-
- TestHostPreflightInBuiltSpec
639-
- TestSingleNodeInstallation
640-
- TestSingleNodeInstallationAlmaLinux8
641-
- TestSingleNodeInstallationDebian11
642-
- TestSingleNodeInstallationDebian12
643-
- TestSingleNodeInstallationCentos9Stream
644-
- TestSingleNodeUpgradePreviousStable
645-
- TestInstallFromReplicatedApp
646-
- TestUpgradeFromReplicatedApp
647-
- TestUpgradeEC18FromReplicatedApp
648-
- TestResetAndReinstall
649-
# - TestOldVersionUpgrade
650-
- TestInstallSnapshotFromReplicatedApp
651-
- TestMultiNodeInstallation
652-
- TestMultiNodeHAInstallation
653-
- TestSingleNodeDisasterRecovery
654-
- TestSingleNodeLegacyDisasterRecovery
655-
- TestSingleNodeResumeDisasterRecovery
656-
- TestMultiNodeHADisasterRecovery
657-
- TestSingleNodeInstallationNoopUpgrade
658-
- TestCustomCIDR
659-
- TestLocalArtifactMirror
660-
- TestMultiNodeReset
661-
- TestCollectSupportBundle
662-
- TestUnsupportedOverrides
663-
- TestHostCollectSupportBundleInCluster
664-
- TestInstallWithConfigValues
635+
# - TestPreflightsNoexec
636+
# - TestMaterialize
637+
# - TestHostPreflightCustomSpec
638+
# - TestHostPreflightInBuiltSpec
639+
# - TestSingleNodeInstallation
640+
# - TestSingleNodeInstallationAlmaLinux8
641+
# - TestSingleNodeInstallationDebian11
642+
# - TestSingleNodeInstallationDebian12
643+
# - TestSingleNodeInstallationCentos9Stream
644+
# - TestSingleNodeUpgradePreviousStable
645+
# - TestInstallFromReplicatedApp
646+
# - TestUpgradeFromReplicatedApp
647+
# - TestUpgradeEC18FromReplicatedApp
648+
# - TestResetAndReinstall
649+
# # - TestOldVersionUpgrade
650+
# - TestInstallSnapshotFromReplicatedApp
651+
# - TestMultiNodeInstallation
652+
# - TestMultiNodeHAInstallation
653+
# - TestSingleNodeDisasterRecovery
654+
# - TestSingleNodeLegacyDisasterRecovery
655+
# - TestSingleNodeResumeDisasterRecovery
656+
# - TestMultiNodeHADisasterRecovery
657+
# - TestSingleNodeInstallationNoopUpgrade
658+
# - TestCustomCIDR
659+
# - TestLocalArtifactMirror
660+
# - TestMultiNodeReset
661+
# - TestCollectSupportBundle
662+
# - TestUnsupportedOverrides
663+
# - TestHostCollectSupportBundleInCluster
664+
# - TestInstallWithConfigValues
665665
steps:
666666
- name: Checkout
667667
uses: actions/checkout@v4
@@ -725,32 +725,32 @@ jobs:
725725
fail-fast: false
726726
matrix:
727727
test:
728-
- TestVersion
729-
- TestCommandsRequireSudo
730-
- TestResetAndReinstallAirgap
731-
- TestSingleNodeAirgapUpgrade
732-
- TestSingleNodeAirgapUpgradeConfigValues
733-
- TestSingleNodeAirgapUpgradeCustomCIDR
734-
- TestSingleNodeDisasterRecoveryWithProxy
735-
- TestProxiedEnvironment
736-
- TestProxiedCustomCIDR
728+
# - TestVersion
729+
# - TestCommandsRequireSudo
730+
# - TestResetAndReinstallAirgap
731+
# - TestSingleNodeAirgapUpgrade
732+
# - TestSingleNodeAirgapUpgradeConfigValues
733+
# - TestSingleNodeAirgapUpgradeCustomCIDR
734+
# - TestSingleNodeDisasterRecoveryWithProxy
735+
# - TestProxiedEnvironment
736+
# - TestProxiedCustomCIDR
737737
- TestInstallWithPrivateCAs
738-
- TestInstallWithMITMProxy
739-
include:
740-
- test: TestMultiNodeAirgapUpgrade
741-
runner: embedded-cluster-2
742-
- test: TestMultiNodeAirgapUpgradeSameK0s
743-
runner: embedded-cluster-2
744-
- test: TestMultiNodeAirgapUpgradePreviousStable
745-
runner: embedded-cluster-2
746-
- test: TestAirgapUpgradeFromEC18
747-
runner: embedded-cluster-2
748-
- test: TestSingleNodeAirgapDisasterRecovery
749-
runner: embedded-cluster-2
750-
- test: TestMultiNodeAirgapHAInstallation
751-
runner: embedded-cluster-2
752-
- test: TestMultiNodeAirgapHADisasterRecovery
753-
runner: embedded-cluster-2
738+
# - TestInstallWithMITMProxy
739+
include: []
740+
# - test: TestMultiNodeAirgapUpgrade
741+
# runner: embedded-cluster-2
742+
# - test: TestMultiNodeAirgapUpgradeSameK0s
743+
# runner: embedded-cluster-2
744+
# - test: TestMultiNodeAirgapUpgradePreviousStable
745+
# runner: embedded-cluster-2
746+
# - test: TestAirgapUpgradeFromEC18
747+
# runner: embedded-cluster-2
748+
# - test: TestSingleNodeAirgapDisasterRecovery
749+
# runner: embedded-cluster-2
750+
# - test: TestMultiNodeAirgapHAInstallation
751+
# runner: embedded-cluster-2
752+
# - test: TestMultiNodeAirgapHADisasterRecovery
753+
# runner: embedded-cluster-2
754754
steps:
755755
- name: Checkout
756756
uses: actions/checkout@v4
@@ -776,6 +776,9 @@ jobs:
776776
k0s-version-previous: ${{ needs.build-previous-k0s.outputs.k0s_version }}
777777
k0s-version-previous-stable: ${{ needs.find-previous-stable.outputs.k0s_version }}
778778
version-specifier: ${{ needs.export-version-specifier.outputs.version_specifier }}
779+
cmx-replicated-api-token: ${{ secrets.CMX_REPLICATED_API_TOKEN }}
780+
replicatedvm-ssh-user: ${{ secrets.REPLICATEDVM_SSH_USER }}
781+
replicatedvm-ssh-key: ${{ secrets.REPLICATEDVM_SSH_KEY }}
779782

780783
e2e-main:
781784
name: E2E (on merge)

‎Makefile

+3
Original file line numberDiff line numberDiff line change
@@ -284,9 +284,12 @@ vet:
284284

285285
.PHONY: e2e-tests
286286
e2e-tests: embedded-release
287+
e2e-tests: export SHORT_SHA = dev-$(shell git rev-parse --short HEAD)
288+
e2e-tests:
287289
go test -tags $(GO_BUILD_TAGS) -timeout 60m -ldflags="$(LD_FLAGS)" -parallel 1 -failfast -v ./e2e
288290

289291
.PHONY: e2e-test
292+
e2e-test: export SHORT_SHA = dev-$(shell git rev-parse --short HEAD)
290293
e2e-test:
291294
go test -tags $(GO_BUILD_TAGS) -timeout 60m -ldflags="$(LD_FLAGS)" -v ./e2e -run ^$(TEST_NAME)$$
292295

‎e2e/README.md

+55-43
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,58 @@
1-
## E2E tests
1+
## E2E Tests
2+
3+
### Buildting the release for E2E tests
4+
5+
```bash
6+
./scripts/build-and-release.sh ARCH=amd64 APP_VERSION="appver-dev-local-$USER"
7+
```
8+
9+
### Running individual CMX tests locally
10+
11+
You can run a single test with:
12+
13+
```bash
14+
export SHORT_SHA="dev-local-$USER"
15+
export LICENSE_ID="$EC_SMOKE_TEST_LICENSE_ID"
16+
make e2e-test TEST_NAME=TestSomething
17+
```
18+
19+
TestSomething is the name of the test function you want to run.
20+
21+
### Running individual Docker tests locally
22+
23+
You can run a single test with:
24+
25+
```bash
26+
export SHORT_SHA="dev-local-$USER"
27+
export LICENSE_ID="$EC_SMOKE_TEST_LICENSE_ID"
28+
make e2e-test TEST_NAME=TestSomething
29+
```
30+
31+
TestSomething is the name of the test function you want to run.
32+
33+
### Adding more tests
34+
35+
Tests are added as Go tests in the e2e/ directory.
36+
Tests must be added to the ci.yaml and release-prod.yaml GitHub workflows to be run in CI.
37+
38+
### Kots test application
39+
40+
During end to end tests we embed a license for a smoke test kots app.
41+
This app can be found under the 'Replicated, Inc.' team on staging:
42+
43+
https://vendor.staging.replicated.com/apps/embedded-cluster-smoke-test-staging-app
44+
45+
New releases are created using the corresponding YAML files in the e2e/kots-release-* directories.
46+
47+
### Playwright
48+
49+
We use [Playwright](https://playwright.dev/) to run end to end tests on the UI.
50+
The tests live in the `playwright` directory.
51+
52+
For more details on how to write tests with Playwright, refer to the [Playwright documentation](https://playwright.dev/docs/writing-tests).
53+
54+
55+
## DEPRECATED: E2E tests
256

357
Integration tests depends on LXD. The following procedure shows
458
how to get everything installed on Ubuntu 22.04.
@@ -31,45 +85,3 @@ Scripts inside the `scripts` directory are copied to all nodes.
3185
If you have a new test you want to add then start by creating a
3286
shell script to execute it and save it under the `scripts` dir.
3387
You can then call the script from your Go code.
34-
35-
### Running all the tests
36-
37-
You can run the tests from within this directory:
38-
39-
```
40-
$ make e2e-tests
41-
```
42-
43-
### Running individual tests
44-
45-
You can run a single test with:
46-
47-
```
48-
$ make e2e-test TEST_NAME=TestSomething
49-
```
50-
51-
TestSomething is the name of the test function you want to run.
52-
53-
### Adding more tests
54-
55-
To add more tests you just need to create one inside this directory
56-
and then add it to the `.github/workflows/e2e.yaml` file.
57-
58-
59-
### Kots test application
60-
61-
During end to end tests we embed a license for a smoke test kots app,
62-
this app can be found under the 'Replicated, Inc.' team on staging:
63-
64-
https://vendor.staging.replicated.com/apps/embedded-cluster-smoke-test-staging-app
65-
66-
Make sure to update the application yaml files under kots-release-onmerge
67-
and kots-release-onpr directories if you create a new release of the remote
68-
application.
69-
70-
### Playwright
71-
72-
We use [Playwright](https://playwright.dev/) to run end to end tests on the UI.
73-
The tests live in the `playwright` directory.
74-
75-
For more details on how to write tests with Playwright, refer to the [Playwright documentation](https://playwright.dev/docs/writing-tests).

0 commit comments

Comments
 (0)