Skip to content

Commit 697e5b9

Browse files
emosbaughemosbaugh
committed
run e2e tests on cmx
1 parent 0c0ba24 commit 697e5b9

38 files changed

+452
-232
lines changed

‎.github/actions/e2e/action.yml

+23
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,15 @@ inputs:
4343
version-specifier:
4444
description: 'the git sha or tag used to generate application version strings'
4545
required: true
46+
cmx-replicated-api-token:
47+
description: 'the replicated api token to use for cmx e2e tests'
48+
required: false
49+
replicatedvm-ssh-user:
50+
description: 'the user to use for ssh to the replicated vm'
51+
required: false
52+
replicatedvm-ssh-key:
53+
description: 'the ssh key to use for ssh to the replicated vm'
54+
required: false
4655
upgrade-target-ec-version:
4756
description: 'the embedded cluster version to expect after upgrades complete'
4857
required: false # this is only set by post-release testing
@@ -91,6 +100,20 @@ runs:
91100
with:
92101
go-version-file: go.mod
93102
cache-dependency-path: "**/*.sum"
103+
- name: Install Replicated CLI
104+
shell: bash
105+
run: |
106+
curl -o install.sh -sSL https://raw.githubusercontent.com/replicatedhq/replicated/master/install.sh
107+
sudo bash ./install.sh
108+
- name: Output CMX environment variables
109+
shell: bash
110+
if: ${{ inputs.cmx-replicated-api-token != '' && inputs.replicatedvm-ssh-key != '' && inputs.replicatedvm-ssh-user != '' }}
111+
run: |
112+
echo "${{ inputs.replicatedvm-ssh-key }}" | base64 --decode > /tmp/replicatedvm-ssh-key
113+
chmod 400 /tmp/replicatedvm-ssh-key
114+
echo "CMX_REPLICATED_API_TOKEN=${{ inputs.cmx-replicated-api-token }}" >> $GITHUB_ENV
115+
echo "REPLICATEDVM_SSH_USER=${{ inputs.replicatedvm-ssh-user }}" >> $GITHUB_ENV
116+
echo "REPLICATEDVM_SSH_KEY=/tmp/replicatedvm-ssh-key" >> $GITHUB_ENV
94117
- name: E2E
95118
shell: bash
96119
run: |

‎.github/workflows/ci.yaml

+58-55
Original file line numberDiff line numberDiff line change
@@ -622,36 +622,36 @@ jobs:
622622
matrix:
623623
test:
624624
- TestPreflights
625-
- TestPreflightsNoexec
626-
- TestMaterialize
627-
- TestHostPreflightCustomSpec
628-
- TestHostPreflightInBuiltSpec
629-
- TestSingleNodeInstallation
630-
- TestSingleNodeInstallationAlmaLinux8
631-
- TestSingleNodeInstallationDebian11
632-
- TestSingleNodeInstallationDebian12
633-
- TestSingleNodeInstallationCentos9Stream
634-
- TestSingleNodeUpgradePreviousStable
635-
- TestInstallFromReplicatedApp
636-
- TestUpgradeFromReplicatedApp
637-
- TestUpgradeEC18FromReplicatedApp
638-
- TestResetAndReinstall
639-
# - TestOldVersionUpgrade
640-
- TestInstallSnapshotFromReplicatedApp
641-
- TestMultiNodeInstallation
642-
- TestMultiNodeHAInstallation
643-
- TestSingleNodeDisasterRecovery
644-
- TestSingleNodeLegacyDisasterRecovery
645-
- TestSingleNodeResumeDisasterRecovery
646-
- TestMultiNodeHADisasterRecovery
647-
- TestSingleNodeInstallationNoopUpgrade
648-
- TestCustomCIDR
649-
- TestLocalArtifactMirror
650-
- TestMultiNodeReset
651-
- TestCollectSupportBundle
652-
- TestUnsupportedOverrides
653-
- TestHostCollectSupportBundleInCluster
654-
- TestInstallWithConfigValues
625+
# - TestPreflightsNoexec
626+
# - TestMaterialize
627+
# - TestHostPreflightCustomSpec
628+
# - TestHostPreflightInBuiltSpec
629+
# - TestSingleNodeInstallation
630+
# - TestSingleNodeInstallationAlmaLinux8
631+
# - TestSingleNodeInstallationDebian11
632+
# - TestSingleNodeInstallationDebian12
633+
# - TestSingleNodeInstallationCentos9Stream
634+
# - TestSingleNodeUpgradePreviousStable
635+
# - TestInstallFromReplicatedApp
636+
# - TestUpgradeFromReplicatedApp
637+
# - TestUpgradeEC18FromReplicatedApp
638+
# - TestResetAndReinstall
639+
# # - TestOldVersionUpgrade
640+
# - TestInstallSnapshotFromReplicatedApp
641+
# - TestMultiNodeInstallation
642+
# - TestMultiNodeHAInstallation
643+
# - TestSingleNodeDisasterRecovery
644+
# - TestSingleNodeLegacyDisasterRecovery
645+
# - TestSingleNodeResumeDisasterRecovery
646+
# - TestMultiNodeHADisasterRecovery
647+
# - TestSingleNodeInstallationNoopUpgrade
648+
# - TestCustomCIDR
649+
# - TestLocalArtifactMirror
650+
# - TestMultiNodeReset
651+
# - TestCollectSupportBundle
652+
# - TestUnsupportedOverrides
653+
# - TestHostCollectSupportBundleInCluster
654+
# - TestInstallWithConfigValues
655655
steps:
656656
- name: Checkout
657657
uses: actions/checkout@v4
@@ -715,32 +715,32 @@ jobs:
715715
fail-fast: false
716716
matrix:
717717
test:
718-
- TestVersion
719-
- TestCommandsRequireSudo
720-
- TestResetAndReinstallAirgap
721-
- TestSingleNodeAirgapUpgrade
722-
- TestSingleNodeAirgapUpgradeConfigValues
723-
- TestSingleNodeAirgapUpgradeCustomCIDR
724-
- TestSingleNodeDisasterRecoveryWithProxy
725-
- TestProxiedEnvironment
726-
- TestProxiedCustomCIDR
718+
# - TestVersion
719+
# - TestCommandsRequireSudo
720+
# - TestResetAndReinstallAirgap
721+
# - TestSingleNodeAirgapUpgrade
722+
# - TestSingleNodeAirgapUpgradeConfigValues
723+
# - TestSingleNodeAirgapUpgradeCustomCIDR
724+
# - TestSingleNodeDisasterRecoveryWithProxy
725+
# - TestProxiedEnvironment
726+
# - TestProxiedCustomCIDR
727727
- TestInstallWithPrivateCAs
728-
- TestInstallWithMITMProxy
729-
include:
730-
- test: TestMultiNodeAirgapUpgrade
731-
runner: embedded-cluster-2
732-
- test: TestMultiNodeAirgapUpgradeSameK0s
733-
runner: embedded-cluster-2
734-
- test: TestMultiNodeAirgapUpgradePreviousStable
735-
runner: embedded-cluster-2
736-
- test: TestAirgapUpgradeFromEC18
737-
runner: embedded-cluster-2
738-
- test: TestSingleNodeAirgapDisasterRecovery
739-
runner: embedded-cluster-2
740-
- test: TestMultiNodeAirgapHAInstallation
741-
runner: embedded-cluster-2
742-
- test: TestMultiNodeAirgapHADisasterRecovery
743-
runner: embedded-cluster-2
728+
# - TestInstallWithMITMProxy
729+
include: []
730+
# - test: TestMultiNodeAirgapUpgrade
731+
# runner: embedded-cluster-2
732+
# - test: TestMultiNodeAirgapUpgradeSameK0s
733+
# runner: embedded-cluster-2
734+
# - test: TestMultiNodeAirgapUpgradePreviousStable
735+
# runner: embedded-cluster-2
736+
# - test: TestAirgapUpgradeFromEC18
737+
# runner: embedded-cluster-2
738+
# - test: TestSingleNodeAirgapDisasterRecovery
739+
# runner: embedded-cluster-2
740+
# - test: TestMultiNodeAirgapHAInstallation
741+
# runner: embedded-cluster-2
742+
# - test: TestMultiNodeAirgapHADisasterRecovery
743+
# runner: embedded-cluster-2
744744
steps:
745745
- name: Checkout
746746
uses: actions/checkout@v4
@@ -766,6 +766,9 @@ jobs:
766766
k0s-version-previous: ${{ needs.build-previous-k0s.outputs.k0s_version }}
767767
k0s-version-previous-stable: ${{ needs.find-previous-stable.outputs.k0s_version }}
768768
version-specifier: ${{ needs.export-version-specifier.outputs.version_specifier }}
769+
cmx-replicated-api-token: ${{ secrets.CMX_REPLICATED_API_TOKEN }}
770+
replicatedvm-ssh-user: ${{ secrets.REPLICATEDVM_SSH_USER }}
771+
replicatedvm-ssh-key: ${{ secrets.REPLICATEDVM_SSH_KEY }}
769772

770773
e2e-main:
771774
name: E2E (on merge)

‎Makefile

+3-1
Original file line numberDiff line numberDiff line change
@@ -276,10 +276,12 @@ vet:
276276
go vet -tags exclude_graphdriver_btrfs ./...
277277

278278
.PHONY: e2e-tests
279-
e2e-tests: embedded-release
279+
e2e-tests: export SHORT_SHA = dev-$(shell git rev-parse --short HEAD)
280+
e2e-tests:
280281
go test -tags exclude_graphdriver_btrfs -timeout 60m -ldflags="$(LD_FLAGS)" -parallel 1 -failfast -v ./e2e
281282

282283
.PHONY: e2e-test
284+
e2e-test: export SHORT_SHA = dev-$(shell git rev-parse --short HEAD)
283285
e2e-test:
284286
go test -tags exclude_graphdriver_btrfs -timeout 60m -ldflags="$(LD_FLAGS)" -v ./e2e -run ^$(TEST_NAME)$$
285287

‎e2e/README.md

+55-43
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,58 @@
1-
## E2E tests
1+
## E2E Tests
2+
3+
### Buildting the release for E2E tests
4+
5+
```bash
6+
./scripts/build-and-release.sh ARCH=amd64 APP_VERSION="appver-dev-local-$USER"
7+
```
8+
9+
### Running individual CMX tests locally
10+
11+
You can run a single test with:
12+
13+
```bash
14+
export SHORT_SHA="dev-local-$USER"
15+
export LICENSE_ID="$EC_SMOKE_TEST_LICENSE_ID"
16+
make e2e-test TEST_NAME=TestSomething
17+
```
18+
19+
TestSomething is the name of the test function you want to run.
20+
21+
### Running individual Docker tests locally
22+
23+
You can run a single test with:
24+
25+
```bash
26+
export SHORT_SHA="dev-local-$USER"
27+
export LICENSE_ID="$EC_SMOKE_TEST_LICENSE_ID"
28+
make e2e-test TEST_NAME=TestSomething
29+
```
30+
31+
TestSomething is the name of the test function you want to run.
32+
33+
### Adding more tests
34+
35+
Tests are added as Go tests in the e2e/ directory.
36+
Tests must be added to the ci.yaml and release-prod.yaml GitHub workflows to be run in CI.
37+
38+
### Kots test application
39+
40+
During end to end tests we embed a license for a smoke test kots app.
41+
This app can be found under the 'Replicated, Inc.' team on staging:
42+
43+
https://vendor.staging.replicated.com/apps/embedded-cluster-smoke-test-staging-app
44+
45+
New releases are created using the corresponding YAML files in the e2e/kots-release-* directories.
46+
47+
### Playwright
48+
49+
We use [Playwright](https://playwright.dev/) to run end to end tests on the UI.
50+
The tests live in the `playwright` directory.
51+
52+
For more details on how to write tests with Playwright, refer to the [Playwright documentation](https://playwright.dev/docs/writing-tests).
53+
54+
55+
## DEPRECATED: E2E tests
256

357
Integration tests depends on LXD. The following procedure shows
458
how to get everything installed on Ubuntu 22.04.
@@ -31,45 +85,3 @@ Scripts inside the `scripts` directory are copied to all nodes.
3185
If you have a new test you want to add then start by creating a
3286
shell script to execute it and save it under the `scripts` dir.
3387
You can then call the script from your Go code.
34-
35-
### Running all the tests
36-
37-
You can run the tests from within this directory:
38-
39-
```
40-
$ make e2e-tests
41-
```
42-
43-
### Running individual tests
44-
45-
You can run a single test with:
46-
47-
```
48-
$ make e2e-test TEST_NAME=TestSomething
49-
```
50-
51-
TestSomething is the name of the test function you want to run.
52-
53-
### Adding more tests
54-
55-
To add more tests you just need to create one inside this directory
56-
and then add it to the `.github/workflows/e2e.yaml` file.
57-
58-
59-
### Kots test application
60-
61-
During end to end tests we embed a license for a smoke test kots app,
62-
this app can be found under the 'Replicated, Inc.' team on staging:
63-
64-
https://vendor.staging.replicated.com/apps/embedded-cluster-smoke-test-staging-app
65-
66-
Make sure to update the application yaml files under kots-release-onmerge
67-
and kots-release-onpr directories if you create a new release of the remote
68-
application.
69-
70-
### Playwright
71-
72-
We use [Playwright](https://playwright.dev/) to run end to end tests on the UI.
73-
The tests live in the `playwright` directory.
74-
75-
For more details on how to write tests with Playwright, refer to the [Playwright documentation](https://playwright.dev/docs/writing-tests).

0 commit comments

Comments
 (0)