Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Speed up Go tests by fixing the slow fleetctl tests #27927

Open
21 tasks
getvictor opened this issue Apr 6, 2025 · 0 comments
Open
21 tasks

Speed up Go tests by fixing the slow fleetctl tests #27927

getvictor opened this issue Apr 6, 2025 · 0 comments
Labels
~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. story A user story defining an entire feature

Comments

@getvictor
Copy link
Member

Goal

User story
As a contributor,
I want to I want to see the fleetctl CI test results as soon as possible
so that I can so that I can merge my changes and increase dev velocity.

Testing the fleetctl package is currently the long pole in our CI tests. The package needs to be refactored so that tests can run in parallel. The issue is that fleetctl is a main package -- it needs to be split into main, fleetctl, and integrationtest packages.

image.png

Key result

Original requests

Context

  • Product Designer: _________________________

Changes

Product

  • UI changes: TODO
  • CLI (fleetctl) usage changes: TODO
  • YAML changes: TODO
  • REST API changes: TODO
  • Fleet's agent (fleetd) changes: TODO
  • GitOps mode changes: TODO
  • Activity changes: TODO
  • Permissions changes: TODO
  • Changes to paid features or tiers: TODO
  • Transparency changes: TODO
  • First draft of test plan added
  • Other reference documentation changes: TODO
  • Once shipped, requester has been notified
  • Once shipped, dogfooding issue has been filed

Engineering

  • Test plan is finalized
  • Contributor API changes: TODO
  • Feature guide changes: TODO
  • Database schema migrations: TODO
  • Load testing: TODO

ℹ️  Please read this issue carefully and understand it. Pay special attention to UI wireframes, especially "dev notes".

QA

Risk assessment

  • Requires load testing: TODO
  • Risk level: Low / High TODO
  • Risk description: TODO

Test plan

Make sure to go through the list and consider all events that might be related to this story, so we catch edge cases earlier.

  1. Step 1
  2. Step 2
  3. Step 3

Testing notes

Confirmation

  1. Engineer: Added comment to user story confirming successful completion of test plan.
  2. QA: Added comment to user story confirming successful completion of test plan.
@getvictor getvictor added story A user story defining an entire feature ~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. labels Apr 6, 2025
@getvictor getvictor mentioned this issue Apr 6, 2025
Merged
getvictor added a commit that referenced this issue Apr 7, 2025
@getvictor
Improvements to test-go CI job (#27916)
7558d5a 
This PR makes several improvements to test-go CI job
- remove ZSH dependency from all test suites except for a new `scripts`
suite
- add a `fast` suite that does not have out-of-process dependencies,
which completes in ~1.5 minutes
- contributors can add their fast tests to this suite so they can see
the results in CI faster
- Rename `core` to `main` test suite to be consistent with Makefile. It
is the default bucket for tests.
- Cleaned up Makefile so that it is more straightforward to add new test
suites or move Go packages between suites
- Do not stop the test suites on a fail.
- We do not want to be blocked by a test fail that another product team
introduced
- Sometimes, we want to see all test failures so we can fix them all at
once.
- Removed `test-schema` and `mock` prerequisites for `test-go` since
they are not needed and just take up time.
- But also added `test-schema` run to one of the test suites just in
case.

Unfortunately, `fleetctl` is still the bottleneck and needs to be
refactored. New issue filed:
#27927

## Before
<img width="248" alt="image"
src="https://github.com/user-attachments/assets/110ffc1d-f090-4d3e-be77-0419b9577d20"
/>

## After
<img width="320" alt="image"
src="https://github.com/user-attachments/assets/8d01ea11-408f-4eb6-81d8-9c25410b8830"
/>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
~engineering-initiated Engineering-initiated story, such as a bug, refactor, or contributor experience improvement. story A user story defining an entire feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@getvictor