Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "smoke test" integration tests that do not require upstream artifacts #15695

Open
robbavey opened this issue Dec 15, 2023 · 1 comment
Open

Comments

@robbavey
Copy link
Member

After a version bump, we tend to see CI failures for integration tests for a significant amount of time after, as we wait for upstream artifacts to be created for the newly bumped version of the stack. This leaves us in a state where we have unreliable CI for potentially days until all of the artifacts are created.

During this period, we may make changes that effectively go untested, and get approved for expediency, or because the changes are deemed "low risk". However, these "low risk" changes, can still introduce unexpected bugs such as #15687.

We should consider adding a set of integration tests that do not require upstream artifacts, and will run regardless of whether the upstream artifacts are available. There are a number of our integration tests that fit this bill - such as the cli smoke tests for logstash-plugin. We could also take advantage of the serverless offering to add integration tests that require an Elasticsearch instance.

@robbavey
Copy link
Member Author

cc @dliappis

@robbavey robbavey changed the title Add "smoke test" integration tests that do not require on upstream artifacts Add "smoke test" integration tests that do not require upstream artifacts Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant