A Buildkite plugin that parses junit.xml artifacts (generated across any number of parallel steps) and creates a build annotation listing the individual tests that failed.
The following pipeline will run test.sh jobs in parallel, and then process all the resulting JUnit XML files to create a summary build annotation.
steps:
- command: test.sh
parallelism: 50
artifact_paths: tmp/junit-*.xml
- wait: ~
continue_on_failure: true
- plugins:
- junit-annotate#v2.5.0:
artifacts: tmp/junit-*.xmlFor scenarios where you have different artifact paths that you want to add as annotation then call the plugin multiple times in the pipeline with different contexts as shown below:
steps:
- command: test.sh
parallelism: 50
artifact_paths: tmp/junit-*.xml
- command: anothertest.sh
artifact_paths: artifacts/junit-*.xml
- wait: ~
continue_on_failure: true
- plugins:
- junit-annotate#v2.5.0:
artifacts: tmp/junit-*.xml
- plugins:
- junit-annotate#v2.5.0:
artifacts: artifacts/junit-*.xml
context: junit-artifactsThe artifact glob path to find the JUnit XML files.
Example: tmp/junit-*.xml
Forces the creation of the annotation even when no failures or errors are found
Default: junit
The buildkite annotation context to use. Useful to differentiate multiple runs of this plugin in a single pipeline.
Default: -(.*).xml
The regular expression (with capture group) that matches the job UUID in the junit file names. This is used to create the job links in the annotation.
To use this, configure your test reporter to embed the $BUILDKITE_JOB_ID environment variable into your junit file names. For example "junit-buildkite-job-$BUILDKITE_JOB_ID.xml".
This setting controls the format of your failed test in the main annotation summary.
There are two options for this:
classname(the default)- displays:
MyClass::UnderTest text of the failed expectation in path.to.my_class.under_test
- displays:
file- displays:
MyClass::UnderTest text of the failed expectation in path/to/my_class/under_test.file_ext
- displays:
Default: false
If this setting is true and any errors are found in the JUnit XML files during parsing, the annotation step will exit with a non-zero value, which should cause the build to fail.
Default: 2
Exit code of the plugin if the call to buildkite-agent artifact download fails.
Minimum amount of run tests that need to be analyzed or a failure will be reported. It is useful to ensure that tests are actually run and report files to analyze do contain information.
Default: false
Will add a list of skipped tests at the end of the annotation. Note that even if there are skipped tests, the annotation may not be added unless other options or results of the processing forces it to.
Default: 0
Include the specified number of slowest tests in the annotation. The annotation will always be shown.
The docker image to use for running the analysis code. Must be a valid image reference that can run the corresponding ruby code and the agent running the step must be able to pull it if not already present.
Default: ruby:3.1-alpine@sha256:a39e26d0598837f08c75a42c8b0886d9ed5cc862c4b535662922ee1d05272fca
To run testing, shellchecks and plugin linting use use bk run with the Buildkite CLI.
bk runOr if you want to run just the plugin tests, you can use the docker Plugin Tester:
docker run --rm -ti -v "${PWD}":/plugin buildkite/plugin-tester:latestTo test the Ruby code with rake in docker:
docker-compose run --rm rubyTo test your plugin in your builds prior to opening a pull request, you can refer to your fork and SHA from a branch in your pipeline.yml.
steps:
- label: Annotate
plugins:
- YourGithubHandle/junit-annotate#v2.5.0:
...
MIT (see LICENSE)