From 76339c05fe4bc15f31c3fd2e8e7af7404e4355e2 Mon Sep 17 00:00:00 2001 From: Ben Alkov Date: Fri, 22 Mar 2024 13:54:30 -0400 Subject: [PATCH] DRAFT: walkthrough To be squashed after review --- docs/writing_integration_tests.md | 46 +++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/docs/writing_integration_tests.md b/docs/writing_integration_tests.md index 6450808ff..c63925b40 100644 --- a/docs/writing_integration_tests.md +++ b/docs/writing_integration_tests.md @@ -35,6 +35,52 @@ the repo in the [cachito-testing][] org). At this point, you should be able to test locally. +### A walkthrough + +1. Fork a test repo under [cachito-testing][] which uses the package manager you + need to test, and/or is otherwise related +1. Clone the new fork to your local machine +1. Create a new branch in the repo, named to reflect the purpose of your test(s) +1. The goal here is to create what looks like a real + Go/Javascript/Python/whichever project, but simplified enough to *only* include + the required files which cachi2 needs to find and resolve dependencies +1. Commit the test code, and push it to your fork in GitHub. Take note of the + commit hash. + +#### Testing your tests + +1. Add your pytest scenarios to the appropriate integration test source file + under 'cachi2/tests/integration'. You'll need to put the following in + `utils.TestParameters` for your test case + + - `repo="https://github.com/cachito-testing/cachito-pip-without-deps.git"` # this will be the name of the test repo you forked + - `ref="3fe2fc3cb8ffa36317cacbd9d356e35e17af2824"` # this will be the commit hash noted previously + - `packages=({"path": ".", "type": "pip"},)` # for pip, for example + - `check_vendor_checksums=False` # or `True`, depending on the scenario + +1. In your local **cachi2** repo, make sure you have 'pytest' and 'jsonschema' + pip-installed in the cachi2 venv, so that you can run pytest without tox. Without + `tox`, it's **much** easier to run pytest with *only* your selected test + scanarios, and get useful, accessible logs +1. Bonus: pip-install 'pytest-html' in the venv, and add (e.g.) + `--html=path_to_reports/pytest-report.html` to your `pytest` command line + for a very nicely formatted HTML report +1. To summarize + + - You're in the cachi2 repo directory + - Your test scenario's source is in 'tests/integration/test_foo.py' + - Your test scenario is called `test_foo_package`, and its pytest 'id' is + `foo_incorrect_checksum` + +1. Now run + + ```bash + CACHI2_IMAGE=localhost/cachi2:latest pytest -rA -vvvv --confcutdir=tests/integration --log-cli-level=DEBUG tests/integration/test_foo.py::test_foo_package[foo_incorrect_checksum] + ``` + + which will run *only* 'tests/integration/test_foo.py::test_foo_package' with + it's "foo_incorrect_checksum" ID'd parameter set with a pre-built cachi2 container + ## Running the test suite It's a good idea to run the whole cachi2 integration test suite, just to make