update tests docs, align with code (#999)

docs/testing.rst

@@ -3,31 +3,44 @@
 Testing
 =======
 
+There are a number of test suites that perform mostly functional testing.
+These tests ensure that pycsw operates correctly and is compliant with the 
+various supported standards. There is also a growing set of unit tests. 
+These focus on smaller scope testing, in order to verify that individual 
+bits of code are working as expected.
+
+Tests can be run locally as part of the development cycle. They are also
+run on pycsw's `GitHub Actions`_ continuous integration setup against all pushes and
+pull requests to the code repository.
+
 pycsw uses `pytest`_ for managing its automated tests.
 
+Install pytest from the development requirements.
+
+.. code:: bash
+
+   pip3 install -r requirements-dev.txt
+
 OGC API - Records
 -----------------
 
-Tests for OGC API - Records are located in ``tests/unittests/test_oarec.py``. They
+Tests for OGC API - Records are located in ``tests/functionaltests/suites/oarec``. They
 can be run as follows:
 
 .. code:: bash
 
-   pytest tests/unittests/test_oarec.py
+   pytest tests/functionaltests/suites/oarec
 
 
 OGC CSW
 -------
 
-There are a number of test suites that perform mostly functional testing.
-These tests ensure that pycsw is compliant with the various supported standards.
-There is also a growing set of unit tests. These focus on smaller scope 
-testing, in order to verify that individual bits of code are working as
-expected.
+Tests for OGC CSW are located in ``tests/functionaltests/suites/csw30``. They
+can be run as follows:
 
-Tests can be run locally as part of the development cycle. They are also
-run on pycsw's `GitHub Actions`_ continuous integration setup against all pushes and
-pull requests to the code repository.
+.. code:: bash
+
+   pytest tests/functionaltests/suites/csw30
 
 
 .. _ogc-cite:
@@ -39,6 +52,13 @@ In addition to pycsw's own tests, all public releases are also tested via the
 OGC `Compliance & Interoperability Testing & Evaluation Initiative`_ (CITE).
 The pycsw `wiki`_ documents CITE testing procedures and status.
 
+Tests for OGC CITE are located in ``tests/functionaltests/suites/cite``. They
+can be run as follows:
+
+.. code:: bash
+
+   pytest tests/functionaltests/suites/cite
+
 
 Functional test suites
 ----------------------
@@ -126,36 +146,36 @@ them in isolation:
 .. code:: bash
 
    # running only the unit tests (not the functional ones)
-   py.test -m unit
+   pytest -m unit
 
 
 
 Running tests
 -------------
 
-Since pycsw uses `pytest`_, tests are run with the ``py.test`` runner. A basic
+Since pycsw uses `pytest`_, tests are run with the ``pytest`` runner. A basic
 test run can be made with:
 
 .. code:: bash
 
-   py.test
+   pytest
 
 This command will run all tests and report on the number of successes, failures
-and also the time it took to run them. The `py.test` command accepts several
+and also the time it took to run them. The `pytest` command accepts several
 additional parameters that can be used in order to customize the execution of
 tests. Look into `pytest's invocation documentation`_ for a more complete
 description. You can also get a description of the available parameters by
 running:
 
 .. code:: bash
 
-   py.test --help
+   pytest --help
 
 
 Running specific suites and test cases
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-py.test allows tagging tests with markers. These can be used to selectively run
+pytest allows tagging tests with markers. These can be used to selectively run
 some tests. pycsw uses two markers:
 
 * ``unit`` - run only input tests
@@ -165,17 +185,17 @@ Markers can be specified by using the ``-m <marker_name>`` flag.
 
 .. code:: bash
 
-   py.test -m functional  # run only functional tests
+   pytest -m functional  # run only functional tests
 
 You can also use the ``-k <name_expression>`` flag to select which tests to run. Since each
 test's name includes the suite name, http method and an identifier for the
 test, it is easy to run only certain tests.
 
 .. code:: bash
 
-   py.test -k "apiso and GetRecords"  # run only tests from the apiso suite that have GetRecords in their name
-   py.test -k "post and GetRecords"  # run only tests that use HTTP POST and GetRecords in their name
-   py.test -k "not harvesting"  # run all tests except those from the harvesting suite
+   pytest -k "apiso and GetRecords"  # run only tests from the apiso suite that have GetRecords in their name
+   pytest -k "post and GetRecords"  # run only tests that use HTTP POST and GetRecords in their name
+   pytest -k "not harvesting"  # run all tests except those from the harvesting suite
 
 
 The ``-m`` and ``-k`` flags can be combined.
@@ -189,7 +209,7 @@ immediately as soon as a test case fails.
 
 .. code:: bash
 
-   py.test --exitfirst
+   pytest --exitfirst
 
 
 Seeing more output
@@ -209,24 +229,24 @@ There are three main ways to get more output from running tests:
 
 .. code:: bash
 
-   py.test --verbose
-   py.test --pycsw-loglevel=debug
-   py.test -v --capture=no --pycsw-loglevel=debug
+   pytest --verbose
+   pytest --pycsw-loglevel=debug
+   pytest -v --capture=no --pycsw-loglevel=debug
 
 
 Comparing xml-based suite results with difflib instead of XML c14n
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Functional tests for XML-based suites compare results with their expected
 values by using `XML canonicalisation - XML c14n`_.
-Alternatively, you can call py.test with the ``--functional-prefer-diffs``
+Alternatively, you can call pytest with the ``--functional-prefer-diffs``
 flag. This will enable comparison based on Python's ``difflib``. Comparison
 is made on a line-by-line basis and in case of failure, a unified diff will
 be printed to standard output.
 
 .. code:: bash
 
-   py.test -m functional -k 'harvesting' --functional-prefer-diffs
+   pytest -m functional -k 'harvesting' --functional-prefer-diffs
 
 
 Saving test results for disk
@@ -238,7 +258,7 @@ after the test identifier it has when running with pytest.
 
 .. code:: bash
 
-   py.test -m functional -k 'not harvesting' --functional-save-results-directory=/tmp/pycsw-test-results
+   pytest -m functional -k 'not harvesting' --functional-save-results-directory=/tmp/pycsw-test-results
 
 
 
@@ -250,7 +270,7 @@ possible to get output in a variety of formats.
 
 .. code:: bash
 
-   py.test --cov pycsw
+   pytest --cov pycsw
 
 
 Specifying a timeout for tests
@@ -262,7 +282,7 @@ a float, so it is possibe to specify sub-second timeouts
 
 .. code:: bash
 
-   py.test --timeout=1.5
+   pytest --timeout=1.5
 
 
 Linting with flake8
@@ -273,7 +293,7 @@ guide
 
 .. code:: bash
 
-   py.test --flake8
+   pytest --flake8
 
 
 Testing multiple Python versions
@@ -282,8 +302,8 @@ Testing multiple Python versions
 For testing multiple Python versions and configurations simultaneously you can
 use `tox`_. pycsw includes a `tox.ini` file with a suitable configuration. It
 can be used to run tests against multiple Python versions and also multiple
-database backends. When running `tox` you can send arguments to the `py.test`
-runner by using the invocation `tox <tox arguments> -- <py.test arguments>`.
+database backends. When running `tox` you can send arguments to the `pytest`
+runner by using the invocation `tox <tox arguments> -- <pytest arguments>`.
 Examples:
 
 .. code:: bash