Los Angeles Meetup Thoughts: Moving from clojure.test to lazytest

[NoahTheDuke]

From the slack channel:

The Los Angeles meetup did a walkthrough of LazyTest tonight -- all very interesting stuff -- and the question came up:
If you have an existing codebase with a lot of existing tests, written with clojure.test, what is the story for a migration to LazyTest?
You can't do it strictly incrementally, because whatever clojure.test-compatible test runner you're using won't see the lazy tests, and the LazyTest runner won't see the existing tests.
Even a global find'n'replace on clojure.test to lazytest.experimental.interfaces.clojure-test wouldn't be sufficient (use-fixtures, for example).

[seancorfield]

My "solution" to this is to modify my test runners to look for both types of tests and run both clojure.test/run-tests and lazytest.repl/run-tests and merge the results -- once LazyTest stops using :test metadata (see #4).

My Polylith runner current looks for tests like this: https://github.com/seancorfield/polylith-external-test-runner/blob/main/bases/external-test-runner-cli/src/org/corfield/external_test_runner_cli/main.clj#L64-L70

Then it runs both and merges the result: https://github.com/seancorfield/polylith-external-test-runner/blob/main/bases/external-test-runner-cli/src/org/corfield/external_test_runner_cli/main.clj#L102-L106

And my Calva REPL snippets have been updated to run both on a per-namespace basis: https://github.com/seancorfield/vscode-calva-setup/blob/develop/calva/config.edn#L114-L143

[NoahTheDuke]

For results displaying/checking:

I did something similar when I moved Splint from clojure.test to Lazytest, as seen in this now-deleted file: https://github.com/NoahTheDuke/splint/blob/f851ef7933a74f54187d466a29482a1e4b4b577b/test/noahtheduke/splint/utils/test_runner.clj

A simple (merge-with (fnil + 0 0) ctr-result lt-result) works, but you can be more clever with it as you desire.

For actually transitioning the tests themselves, here's a couple different ideas:

1. Just move everything in one go. Here's an example from seesaw.
2. Move slowly, one namespace or test at a time, running both clojure.test and lazytest (either in a single call like my example and Sean's above, or in two separate clojure -M:... calls).
3. If you don't rely on use-fixtures or custom reporting functions, you can use the experimental clojure.test interface to switch a namespace by changing (:require [clojure.test :refer [deftest testing is are]]) to (:require [lazytest.experimental.interfaces.clojure-test :refer [deftest testing is are]]).

Fundamentally tho, the switch is pretty one-to-one: deftest becomes defdescribe, is becomes it + expect, and testing becomes describe. The only hitch is that expect must be wrapped in an it, so you have to either use expect-it to stay consistent, or bundle multiple assertions into a single test case.

This does highlight one small gap in Lazytest that I haven't found a good solution for. In clojure.test, you can write a simple deftest with a handful of assertions, but in Lazytest, you need to nest the assertions in an outer test case:

(deftest example-test
  (is (= 1 (one))
  (is (= 2 (two))
  (is (= 3 (three))))

; vs

(defdescribe example
  (it "works"
    (expect (= 1 (one))
    (expect (= 2 (two))
    (expect (= 3 (three)))))

It's a small point of friction, but I'm not pleased with it at the moment.

[seancorfield]

I would consider the necessary addition of something more descriptive as a "feature" not a "bug"! I like testing for that reason.

[NoahTheDuke]

I tend to agree. I think i've felt this pain mostly because I wrote a lot of test files for splint that were named for the splint rule, then the only test was named for the rule, and then the single test case doc string was... what? the rule name again?

for example: https://github.com/NoahTheDuke/splint/blob/main/test/noahtheduke/splint/rules/lint/let_if_test.clj

I think that's a particularly weird scenario tho, and most usage will be better for the requirement.