e2e: log errors #1596
Open
e2e: log errors #1596
+117
−86
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Example failing build |
nirs
commented
Oct 28, 2024
nirs
commented
Oct 28, 2024
Simplify the tests by moving the validation code into the util package.
util.ValidateRamenHubOperator() and util.ValidateDRClusterOperator()
returns now and error if ramen operator is not in valid state, and nil
if the operator is in good state. They also log information about the
pod.
Separate looking up the ramen operator pod, and checking for its status.
This way when the pod is not running we return an error with more
context (e.g pod name, pod phase).
Tests are using sub tests to have one test per cluster, which makes test
output more clear.
We use the virtual cluster names ("c1", "c2"). This can be improved
later by getting the cluster names from the dr policy, the clusterset,
or the environment file.
Example run - ramen not deployed:
% go test -run TestSuites/Validate --config ../e2e.yaml
2024-10-29T19:32:09.522+0200 INFO e2e/main_test.go:41 Global setting {"Timeout": 600}
2024-10-29T19:32:09.523+0200 INFO e2e/main_test.go:42 Global setting {"Retry Interval": 30}
2024-10-29T19:32:09.523+0200 INFO e2e/main_test.go:57 TestSuites
--- FAIL: TestSuites (0.06s)
--- FAIL: TestSuites/Validate (0.06s)
--- FAIL: TestSuites/Validate/hub (0.03s)
validation_suite_test.go:17: no pod with label selector "app=ramen-hub" and identifier "ramen-hub-operator" in namespace "ramen-system"
--- FAIL: TestSuites/Validate/c1 (0.02s)
validation_suite_test.go:23: no pod with label selector "app=ramen-dr-cluster" and identifier "ramen-dr-cluster-operator" in namespace "ramen-system"
--- FAIL: TestSuites/Validate/c2 (0.02s)
validation_suite_test.go:29: no pod with label selector "app=ramen-dr-cluster" and identifier "ramen-dr-cluster-operator" in namespace "ramen-system"
main_test.go:60: failed to validate the test suite
FAIL
exit status 1
FAIL github.com/ramendr/ramen/e2e 0.683s
Example run - ramen deployed:
% go test -run TestSuites/Validate --config ../e2e.yaml
2024-10-29T19:43:51.899+0200 INFO e2e/main_test.go:41 Global setting {"Timeout": 600}
2024-10-29T19:43:51.900+0200 INFO e2e/main_test.go:42 Global setting {"Retry Interval": 30}
2024-10-29T19:43:51.901+0200 INFO e2e/main_test.go:57 TestSuites
2024-10-29T19:43:51.932+0200 INFO util/validation.go:35 Ramen hub operator is running {"pod": "ramen-hub-operator-659658dfcf-c2xnj"}
2024-10-29T19:43:51.951+0200 INFO util/validation.go:59 Ramen dr cluster operator is running {"cluster": "c1", "pod": "ramen-dr-cluster-operator-74f89d85d7-nwx54"}
2024-10-29T19:43:51.970+0200 INFO util/validation.go:59 Ramen dr cluster operator is running {"cluster": "c2", "pod": "ramen-dr-cluster-operator-74f89d85d7-6gjhs"}
PASS
ok github.com/ramendr/ramen/e2e 0.759s
Signed-off-by: Nir Soffer <[email protected]>
nirs
force-pushed
the
e2e-logs
branch
2 times, most recently
from
be2abe1
to
2bd2ced
Compare
When a test fails, we the go test framework logs the failure in an unhelpful way. This works for unit tests, since they are simple and do not log anything, but our tests are complex and log a lot. The failure logs from go test are hidden and hard to find. This change add wrappers for t.Fatal, t.FailNow, t.Skip, and t.Skipf so they use our logger instead of the go test framework logger. This ensures that every failure will have a proper ERROR log which is very easy to find. Errors logs include a traceback making it easier to understand the failure. When handling errors in parent tests we don't have any context and we cannot log any useful error. The actual error that failed the sub test already logged the error, so we just mark the test as failed with util.FailNow(). Fixes: RamenDR#1595 Signed-off-by: Nir Soffer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a test fails, we the go test framework logs the failure in an
unhelpful way. This works for unit tests, since they are simple and do
not log anything, but our tests are complex and log a lot. The failure
logs from go test are hidden and hard to find.
This change add wrappers for t.Fatal, t.FailNow, t.Skip, and t.Skipf so
they use our logger instead of the go test framework logger. This
ensures that every failure will have a proper ERROR log which is very
easy to find. Errors logs include a traceback making it easier to
understand the failure.
When handling errors in parent tests we don't have any context and we
cannot log any useful error. The actual error that failed the sub test
already logged the error, so we just mark the test as failed with
util.FailNow().
Examples errors with this change:
Example build: https://github.com/RamenDR/ramen/actions/runs/11450725254/job/31880167402
Fixes #1595
Rebased on #1625 to save unneeded work.