Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: refactors cli.network suites with 'Integration' to use common function #1652

Merged
merged 12 commits into from
Nov 1, 2023

Conversation

Unique-Divine
Copy link
Member

@Unique-Divine Unique-Divine commented Oct 27, 2023

Summary

  1. Makes our implementation of the convention for skipping network tests with testing.short and the -short flag consistent across modules.

  2. Makes it so that simulations (simapp tests) are filtered out without us needing to use go list. As can be seen in contrib/make/simulation.mk, the simulation tests use an -Enabled=true flag in setup and aren't considered unit tests (using the -short flag). Their setup can follow the same convention as the integration tests by calling testutil.BeforeIntegrationSuite.

  3. ci: In the unit-test.yml GitHub action, there's no reason to save a coverage.txt since it's not published or visible from outside the workflow. We're only using the reports from integration-test.yml, so it makes sense to use make test-unit here and have the action run faster.

Purpose

  1. This change makes the code less repetitive and easier to maintain by having the doc comment on a common function.

  2. Unit tests run in 3-6 minutes now instead of 8+ min.

  3. This will fix the false-alarm CI failures resulting from cli.Network tests running when they shouldn't be. For example, https://github.com/NibiruChain/nibiru/actions/runs/6566058545/job/17835937660?pr=1643:

    [Expand example block]
    -- FAIL: TestIntegrationTestSuite (16.63s)
        --- FAIL: TestIntegrationTestSuite/TestSuccessfulVoting (16.63s)
            network.go:190: acquiring test network lock
            network.go:200: preparing test network...
            network.go:467: starting test network...
            network.go:473: started validator 0
            app_test.go:46: 
                    Error Trace:	/home/runner/work/nibiru/nibiru/x/oracle/integration/app_test.go:46
                                                /home/runner/go/pkg/mod/github.com/stretchr/[email protected]/suite/suite.go:187
                    Error:      	Received unexpected error:
                                    listen tcp 0.0.0.0:35143: bind: address already in use
                    Test:       	TestIntegrationTestSuite/TestSuccessfulVoting
    FAIL
    

@Unique-Divine Unique-Divine requested a review from a team as a code owner October 27, 2023 22:01
@codecov
Copy link

codecov bot commented Oct 27, 2023

Codecov Report

Merging #1652 (f9c9685) into master (2b99ca7) will increase coverage by 0.02%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##           master    #1652      +/-   ##
==========================================
+ Coverage   74.25%   74.28%   +0.02%     
==========================================
  Files         191      191              
  Lines       15099    15096       -3     
==========================================
+ Hits        11212    11214       +2     
+ Misses       3256     3251       -5     
  Partials      631      631              
Files Coverage Δ
x/spot/client/testutil/suite.go 97.93% <100.00%> (+0.38%) ⬆️
x/common/testutil/cases.go 70.00% <40.00%> (+70.00%) ⬆️

Comment on lines 24 to +25
- name: Run all unit tests.
run: make test-coverage
run: make test-unit
Copy link
Member Author

@Unique-Divine Unique-Divine Oct 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the unit-test.yml GitHub action, there's no reason to save a coverage.txt since it's not published or visible. We're only using the reports from the integration-test.yml action. We can use make test-unit here to have the CI run faster.

Comment on lines -7 to +9
.PHONY: test-coverage
test-coverage:
go test ./... $(PACKAGES_NOSIMULATION) -short \
.PHONY: test-coverage-unit
Copy link
Member Author

@Unique-Divine Unique-Divine Oct 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using $PACKAGES_NOSIMULATION is unnecessary since simulations require a specific flag and we're already using the -short flag, which can filter out simulations

@jgimeno jgimeno merged commit 162fd05 into master Nov 1, 2023
16 of 17 checks passed
@jgimeno jgimeno deleted the realu/consistent-integration-suite branch November 1, 2023 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ✅ Completed
Development

Successfully merging this pull request may close these issues.

2 participants