Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix config #561

Closed
wants to merge 16 commits into from
Closed

Fix config #561

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
71bb103
bump
b1ron Aug 21, 2023
afc8031
stats
b1ron Aug 21, 2023
194a0c4
bump
b1ron Aug 21, 2023
3fad45c
stats
b1ron Aug 21, 2023
58067bf
add get_more.js to fail
b1ron Aug 21, 2023
4c2e076
remove dupe
b1ron Aug 21, 2023
79118e2
Merge branch 'main' into fix-config
AlekSi Aug 25, 2023
121adab
roll back
b1ron Aug 25, 2023
76f1e89
stats
b1ron Aug 25, 2023
d51fb41
stats
b1ron Aug 25, 2023
c8a36fa
add comment
b1ron Aug 25, 2023
3069e64
Merge branch 'main' into fix-config
b1ron Aug 25, 2023
1603e57
fix spacing
b1ron Aug 25, 2023
37f5f1f
do not ignore TestCMAPProse, GODRIVER-2851 possibly fixed it
b1ron Aug 28, 2023
85f85dc
implied failure
b1ron Aug 28, 2023
9a160d8
Merge branch 'main' into fix-config
b1ron Sep 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion tests/mongo
Open in desktop
Submodule mongo updated 6666 files
2 changes: 1 addition & 1 deletion tests/mongo-go-driver
Open in desktop
Submodule mongo-go-driver updated 211 files
15 changes: 9 additions & 6 deletions tests/mongo-go-driver.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,8 @@ results:
ferretdb:
stats:
expected_fail: 10
expected_skip: 19
expected_pass: 633
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the number of passing tests went down?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's needs some investigation.

Copy link
Contributor Author

@b1ron b1ron Aug 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We seem to be running less subtests in total with the update. This is not the case for MongoDB. I am unsure why this is happening. I took a lot at the changelog but could not find a commit that might be the culprit. I logged some more output to verify that:

FerretDB:

CURRENT
commit: 6d814184973ae7ffc13281c93b423dc522a4bcca
Expectedly ignored: 16
Total number of tests: 678

UPDATE
commit: 6c7e1241b747d2fb5d164c9ca9d17617f33e7899
Expectedly ignored: 15
Total number of tests: 662

Any ideas?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This bumped our passes back up a little.

expected_skip: 16
expected_pass: 621

ignore:
# returns unknown result.
Expand Down Expand Up @@ -124,8 +124,6 @@ results:
- go.mongodb.org/mongo-driver/mongo/integration/TestClient/retry_writes_error_20_wrapped/write_error_code_19_right_msg
- go.mongodb.org/mongo-driver/mongo/integration/TestClient/retry_writes_error_20_wrapped/write_error_code_20
- go.mongodb.org/mongo-driver/mongo/integration/TestClient/retry_writes_error_20_wrapped/write_error_code_20_wrong_msg
- go.mongodb.org/mongo-driver/mongo/integration/TestCausalConsistency_NotSupported
- go.mongodb.org/mongo-driver/mongo/integration/TestCausalConsistency_NotSupported/afterClusterTime_not_included
- go.mongodb.org/mongo-driver/mongo/integration/TestCausalConsistency_NotSupported/clusterTime_not_included
- go.mongodb.org/mongo-driver/mongo/integration/TestChangeStream_Standalone
- go.mongodb.org/mongo-driver/mongo/integration/TestChangeStream_Standalone/no_custom_standalone_error
Expand Down Expand Up @@ -157,6 +155,8 @@ results:
- go.mongodb.org/mongo-driver/x/mongo/driver/integration/TestAggregate/Multiple_Batches
- go.mongodb.org/mongo-driver/x/mongo/driver/integration/TestAggregate/TestMaxTimeMSInGetMore
fail:
- go.mongodb.org/mongo-driver/mongo/integration/TestCausalConsistency_NotSupported
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There must be a comment explaining why those tests fail

Copy link
Contributor Author

@b1ron b1ron Aug 25, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to look into it more. They were previously passing now they panic.

	=== RUN   TestCausalConsistency_NotSupported
	--- FAIL: TestCausalConsistency_NotSupported (0.08s)
	
go.mongodb.org/mongo-driver/mongo/integration/TestCausalConsistency_NotSupported/afterClusterTime_not_included:
	=== RUN   TestCausalConsistency_NotSupported/afterClusterTime_not_included
	    --- FAIL: TestCausalConsistency_NotSupported/afterClusterTime_not_included (0.08s)
	panic: runtime error: invalid memory address or nil pointer dereference [recovered]
		panic: runtime error: invalid memory address or nil pointer dereference
	[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xe47b76]
	
	goroutine 72 [running]:
	testing.tRunner.func1.2({0xecf8e0, 0x17877e0})
		/usr/local/go/src/testing/testing.go:1526 +0x372
	testing.tRunner.func1()
		/usr/local/go/src/testing/testing.go:1529 +0x650
	panic({0xecf8e0, 0x17877e0})
		/usr/local/go/src/runtime/panic.go:890 +0x263
	go.mongodb.org/mongo-driver/mongo/integration.TestCausalConsistency_NotSupported.func1(0xc0002f0c40)
		/home/byron/workdir/dance/tests/mongo-go-driver/mongo/integration/causal_consistency_test.go:225 +0x256
	go.mongodb.org/mongo-driver/mongo/integration/mtest.(*T).RunOpts.func1(0x0?)
		/home/byron/workdir/dance/tests/mongo-go-driver/mongo/integration/mtest/mongotest.go:266 +0x5a9
	testing.tRunner(0xc0002fbba0, 0xc00003cde0)
		/usr/local/go/src/testing/testing.go:1576 +0x217
	created by testing.(*T).Run
		/usr/local/go/src/testing/testing.go:1629 +0x806

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not super confident on this but it seems that the changes made now cause their event monitor to return nil here when it tries to inspect the command name. Could be related to mongodb/mongo-go-driver#1347?

@AlekSi should I add this failing test to common ignore for now?

Copy link
Contributor Author

@b1ron b1ron Aug 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it only fails on FerretDB which invalidates the above. 👁️

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlekSi should I add this failing test to common ignore for now?

Added to FerretDB failures.

- go.mongodb.org/mongo-driver/mongo/integration/TestCausalConsistency_NotSupported/afterClusterTime_not_included
- go.mongodb.org/mongo-driver/examples/documentation_examples/TestDocumentationExamples

# (NotImplemented) `aggregate` stage "$project" is not implemented yet
Expand All @@ -182,10 +182,13 @@ results:
- go.mongodb.org/mongo-driver/mongo/integration/unified/TestUnifiedSpec
mongodb:
stats:
expected_skip: 432
expected_pass: 1940
expected_skip: 435
expected_pass: 1948

skip:
# skipping due to known failure: Test times out. See GODRIVER-2943
- go.mongodb.org/mongo-driver/mongo/integration/unified/TestUnifiedSpec/server-discovery-and-monitoring/unified/logging-standalone.json/Topology_lifecycle

# execution failed: test must be skipped: "the \"modifyCollection\" operation is not supported"
- go.mongodb.org/mongo-driver/mongo/integration/unified/TestUnifiedSpec/collection-management/modifyCollection-pre_and_post_images.json/modifyCollection_to_changeStreamPreAndPostImages_enabled
- go.mongodb.org/mongo-driver/mongo/integration/unified/TestUnifiedSpec/crud/unified/findOneAndUpdate-errorResponse.json/findOneAndUpdate_document_validation_errInfo_is_accessible
Expand Down
27 changes: 23 additions & 4 deletions tests/mongo.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ args:
results:
ferretdb:
stats:
expected_fail: 62
expected_pass: 39
expected_fail: 63
expected_pass: 38

fail:
# --- aggregation tests ---
Expand Down Expand Up @@ -166,10 +166,14 @@ results:
- mongo/jstests/auth/getMore.js
- mongo/jstests/auth/list_sessions.js

# uncaught exception: SyntaxError: import declarations may only appear at top level of a module
# all these syntax errors are fixed in MongoDB 7.0.
- mongo/jstests/readonly/get_more.js

mongodb:
stats:
expected_fail: 8
expected_pass: 93
expected_fail: 21
expected_pass: 80
fail:
# both tests invoke the MongoRunner and also fail on resmoke.py with exit code 253.
# they both seem to use a key file with incorrect permissions.
Expand All @@ -181,3 +185,18 @@ results:
- mongo/jstests/aggregation/use_query_project_and_sort.js
- mongo/jstests/aggregation/use_query_projection.js
- mongo/jstests/aggregation/use_query_sort.js
# uncaught exception: SyntaxError: import declarations may only appear at top level of a module
# all these syntax errors are fixed in MongoDB 7.0.
b1ron marked this conversation as resolved.
Show resolved Hide resolved
- mongo/jstests/aggregation/accumulators/min_n_max_n.js
- mongo/jstests/aggregation/bugs/cond.js
- mongo/jstests/aggregation/bugs/groupMissing.js
- mongo/jstests/aggregation/bugs/match.js
- mongo/jstests/aggregation/expressions/filter.js
- mongo/jstests/aggregation/expressions/set.js
- mongo/jstests/aggregation/ifnull.js
- mongo/jstests/aggregation/testall.js
- mongo/jstests/core/query/awaitdata_getmore_cmd.js
- mongo/jstests/core/query/cursor/getmore_invalidated_cursors.js
- mongo/jstests/core/query/elemmatch/elemmatch_object.js
- mongo/jstests/core/query/elemmatch/elemmatch_value.js
- mongo/jstests/readonly/get_more.js
Loading