Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Treat warning as error in CI/Dev #973

Merged
merged 2 commits into from
Aug 1, 2024
Merged

Conversation

ndrluis
Copy link
Collaborator

@ndrluis ndrluis commented Jul 27, 2024

This will help us avoid propagating warnings to our users, as occurred in #971.

@sungwy
Copy link
Collaborator

sungwy commented Jul 28, 2024

Hi @ndrluis - thank you for turning the issue in #971 into these strategic items so quickly.

I'm +1 for re-enabling this flag as it will help us avoid issues like you've reported.

Testing this out in my dev environment, I see that other tests would also end up breaking with "expected" UserWarnings.

And example is:

UserWarning: Delete operation did not match any records, when overwrite is invoked in an empty table.

Some of our tests like: tests/catalog/test_sql.py::test_write_pyarrow_schema would fail in this case. One suggestion is to update this test to use table.append(pyarrow_table) instead so that these expected user warnings will not break our CI. WDYT?

@ndrluis
Copy link
Collaborator Author

ndrluis commented Jul 28, 2024

Thank you for the review, @sungwy. I made the change in commit 1c6c409.

@sungwy
Copy link
Collaborator

sungwy commented Jul 31, 2024

@ndrluis shall we rebase this branch with the fix you issued in the other PR #972 ?

ERROR tests/integration/test_writes/test_writes.py::test_rest_catalog_with_empty_catalog_name_append_data - UserWarning: Pydantic serializer warnings:
  Expected `TableIdentifier` but got `dict` - serialized value may not be as expected

@ndrluis ndrluis force-pushed the warning-as-errors branch 2 times, most recently from e8c1fdd to 288e4ec Compare July 31, 2024 02:35
@ndrluis
Copy link
Collaborator Author

ndrluis commented Jul 31, 2024

@sungwy Done!

@sungwy
Copy link
Collaborator

sungwy commented Jul 31, 2024

Thank you! Running the CI now

@Fokko
Copy link
Contributor

Fokko commented Jul 31, 2024

@ndrluis Ohh, this is nice. And sorry, it looks like I disabled this out a while ago 0cbb71c 🙈

This will help us avoid propagating warnings to our users, as occurred in apache#971.
@ndrluis
Copy link
Collaborator Author

ndrluis commented Jul 31, 2024

Just to clarify the test changes:

  "ignore:A plugin raised an exception during an old-style hookwrapper teardown.",
  "ignore:unclosed <socket.socket",
  "ignore:distutils Version classes are deprecated. Use packaging.version instead.",

These are filtering third-party warnings.

I changed the table overwrite to append when it makes sense to avoid the "Delete operation did not match any records" error, and maintained the overwrite in an empty table or a delete operation where there is nothing to delete, as it makes sense.

Additionally, the "Merge on read is not yet supported, falling back to copy-on-write" is an expected warning in the test context.

Copy link
Collaborator

@sungwy sungwy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a fantastic contribution to the repository @ndrluis - thank you very much for putting this in and fixing up the tests 🚀

I'll leave this PR open for comments, and merge it in later today or tomorrow

@sungwy sungwy merged commit dafcf22 into apache:main Aug 1, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants