-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix tests for Aug 2023 updated remote datasets #2636
Conversation
Two synbath tests are not updated due to upstream dataset issue (GenericMappingTools/gmtserver-admin#213). |
Ok, looks they're re-processing the synbath dataset again, let's wait a day or two to see if it gets fixed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- In the tests, we use statements like
npt.assert_allclose(data.min(), -180.40002, rtol=1e-5)
to compare the grid value with the expected value. All datasets have a specific precision (e.g., theearth_geoid
dataset has a precision of 0.01), thus it makes no sense to compareearth_geoid
value with a number like-180.40002
and a relative tolerance of 1.0e-5. Instead, it makes more sense to compare the value with-180.4
and an absolute tolerance of 0.01 (the precision of the earth_geoid dataset). I've changedrtol
toatol
in all these tests.
Cool, setting the absolute tolerance does makes more sense. Looked through each dataset and the precision values look ok, except for the ones indicated below where I couldn't find the precision documented at https://www.generic-mapping-tools.org/remote-datasets/index.html.
I'm wondering if we should file an upstream issue to https://github.com/GenericMappingTools/remote-datasets/issues to have the precision information in the metadata too? That way we could do something like npt.assert_allclose(data.min(), ..., atol=data.attrs["precision"])
instead of hardcoding a value. Just in case the precision changes in the future.
npt.assert_allclose(data.min(), -8600.5, atol=0.5) | ||
npt.assert_allclose(data.max(), 5559.0, atol=0.5) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you get the uncertainty for the earth_relief grids from the paper(s)? I'm not seeing them on https://www.generic-mapping-tools.org/remote-datasets/earth-relief.html#technical-information.
I get the precisions from the recipes(https://github.com/GenericMappingTools/gmtserver-admin/tree/master/recipes) that were used in building the grids (the
I'm not sure, because these values are just the precisions that GMT used to build the grids, not exactly the real precisions of the original datasets. |
Ah I see, if it's just GMT's re-gridded precisions and not the actual precisions, then it probably shouldn't be in the metadata. Let's just wait for the grids to be fixed then, seems like the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May need to manually regenerate the cache by uncommenting
pygmt/.github/workflows/cache_data.yaml
Line 15 in dcf1c7f
# pull_request: |
If you're short on time, I can help push the changes? Just give me a 👍 |
Also setting absolute tolerance to 0.5
This reverts commit 364c1b5.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have updated the cache at 364c1b5 and re-ran the tests with updated minmax numbers, should all work now.
Description of proposed changes
Some tests related to the GMT remote datasets are failing due to the recent updates of these datasets (see https://www.generic-mapping-tools.org/remote-datasets/changes.html for a list of these changes). The min/max values of these datasets change slightly in the new versions.
This PR fixes these failing tests by updating the min/max values.
Notes:
npt.assert_allclose(data.min(), -180.40002, rtol=1e-5)
to compare the grid value with the expected value. All datasets have a specific precision (e.g., theearth_geoid
dataset has a precision of 0.01), thus it makes no sense to compareearth_geoid
value with a number like-180.40002
and a relative tolerance of 1.0e-5. Instead, it makes more sense to compare the value with-180.4
and an absolute tolerance of 0.01 (the precision of the earth_geoid dataset). I've changedrtol
toatol
in all these tests.Reminders
make format
andmake check
to make sure the code follows the style guide.doc/api/index.rst
.Slash Commands
You can write slash commands (
/command
) in the first line of a comment to performspecific operations. Supported slash commands are:
/format
: automatically format and lint the code/test-gmt-dev
: run full tests on the latest GMT development version