Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use CFtime in stars for time management of netCDF files #719

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

pvanlaake
Copy link

This is a new PR after updating code to address issues with #690. This PR has been synched with the main branch minutes ago.

I have made a few updates to the code after studying issues with the tests. There are changes in dimensions.R and aggregate.R, compared to the changes in the previous PR.

Please note that I had to downgrade the dependency on sf: version 1.0-19 is not installing from CRAN (for MacOS) and building locally is impossible because I do not have the GDAL dev environment locally. I do not think that there is any impact of that on this PR.

Considering that there remain differences in the tests compared to the saved output, I have installed a clean version of the stars source from a zip file download from GitHub 2 days ago. Comparing tests output between that version and my PR reveals the following:

  • gridtypes.R, raster.R, stars.R have diffs not related to the PR.
  • area.R gives 15 warnings in both the PR code and the current main stars branch. Is area.Rout.save out of date with only 12 warnings?
  • Similarly with curvilinear.R.
  • dimensions.R will always have some differences because there is code added to deal with CFtime specifics. Otherwise the differences have been resolved.
  • mdim.R The objects created are the same, there are only differences in the attributes which are related to the fact that a CFtime instance is used to represent time from the netCDF file when doing read_mdim().

@pvanlaake
Copy link
Author

@edzer This PR fails on building three of the vignettes, with an error I haven't seen before. The issue is the following:

(stars1.Rmd, line 217) When making a stars object from 9 AVHRR images the result has 36 attributes named like "1.sst", "1.anom", "1.ice", "1.err", ... , "9.err". The "time" dimension is not merging and in the resulting object only the first day of 9 is included. (The error in vignettes 3 and 7 appears to be of the same nature.)

I looked around in the code, specifically in c.stars() and st_as_stars.list() and everything seems to be going fine in the first 9 iterations, but then in a 10th iteration (?) the 36 attributes pop up here:

if (identical_dimensions(dots, tolerance = tolerance)) {
	st_as_stars(setNamesIfnn(do.call(c, lapply(dots, unclass)), nms),
				dimensions = st_dimensions(dots[[1]]))
} else {
      ...

I have been trying to figure out where the code leads (nested functions, do.call(), lapply(); it's not easy to debug!) but I've come up short. Can you tell me which function(s) I should be looking at to merge the "time" dimension of the individual objects such that they merge nicely to 4 attributes? Thanks in advance for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant