Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EHN: Support multiple timeseries files for creating a GeoDataset object #453

Closed
Tjalling-dejong opened this issue Jul 20, 2023 · 3 comments · Fixed by #489
Closed

EHN: Support multiple timeseries files for creating a GeoDataset object #453

Tjalling-dejong opened this issue Jul 20, 2023 · 3 comments · Fixed by #489
Assignees
Labels
Enhancement New feature or request
Milestone

Comments

@Tjalling-dejong
Copy link
Contributor

Kind of request

Adding new functionality

Enhancement Description

This issue was raised in #372. But the change is quite big, so a separate issue is the right thing to do.

The fn_data argument can be multiple (csv) files: for now geodataset require one file per variable with all IDs but for some data this could be organized as one file per ID (with all variables)

This issue depends on how issue #372 is solved.

Use case

@hboisgon Could you elaborate a bit more on the use case of this issue?

Additional Context

No response

@Tjalling-dejong Tjalling-dejong added Enhancement New feature or request Needs refinement issue still needs refinement labels Jul 20, 2023
@Tjalling-dejong Tjalling-dejong changed the title support multiple timeseries files for creating a GeoDataset EHN: Support multiple timeseries files for creating a GeoDataset object Jul 20, 2023
@hboisgon
Copy link
Contributor

Sure! But I think you explained it yourself : for now geodataset require one file per variable with all IDs but for some data this could be organized as one file per ID (with all variables)

My current use case is for lakes/reservoirs where classically you have one file per lake ID with rating curve tables (ie A=f(H) and Q=f(H)). But it can also happens for meteo/water quality observation stations where they have sometimes an excel file with one tab per station (and pandas can only open one tab at a time).

@DirkEilander
Copy link
Contributor

DirkEilander commented Jul 21, 2023

I think what we need to progress on this is a detailed description of how the csv/excel files look that we want to support. In your case each file (or tab) has data (can be multiple variables?) for a single location. But do we also want to support files with a single variables for multiple locations (i.e. one file per variable)? How is the file linked to the geospatial index? Through file naming conventions of the csv file? Or a filename in the attribute table of the locations file? How do we set the names for dimensions and variables if we translate the files to a xarray.Dataset?

In the light of the discussion in #372 I would also argue against including this in the io.open_geodaset method. Instead my suggestion would be to make this a separate driver in the DataFrameAdapter (return a multiindex DataFrame) or a driver in a new DatasetAdapter.

@hboisgon
Copy link
Contributor

hboisgon commented Aug 4, 2023

Need to make a decision what we do want to support and see if this can be part of pre-processing or added in the data adapter

@savente93 savente93 linked a pull request Sep 20, 2023 that will close this issue
5 tasks
@hboisgon hboisgon added this to the Q3 milestone Sep 21, 2023
@hboisgon hboisgon removed the Needs refinement issue still needs refinement label Sep 21, 2023
@savente93 savente93 added the Blocked An issue that cannot be progressed right now label Oct 5, 2023
@savente93 savente93 self-assigned this Oct 5, 2023
@savente93 savente93 modified the milestones: Q3, Q4 Oct 5, 2023
@savente93 savente93 removed the Blocked An issue that cannot be progressed right now label Oct 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants