Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asset downloading (and uploading) #343

Open
matthewhanson opened this issue Aug 4, 2022 · 1 comment
Open

Asset downloading (and uploading) #343

matthewhanson opened this issue Aug 4, 2022 · 1 comment
Labels
enhancement New feature or request

Comments

@matthewhanson
Copy link
Member

Is your feature request related to a problem? Please describe.

No problem, just a convenience

Describe the solution you'd like

I would like to be able to download selected assets from 1 or more STAC Items, save those items to a custom path, and update the hrefs in the assets to point to the new local files.

Similarly, I'd like to have STAC Items with some local assets and be able to upload those to a remote server using a custom path and update the hrefs in the assets.

Describe alternatives you've considered
This functionality was in sat-stac and sat-search and was widely used, but does not in exist in PySTAC or pystac-client. To support downloading from different sources, fsspec could be used which is already a stactools dependency. It's also higher level functionality that seems to make sense in stactools.

Additional context

Consider using fsspec to download from different sources, most important and https and s3 (with requester pays).

Consider async to support simultaneous downloads.

Support STAC Item or ItemCollection

User should be able to specify a save path with a template utilizing fields in the STAC Item. See cirrus-lib for an example in the [get_path](https://github.com/cirrus-geo/cirrus-lib/blob/features/task-class/src/cirrus/lib/utils.py#L54) function. Example template would be "my-output/${collection}/${id}" which would replace collection and id with those fields from the STAC Item. Also see the similar [pystac.layout](https://pystac.readthedocs.io/en/latest/api/layout.html) functionality which does a similar thing except with laying out catalog structures.

Saved assets should default to using a basename with the STAC Item ID and the asset key rather than the original filename. Assets could potentially reuse the same base filename (with different paths) as other assets resulting in one getting overwritten. Include option to use original filenames.

@matthewhanson matthewhanson added the enhancement New feature or request label Aug 4, 2022
@gadomski gadomski added this to the 0.5.0 milestone Mar 17, 2023
@gadomski
Copy link
Member

Once https://github.com/gadomski/stac-asset gets to v0.1 and moves to https://github.com/stac-utils, we could/should close this issue in favor of stac-asset.

@gadomski gadomski removed this from the 0.5.0 milestone Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants