Remove data part & cleanup (#171)

* fast cleanup * linting * update date
dlr-eoc · May 4, 2023 · 0c0a0a3 · 0c0a0a3
1 parent cf53e86
commit 0c0a0a3
Show file tree

Hide file tree

Showing 16 changed files with 12 additions and 1,094 deletions.
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -1,6 +1,12 @@
 Changelog
 =========
 
+[1.5.0] (2023-05-04)
+---------------------
+Deleted
+^^^^^^^
+- Removed data part #170
+
 [1.4.3] (2022-09-19)
 ---------------------
 Fixed

diff --git a/README.md b/README.md
@@ -11,10 +11,7 @@
 [![DOI](https://zenodo.org/badge/259635994.svg)](https://zenodo.org/badge/latestdoi/259635994)
 
 
-The UKIS-pysat package provides generic classes and functions to query, access and process multi-spectral and SAR satellite images.
-
-### data
-Download satellites data from different sources (currently Earth Explorer, SciHub, STAC), deal with and structure metadata.
+The UKIS-pysat package provides generic classes and functions to access and process multi-spectral and SAR satellite images.
 
 
 ### file
@@ -33,28 +30,9 @@ Here's an example about some basic features, it might also help to read through
 
 ```python
 # import all the required libraries
-from ukis_pysat.data import Source
 from ukis_pysat.file import get_sentinel_scene_from_dir
-from ukis_pysat.members import Datahub, Platform
 from ukis_pysat.raster import Image
 
-# connect to Copernicus Open Access Hub  and query metadata
-src = Source(Datahub.Scihub)
-meta = src.query_metadata(
-    platform=Platform.Sentinel2,
-    date=("20200101", "NOW"),
-    aoi=(11.90, 51.46, 11.94, 51.50),
-    cloud_cover=(0, 50),
-)
-for item in meta:  # item is now a PySTAC item
-    print(item.id)
-    uuid = item.properties["srcuuid"]
-
-    # download geocoded quicklook and image
-    src.download_quicklook(product_uuid=uuid, target_dir="/users/username/tmp")
-    src.download_image(product_uuid=uuid, target_dir="/users/username/tmp")
-
-    break
 
 # get sentinel scene from directory
 with get_sentinel_scene_from_dir("/users/username/tmp") as (full_path, ident):
@@ -65,19 +43,9 @@ with get_sentinel_scene_from_dir("/users/username/tmp") as (full_path, ident):
 For working with the Landsat we need an item id for downloading the product
 Check [Pystac](https://pystac.readthedocs.io/en/1.0/) documentation for more functionality on [STAC](https://stacspec.org/).
 
-### Environment variables to configure Datahub credentials
-To use ``ukis_pysat.data`` and to download from the respective Datahub you need to set the credentials as environment variables.
-
-For EarthExplorer that's: \
-``EARTHEXPLORER_USER=your_username`` \
-``EARTHEXPLORER_PW=your_password``
-
-For SciHub that's: \
-``SCIHUB_USER=your_username`` \
-``SCIHUB_PW=your_password``
 
 ## Installation
-The easiest way to install `pysat` is through pip. Be aware, that Rasterio requires GDAL >= 1.11, < 3.1.
+The easiest way to install `ukis-pysat` is through pip. Be aware, that Rasterio requires GDAL >= 1.11, < 3.1.
 
 Most users will want to do this:
 ```shell

diff --git a/docs/index.rst b/docs/index.rst
@@ -6,48 +6,25 @@
 Welcome to UKIS-pysat's documentation!
 ======================================
 
-The UKIS-pysat package provides generic classes and functions to query, access and process multi-spectral and SAR satellite images.
-
-data
-____
-Download satellites data from different sources (currently Earth Explorer, SciHub, STAC), deal with and structure metadata.
+The UKIS-pysat package provides generic classes and functions to access and process multi-spectral and SAR satellite images.
 
 file
 ____
 Work with you local satellite data files and read information out of file names and metadata files. Currently, focusing on Sentinel-1.
 
 raster
 ______
-Reading satellite data and performing simple, but cumbersome tasks. This is just a layer on top of `rasterio <https://github.com/mapbox/rasterio>`__ for stuff we often need. It can very well be that using *rasterio* directly is often the better choice.
+Reading satellite data and performing simple, but cumbersome tasks. This is just a layer on top of `rasterio <https://github.com/rasterio/rasterio>`__ for stuff we often need. It can very well be that using *rasterio* directly is often the better choice.
 
 Example
 _______
 Here's an example about some basic features:
 
 .. code-block:: python
 
-   from ukis_pysat.data import Source
    from ukis_pysat.file import get_sentinel_scene_from_dir
-   from ukis_pysat.members import Datahub, Platform
    from ukis_pysat.raster import Image
 
-   # connect to Copernicus Open Access Hub  and query metadata
-   src = Source(Datahub.Scihub)
-   meta = src.query_metadata(
-       platform=Platform.Sentinel2,
-       date=("20200101", "NOW"),
-       aoi=(11.90, 51.46, 11.94, 51.50),
-       cloud_cover=(0, 50),
-   )
-   for item in meta:  # item is now a PySTAC item
-       print(item.id)
-       uuid = item.properties["srcuuid"]
-
-       # download geocoded quicklook and image
-       src.download_quicklook(product_uuid=uuid, target_dir="/users/username/tmp")
-       src.download_image(product_uuid=uuid, target_dir="/users/username/tmp")
-
-       break
 
    # get sentinel scene from directory
    with get_sentinel_scene_from_dir("/users/username/tmp") as (full_path, ident):
@@ -63,40 +40,6 @@ If the image was initialized with dimension order *last*, the result will be res
 
 Altering the array replaces all bands. If it is intended to alter a particular band, the remaining bands should be copied.
 
-STAC API Item IO Concept
-________________________
-To enable reading from different types of file systems (similar to `PySTAC <https://pystac.readthedocs.io/en/latest/concepts.html#using-stac-io>`__), it is recommended that in the ``__init__.py`` of the client module, or at the beginning of the script using ``ukis_pysat.data``, you overwrite the ``STACAPI_IO.read_item_method`` with function that read and write however you need. The following is an example for a on premise S3 environment:
-
-.. code-block:: python
-
-   from pystac import Item
-   from ukis_pysat.stacapi_io import STACAPI_IO
-
-   def on_premise_s3_url_method(feature, root_bucket="dem"):
-      """the href is build like /collections/*collection_prefix*/items/*item_prefix*
-
-      At some environments we will need to give back the href according to this method.
-      """
-      item = Item.from_dict(feature)
-      href = item.get_self_href()
-      stripped_href = href.replace(r"collections/", "").replace(r"items/", "")
-
-      return Item.from_file(f"s3://{root_bucket}{stripped_href}/{item.id}.json")
-
-   STACAPI_IO.read_item_method = on_premise_s3_url_method
-
-Environment variables
----------------------
-To use ``ukis_pysat.data`` and to download from the respective Datahub you need to set the credentials as environment variables.
-
-For EarthExplorer that's:
- | ``EARTHEXPLORER_USER=your_username``
- | ``EARTHEXPLORER_PW=your_password``
-
-For SciHub that's:
- | ``SCIHUB_USER=your_username``
- | ``SCIHUB_PW=your_password``
-
 
 .. toctree::
    :caption: Contents:

diff --git a/docs/installation.rst b/docs/installation.rst
@@ -16,8 +16,6 @@ There's also some lighter versions with less dependencies:
 
     pip install ukis-pysat  # only install core dependencies (ukis_pysat.file can be used)
 
-    pip install ukis-pysat[data]  # also install dependencies for ukis_pysat.data
-
     pip install ukis-pysat[raster]  # also install dependencies for ukis_pysat.raster
 
 Some helper functions might need additional dependencies like `pandas`, `dask[array]` or `utm`. If this is the case you will receive an `ImportError`.
@@ -29,7 +27,7 @@ If you're having troubles installing GDAL and Rasterio use conda and/or follow t
 
 Tests
 -----
-To run the tests set the Environment variables and then:
+To run the tests:
 
 .. code-block:: console
 

diff --git a/requirements.txt b/requirements.txt
@@ -1,13 +1,3 @@
-## data
-numpy
-pillow
-pystac[validation]>=1.0.0
-python-dateutil
-requests
-sentinelsat>=1.0.1
-shapely
-pydantic
-
 ## raster
 # numpy
 rasterio>=1.1.8
@@ -18,10 +8,8 @@ xmltodict
 ## dev
 dask[array]
 fiona
-landsatxplore>=0.13.0
 pandas
 pyproj
-requests_mock
 utm>=0.7.0
 geojson
 setuptools

diff --git a/setup.py b/setup.py
@@ -25,16 +25,6 @@ def get_version(rel_path):
 
 # please also update requirements.txt for testing. Make sure package versions are the same in each subset.
 extras_require = {
-    "data": [
-        "numpy",
-        "pillow",
-        "pystac>=1.0.0",
-        "python-dateutil",
-        "requests",
-        "sentinelsat>=1.0.1",
-        "shapely",
-        "pydantic",
-    ],
     "raster": [
         "numpy",
         "rasterio>=1.1.8",
@@ -50,10 +40,8 @@ def get_version(rel_path):
     + [
         "dask[array]",
         "fiona",
-        "landsatxplore>=0.13.0",
         "pandas",
         "pyproj>=3.0.0",
-        "requests_mock",
         "utm>=0.7.0",
         "sphinx>=1.3",
         "sphinx_rtd_theme",