Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prep for 1.9.0-rc10 pre-release #1647

Merged
merged 7 commits into from
Oct 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ jobs:
python-version: ["3.10"]
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Setup conda
uses: s-weigand/setup-conda@v1
with:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ on:
- '!contrib/**'

workflow_dispatch:

env:
DOCKER_USER: gadockersvc
DOCKER_IMAGE: opendatacube/datacube-tests:latest1.9
Expand All @@ -25,7 +25,7 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0

Expand Down
4 changes: 2 additions & 2 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
## SPDX-License-Identifier: Apache-2.0
##
# gdal:ubuntu-small no longer comes with netcdf support compiled into gdal
FROM ghcr.io/osgeo/gdal:ubuntu-full-3.9.2 AS builder
FROM ghcr.io/osgeo/gdal:ubuntu-full-3.9.2
FROM ghcr.io/osgeo/gdal:ubuntu-full-3.9.3 AS builder
FROM ghcr.io/osgeo/gdal:ubuntu-full-3.9.3
ARG V_PG=16
ARG V_PGIS=16-postgis-3

Expand Down
11 changes: 11 additions & 0 deletions docker/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,14 @@ docker run --rm \
opendatacube/datacube-tests:latest \
./check-code.sh integration_tests
```

## Updating Dependencies

Run the command:

```shell
docker run --rm \
-v $(pwd):/code -w /code/docker \
-it opendatacube/datacube-tests \
bash -c "python3 -m pip install pip-tools && pip-compile --upgrade --output-file=constraints.txt --strip-extras constraints.in"
```
40 changes: 20 additions & 20 deletions docs/MIGRATION-1.8-to-1.9.rst
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ Major Changes between 1.8.x and 1.9.x
metadata, such tools may not be possible in all cases. (The legacy postgres driver will continue to support
non-EO3 metadata types until it is dropped in 2.0)

The postgis driver will support the creation of PostGIS spatial indexes for arbitrary CRSs. This will improve
The postgis driver will support the creation of PostGIS spatial indexes for arbitrary CRSes. This will improve
efficiency and accuracy of database searches, particularly when working with data covering regions where
conversions to/from EPSG:4326 lat/long coordinates are highly non-linear (e.g. the Pacific around the
anti-meridian and the north and south polar regions).
Expand Down Expand Up @@ -140,43 +140,43 @@ Configuration
The configuration for a postgis index looks the same as the configuration for a legacy postgres index, you simply
set the ``index_driver`` setting to ``postgis``::

[default]
alias: prod
[default]
alias: prod

[old]
index_driver: postgres
db_hostname: production.dbs.internal
db_database: odc_prod
db_username: odc
db_password: secret_and_secure
[old]
index_driver: postgres
db_hostname: production.dbs.internal
db_database: odc_prod
db_username: odc
db_password: secret_and_secure

[new]
index_driver: postgis
db_hostname: dev.dbs.example.net
db_database: odc_dev
db_username: odc
[new]
index_driver: postgis
db_hostname: dev.dbs.example.net
db_database: odc_dev
db_username: odc

Initialisation
++++++++++++++

You then initialise the database as previously, using ``system init`` command (-E new says to use the ``new`` environment
from the configuration file)::

datacube -E new system init
datacube -E new system init

You should also create Postgis spatial indexes for any CRS you want to be able to search on (note that an EPSG:4326
spatial index is created by default). Postgis spatial indexes should be created before indexing any data where
possible. Adding a new spatial index to a populated index will be very slow. For example to create a spatial index
for EPSG:3577::

datacube -E new spindex create 3577
datacube -E new spindex create 3577

Migrating (Cloning) Data From a Postgres Index
++++++++++++++++++++++++++++++++++++++++++++++

To clone data from an old index to a new one (the two indexes may use different index drivers)::

datacube -E new system clone old
datacube -E new system clone old

Note that the target index is specified with the ``-E`` flag and the source index is provided as an argument to the
``system clone`` command.
Expand All @@ -198,7 +198,7 @@ Geospatial search

Geopolygons for spatial search can be passed to ``dc.load()``, as before::

dc.load(...., geopolygon=poly, ...)
dc.load(...., geopolygon=poly, ...)

In the postgres driver, the search is done against a bounding box around the polygon projected into EPSG:4326,
then the extents of datasets returned by the bounding box search are checked for overlap with the original
Expand All @@ -207,5 +207,5 @@ geopolygon. In the postgis driver, the polygon is passed directly to Postgis fo
* Only datasets whose extents overlap the geopolygon will be loaded.
* Geopolygons whose CRS does NOT have a native spatial index available will be projected to EPSG:4326 for search
purposes.
* Datasets whose projected extents are not contained within a given CRS's "valid area" will NOT be included in that
CRS's spatial index.
* Datasets whose projected extents are not contained within a given CRSes "valid area" will NOT be included in that
CRSes spatial index.
4 changes: 2 additions & 2 deletions docs/README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Developing Locally

Requires a Unix like system that includes ``make``.

#. Clone the datacube-core repository. If you don't have permissions to push to the datacube-core library, you will need to fork the repo and clone your fork.
#. Clone the datacube-core repository. If you don't have permissions to push to the datacube-core library, you will need to fork the repository and clone your fork.

.. code-block:: bash

Expand Down Expand Up @@ -48,7 +48,7 @@ Requires a Unix like system that includes ``make``.

.. code-block:: bash

pip install -r requirements.txt
pip install -r requirements.txt

#. Run the autobuild.

Expand Down
1 change: 1 addition & 0 deletions docs/about/license.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@ ODC License
***********

.. literalinclude::

../../LICENSE
6 changes: 5 additions & 1 deletion docs/about/whats_new.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ What's New
v1.9.next
=========

v1.9.0-rc10 (25th October 2024)
===============================

- Permissions management cleanup in postgis driver. (:pull:`1613`)
- Add `skip_broken_datasets` and `dc_load_limit` config options. (:pull:`1616`)
- Enable global environment variables with `ODC_ALL` naming convention (:pull:`1616`)
Expand All @@ -23,6 +26,7 @@ v1.9.next
- Updates to index APIs - add `order_by` to dataset search, index name attribute, product most recent change method (:pull:`1643`)
- Fix alembic migrations (:pull:`1645`)
- Make psycopg2 an optional dependency (:pull:`1648`)
- Cherry pick recent merged PRs from develop branch and update whats_new for rc10 release. (:pull:`1647`)

v1.9.0-rc9 (3rd July 2024)
==========================
Expand Down Expand Up @@ -133,7 +137,7 @@ v1.9.0-rc1 (27th March 2024)
- Deprecate search_eager and search_summaries and add `archived` arg to all dataset search/count methods. (:pull:`1550`)
- Compatibility fix - dc.load can take odc.geo GeoBox (:pull:`1551`)
- Migrate away from deprecated Python pkg_resources module (:pull:`1558`)
- Add ``custom_offsets`` and ``order_by`` arguments to search_retunrning() - order_by still unimplemented. (:pull:`1557`)
- Add ``custom_offsets`` and ``order_by`` arguments to search_returning() - order_by still unimplemented. (:pull:`1557`)
- Fix and enhance typehints, automated static type checking with mypy. (:pull:`1562`)
- Improve SQLAlchemy join hints, addressing an recurring but intermittent bug. (:pull:`1564`)
- Improve typehints and update docstrings in datacube/api/core.py (:pull:`1567`)
Expand Down
Loading
Loading