Skip to content

Commit

Permalink
Fix volume/path issues that prevented hwitw_update.sh from running.
Browse files Browse the repository at this point in the history
Cronjob now launches successfully. Test with a job as described in README.
  • Loading branch information
mbjones committed Jul 7, 2023
1 parent c10f5dd commit e6e53e1
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 6 deletions.
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,12 @@ helm upgrade -n hwitw hwitw ./helm

You may want to modify the values passed to the application to set appropriate mount points for persistent storage and other relevant configuration variables. See the `values.yaml` file for details.

To manually start a job from the cronjob to update the CDS data through today, create a job with:

```sh
kubectl create job --from=cronjob.batch/hwitw-cdstool-cronjob cdstool-job
```

## Docker image builds and publication to GHCR

An open source versions of the `docker` commandline tool is `nerdctl`, which works pretty much as
Expand Down
4 changes: 2 additions & 2 deletions helm/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.8.0
version: 0.8.2

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "0.8.0"
appVersion: "0.8.2"
2 changes: 1 addition & 1 deletion helm/templates/cronjob-cdstool.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ spec:
readOnly: false
#command: ['sh', '-c', 'echo "TRON Liftoff!" > {{ .Values.persistence.mountPath }}/testfile.txt && cat {{ .Values.persistence.mountPath }}/testfile.txt']
#command: ['sh', '-c', 'cd {{ .Values.persistence.mountPath }} && python /cdstotile/cdstool.py --startyear 2023']
command: ['sh', '-c', '/cdstotile/hwitw_update.sh']
command: ['sh', '-c', 'cd {{ .Values.persistence.mountPath }} && /cdstotile/hwitw_update.sh']
volumes:
- name: config
secret:
Expand Down
6 changes: 3 additions & 3 deletions src/cdstotile/hwitw_update.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@
# (some tricky bash to give variable a default value if not set)
: ${CDS_DOWNLOAD_DIR:='.'}
: ${DATA_OUTPUT_DIR:='.'}
: ${CDSTOOL_DIR:=.}
: ${CDS_TOOL_DIR:=.}

# Start from the current year, or from $CDS_START_YEAR if it is already set
: ${CDS_START_YEAR:=$(date +%Y)}

set -e
# Download the latest Copernicus CDS data from $CDS_START_YEAR forward.
python3 ${CDSTOOL_DIR}/cdstool.py --startyear ${CDS_START_YEAR} --output "$CDS_DOWNLOAD_DIR"
python3 ${CDS_TOOL_DIR}/cdstool.py --startyear ${CDS_START_YEAR} --output "$CDS_DOWNLOAD_DIR"

# Process the data into our custom netcdfs and then into the wxdb
python3 ${CDSTOOL_DIR}/tiletool.py --start ${CDS_START_YEAR} --update --input "$CDS_DOWNLOAD_DIR" --output "$DATA_OUTPUT_DIR"
python3 ${CDS_TOOL_DIR}/tiletool.py --start ${CDS_START_YEAR} --update --input "$CDS_DOWNLOAD_DIR" --output "$DATA_OUTPUT_DIR"

0 comments on commit e6e53e1

Please sign in to comment.