Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

csv2bufr to use data from station metadata when producing BUFR #763

Open
maaikelimper opened this issue Sep 6, 2024 · 2 comments
Open
Assignees
Labels
bug Something isn't working data transformation Data transformation
Milestone

Comments

@maaikelimper
Copy link
Collaborator

In the training material for csv2bufr Dave originally wrote this section:

!!! bug "Station metadata" 
    The CSV to BUFR conversion tool has been developed as a standalone tool separate from the wis2box.
    As such, all the data to be encoded in BUFR must be present in the CSV file, including the station metadata
    like the location and sensor heights. In future versions the web-application will be updated to load the metadata
    from the wis2box API and to insert the values into the CSV, similar to the process used in FM-12 SYNOP to BUFR.

I think we could implement some code to retrieve missing columns from the station metadata in the csv2bufr process in the wis2box-api plugin:
https://github.com/wmo-im/wis2box-api/blob/main/wis2box_api/plugins/process/csv2bufr.py

@david-i-berry could you help me brainstorm how this could be done ?

@maaikelimper maaikelimper added bug Something isn't working data transformation Data transformation labels Sep 6, 2024
@maaikelimper maaikelimper added this to the sprint-016 milestone Sep 6, 2024
@david-i-berry
Copy link
Member

@maaikelimper
Copy link
Collaborator Author

The input-columns depend on the mapping being used.
Each template might have different column-names to refer to the columns like 'latitude' 'longitude' 'altitude' 'barometer_height_above_sealevel' and we need to ensure the csv-string after the join matches the expected input-string for the template ...
I think in principle we have all the info we need in the mapping-object from this line onwards:
https://github.com/wmo-im/wis2box-api/blob/3f2499828df38598a8c39982170a55386680087c/wis2box_api/plugins/process/csv2bufr.py#L179

Do you have some suggested pseudo-code on how the mappings-object and the input csv could be used to provide a valid csv-string for the mapping template ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working data transformation Data transformation
Projects
None yet
Development

No branches or pull requests

2 participants