Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

used osm-rawdata instead of osm-fieldwork for data extracts from osm #877

Closed
wants to merge 3 commits into from

Conversation

Sujanadh
Copy link
Collaborator

@Sujanadh Sujanadh commented Oct 5, 2023

In this committed changes, osm-rawdata package is used in place of osm-fieldwork during data extraction from osm. It also includes features to handle custom config file upload for custom form. Similarly, it handles the situation if config file is not provided and the category matches with xlsform title using the config yaml file from the package it self.

@nrjadkry
Copy link
Collaborator

nrjadkry commented Oct 5, 2023

@Sujanadh do we need a new release of osm-fieldwork for this or are we good with the current?

@Sujanadh
Copy link
Collaborator Author

Sujanadh commented Oct 5, 2023

@nrjadkry yes we need the latest version of osm-fieldwork which is osm-fieldwork==0.3.6rc2, sorry i forgot to mention that. Our osm-rawdata returns error while using the yaml file from the package of older version.

@robsavoye robsavoye added enhancement New feature or request backend Related to backend code labels Oct 5, 2023
@robsavoye
Copy link
Collaborator

The latest version of osm-fieldwork uses osm-rawdata. The one thing to remember though is for a data extract to work in ODK Collect, you have to filter out anything not in the choices sheet. That's what osm-fieldwork does, it filters the data after making the extract. You can optionally call use the same filter_data.py class with osm-rawdata to do the same thing. More information on making the data extracts is here: https://hotosm.github.io/osm-fieldwork/about/externaldata/.

@spwoodcock
Copy link
Member

This PR should be using osm-fieldwork as Rob says, not osm-rawdata directly.

There is the functionality make_data_extract for this purpose in osm-fieldwork.

@robsavoye
Copy link
Collaborator

Actually it's ok to use osm-rawdata directly, but it is code duplication for what PostgresClient,getFeatures() does.

@spwoodcock spwoodcock closed this Oct 8, 2023
@spwoodcock
Copy link
Member

After discussion we decided the current implementation is good 👍

If we used osm-fieldwork, the issue will be that for a custom XLSFile and it's custom YAML file, those will be in the database, where fieldwork doesn't know how to look for them. So it may actually be easier to use osm-rawdata as it's more flexibile. Or we update fieldwork to not use the XLSForm Library all the time.

Let's continue with #881.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Related to backend code enhancement New feature or request
Projects
Development

Successfully merging this pull request may close these issues.

4 participants