Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consolidate multiple Parsers/readers from same instrument manuf. #592

Open
ocehugo opened this issue Oct 1, 2019 · 0 comments
Open

Consolidate multiple Parsers/readers from same instrument manuf. #592

ocehugo opened this issue Oct 1, 2019 · 0 comments
Labels
Status:Blocked hard dependent or requires external inputs Type:code_repo_consistency code-repo wise quirks Type:enhancement General enhancements Unit:Instrument Reader Instrument parsers and related Unit:Processing preprocessing and transformations
Milestone

Comments

@ocehugo
Copy link
Contributor

ocehugo commented Oct 1, 2019

It would be nice if we can reduce the number of Parsers and readers in the Parsers directory, do some refactoring and reduce the range of parsers provided to the user.

Some parsers use weak logic, exists only given historical changes, don't use enough regexes to support reading important information, don't read the files with correct encoding, etc.

Examples:

  1. RBR XR420 is only used with a weak filename matching.
  2. Most SBEs cnvs files are relatively well described, but logic is locked in several parsers/readers.
  3. WQMRaw/WQMdat - exists mostly to match different field/var names.
  4. wrong file encoding creates all sort of OR string matches.

We should aim to have a reader per Manufacturer with enough logic inside to handle multiple instruments and assigned names. This enables less guessing which reader to use, a single point of failure/success, less error in things such as names, comments, conversions, etc.
Also, some institutes created their parsers, and it can be hard to include then or merge changes without breakage.

However, only with a good set of tests, we can completely substitute the current infrastructure.

I will have to populate this issue with a list of related parsers, why they differ, etc. This way I can try to pinpoint the logic required for each manufacturer and if the generic parser needs updates.

For this issue to start, we've to collect more files for testing - I would say three variations for each equivalent instrument/parser.

@ocehugo ocehugo added Type:enhancement General enhancements Unit:Instrument Reader Instrument parsers and related Status:Blocked hard dependent or requires external inputs Type:code_repo_consistency code-repo wise quirks Unit:Processing preprocessing and transformations labels Oct 1, 2019
@ocehugo ocehugo modified the milestones: Q2 20/21, Q4 20/21, Q3 20/21 May 1, 2020
@ocehugo ocehugo modified the milestones: Q3 20/21, Q4 20/21, Q1/2022 Feb 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status:Blocked hard dependent or requires external inputs Type:code_repo_consistency code-repo wise quirks Type:enhancement General enhancements Unit:Instrument Reader Instrument parsers and related Unit:Processing preprocessing and transformations
Projects
None yet
Development

No branches or pull requests

1 participant