Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues with auto-derive xfac from manual metadata #197

Open
veenstrajelmer opened this issue Dec 20, 2023 · 0 comments
Open

Issues with auto-derive xfac from manual metadata #197

veenstrajelmer opened this issue Dec 20, 2023 · 0 comments

Comments

@veenstrajelmer
Copy link
Collaborator

veenstrajelmer commented Dec 20, 2023

Issues:
xfac=True (empirisch) according to compfile, but xfac=False would be better according to AB/KD. Accidentally already added to stats_xfac0 in example files. Prevent mixing of xfac=False components with xfac=True prediction:

  • L9PFM
  • Q1

Empirisch/theoretisch is not mentioned in component file (so assuming xfac=True), but xfac=False resulted in good reproductions of components/prediction. Indeed theoretical f/u were used according to KD:

  • AUKFPFM
  • CUXHVN
  • K13APFM
  • NORTHCMRT (although "theor. f" for SA/SM)

This can be tested with xfac = COMP_merged.xfac in predictie_2019_frommergedcomp.py. If conclusive, update stats_xfac0 in example scripts and maybe update excel file. Comparing metadata is possible with:

import os
import hatyan

dir_testdata = 'C:\\DATA\\hatyan_data_acceptancetests'
stats_xfac0 = ['A12','ABDN','AUKFPFM','BOURNMH','CROMR','CUXHVN','D15','DEVPT','DOVR','F16','F3PFM','FELSWE','FISHGD','IMMHM','J6','K13APFM','K14PFM','KINLBVE','L9PFM','LEITH','LLANDNO','LOWST','NEWHVN','NEWLN','NORTHCMRT','NORTHSS','PORTSMH','Q1','SHEERNS','STORNWY','WEYMH','WHITBY','WICK']
selected_stations = ["L9PFM","Q1","AUKFPFM","CUXHVN","K13APFM","NORTHCMRT"]

for current_station in selected_stations:
    print(f'current_station: {current_station}')
    if current_station in stats_xfac0:
        xfac=False
    else:
        xfac=True
    file_comp1 = os.path.join(dir_testdata,'predictie2019',f'{current_station}_ana.txt')
    comp_fromfile = hatyan.read_components(filename=file_comp1)
    compfile_xfac = comp_fromfile.attrs["xfac"]
    print(f'{current_station}: xfac={compfile_xfac} in file, xfac={xfac} in practice')
    # assert compfile_xfac == xfac

After fix, also re-enable in acceptance tests again:

  • After implementing 139 improve metadata in functions and component file #218, it is not possible anymore to make predictions with different xfac than derived from components file, so these stations cannot be reproduced anymore.
  • this is all acceptable, but useful to update the data files to fix all example testcases again.
  • CUXHVN in numbering_extremes.py
  • AUKFPFM in predictie_2019_frommergedcomp.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant