-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow bypassing validation to read faulty input #207
Comments
While it's good to explore what's possible in Pydantic in #209, I fear this issue needs a written out plan first. Hydrolib-core is designed to match valid model input. That's already a range of possible models, links, defaults, versions (#180). I understand the feature request of being able to fix a broken input, but how broken? A single NaN? Or links between models that are not valid? Or too small a model to be valid? How do you migrate from these broken states to a valid one? Does that introduce a(nother) state? What's the API for that? Worse, most of our logic is now based on the guarantees that validation gives us. We assume something is a number, list or specific instance. If it's not, things will inevitably break. Also, it might be good to find out how existing models (as in files on disk) as proposed in #199, could be missing something like friction values. |
Indeed, before just making this, let's take one step back: what does a user expect to do with model input that contains invalid data? Is it convenient for users to repair invalid input directly in object state? (rather than in the input files)Missing or wrong optional numeric/character string values in a particular field: Missing or wrong filenames: How to fix invalid input files, still taking as much benefit as possible from Python-based validation?The validation error can be caught, and the ValidationError object inspected on its detailed contents, for example using: from pydantic import ValidationError
from hydrolib.core.io.mdu.models import FMModel
filepath = test_data_dir / "input/e02/c11_korte-woerden-1d/dimr_model/dflowfm/FlowFM.mdu
try:
fm_model = FMModel(filepath)
except ValidationError as error:
errorlist = error.errors()
print(f"Number of validation errors: {len(errorlist)}.")
filelist = set(
map(
lambda e: e["loc"][4]
if isinstance((loc3 := e["loc"][3]), int)
else loc3,
errors,
)
)
print(f"Files with errors: {', '.join(filelist)}.")
print(f"Error details:\n{error}") which could produce:
@myrthearcadis, @ABuijert : can you please check whether the above example code helps you sufficiently? |
Hello,
Thanks for your response. This is a valid point and the sample code could help with this. For us this is less relevant:
* We are only interested in a small part of the model, but we cant read this part due tot errors in other parts
* The model is valid in the GUI en DIMR. We can view en run the model. Only not in Python. Possibly due to not used profiles oid.
* It is understandable that you can not write a corrupt model, but it should be possible to read the valid parts (also in light of coming D-Hydro changes)
* This could also be placed under an additional option or debug option
Hope this helps this discussion.
Arjon
|
Introduction:
This issue originates from multiple questions (a.o. #199), where sometimes it is desirable to bypass or switch off validation when reading model input files. Two different example situations:
Question:
How to bypass validation when reading a faulty D-HYDRO input file using HYDROLIB-core?
The text was updated successfully, but these errors were encountered: