Valijson and JSON Schema

[jviotti]

Hey there! I'm a TSC member of JSON Schema focused on building better and faster JSON Schema utilities for C++.

I saw this repo makes use of Valijson and its (limited) JSON Schema implementation. How is your use of Valijson going? Anything you like or dislike? Did you find a need for supporting newer dialects of JSON Schema (such as 2020-12)?

Any honest feedback would be super appreciated to make JSON Schema and my C++ efforts in there more valuable! 🙏🏻

[ifadams]

Hi @jviotti, we'll bring this up in our internal weekly working group meeting for further discussions, and below I'll add a few comments here based on my experience. (@cwlacewe know you're on leave, but a courtesy cc)

@araghuna1 can you add this to the agenda?

@rv355 you're the other person who's done mods to the JSON API definition, any thoughts here?

Forgive me if this is outside your area, but heres a couple thoughts I have based on my work with our JSON API and defintions broadly.

1. The error reporting when something fails to validate/parse is often a hot mess and not informative of what actually went wrong. Often times, I'll get a multi-page recursive-looking error for basic input mismatches that fail to validate.

2. I realize the API defintion file follows something of a standard, but pretty clunky and messy to use with more complex calls. Not sure there's a lot to be done there, its usable, but not always intuitive, particularly the "properties" keyword which is often also used a field name elsewhere in a variety of applications.

If I think of others, I'll let you know!

[jviotti]

Thanks a lot @ifadams !

The error reporting when something fails to validate/parse is often a hot mess and not informative of what actually went wrong. Often times, I'll get a multi-page recursive-looking error for basic input mismatches that fail to validate.

100%. This is a very hot problem on validators right now (even beyond RapidJSON) and I have my eyes on it. I'm looking to make this much better than the current status quo.

I realize the API defintion file follows something of a standard, but pretty clunky and messy to use with more complex calls. Not sure there's a lot to be done there, its usable, but not always intuitive, particularly the "properties" keyword which is often also used a field name elsewhere in a variety of applications.

Assuming you mean,

vdms/utils/src/api_schema/api_schema.json

Line 31 in 4eaa750

"description": "VDMS API",

, it can definitely be improved in various ways. The one that I would recommend the most to start with is breaking it up into many files. In the same way that we wouldn't write a lot of code in a single file, a huge schema in a single file is pretty hard to read and understand.

The classic approach is what's called JSON Schema Bundling. The idea is that schemas can reference schemas in other files through $ref (think like the C++ preprocessor #include) and the bundler outputs a monolithic one at build time (think like actually running the preprocessor). My CLI (as well as many other tools out there) support this workflow: https://github.com/Intelligence-AI/jsonschema/blob/main/docs/bundle.markdown.

Once you have that you can make sure it's fully compliant, run unit tests against the schema (so you know you are defining things well), linting, formatting, etc. All the typical stuff you would do with code.

If you want any help with it, let me know! Happy to plan and collaborate on it to polish the schema, etc :)

[ifadams]

@jviotti Thank you for the detailed response!

We're swamped at the moment, but if there's some obvious stuff you see within our API defintion schema that could use refining please let us know! I'm going to keep this ticket open for the moment so we dont lose track of this as a longer term tasks/discussion.

We're in the process of really digging into the open-source community aspect of contributions as well, so feel free to suggest things, open a new ticket/branch if you've got ideas, etc.