Update Submissions OData to include deletedAt

[sadiqkhoja]

Part of getodk/central#709

What has been done to verify that this works as intended?

• Added unit and integration tests.
• Manually verified with the frontend.
• Checked with Excel to ensure there's no regression.

Why is this the best possible solution? Were any other approaches considered?

I have added __system/deletedAt field in the OData response and made it filterable. If client’s $filter expression doesn’t contain __system/deletedAt related clause then default __system/deletedAt eq null is added.

This approach keeps our Odata API consistent with respect to other system fields and quite useful for clients who are interested in filtering Submissions by deletedAt column. Maybe in future we also want to add this on the frontend.

Other option was to add a special filterable boolean field __system/deleted, which is simple but doesn't come with the benefits mentioned above.

OData spec doesn’t have anything explicit mechanism to track deleted records. There is a concept of deltaLink that returns list of new/changed/deleted records since last request. That concept is not related to what are looking for here.

---

I have added a new function odataExcludeDeleted that analyzes $filter expression for the existence of __system/deletedAt related clause. Initially, I had modified odataFilter function with the same logic but that made it convoluted, hence I refactored it into a separate function for the sake of simplicity at the cost of parsing $filter expression twice.

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

None.

Does this change require updates to the API documentation? If so, please update docs/api.yaml as part of this PR.

Done.

Before submitting this PR, please make sure you have:

• run make test and confirmed all checks still pass OR confirm CircleCI build passes
• verified that any code from external sources are properly credited in comments or that everything is internally sourced