Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: Add check for schema read compatibility (delta-io#554)
## What changes are proposed in this pull request? This PR introduces the `schema::compare` module to handle schema compatibility comparisons. This module introduces the `SchemaComparison` trait that adds the `can_read_as` method. This checks that for data written with schema `schema`, whether it can be read with the `read_schema`. This check is useful for implementing schema evolution checks in CDF. Closes delta-io#523 ## How was this change tested? Schema compatibility tests are added that check the following: - `can_read_as` is reflexive - adding a nullable column to each the key and value of a map succeeds. - changing a map value from nullable to non-nullable fails - same schema with different field name case fails - changing column type from long to integer fails. - Setting nullability from false to true succeeds - Setting nullability from true to false fails - Adding a nullable column succeeds - Adding a non-nullable column fails --------- Co-authored-by: Oussama Saoudi <[email protected]> Co-authored-by: OussamaSaoudi <[email protected]>
- Loading branch information