Geoparquet column 'geometry' does not have geometry types

[jutiss]

After upgrading to 1.1.0, it is no longer possible to open old parquet files created with WKB geometry.

Invalid Input Error: Geoparquet column 'geometry' does not have geometry types

The error occurs even when we do not select geometry columns. I assume that DuckDB is trying to automatically convert geometry columns to GEOMETRY type. Is there any way to disable this? Or maybe you can recommend some other way to handle this?

[carlopi]

This looks to me connected to: duckdb/duckdb-node#124.

A very rough workaround is suggested, but we aim at solving this at the duckdb library level.

[Maxxen]

How was the GeoParquet file created? This error occurs because DuckDB detects that the file is a geoparquet file based on the presence of geoparquet key-value metadata in the footer, but the geometry_types REQUIRED field is missing (or is not an array). Theoretically this metadata isn't used by DuckDB anyway, so we could just ignore its presence, but that would technically be against the geoparquet standard.

[jutiss]

@Maxxen Unfortunately, I don't have all the details of how these parquet files were created, as they have been transferred to me. In the metadata, I see that something called parquet-cpp-arrow version 10.0.1 was used and the person processing them told me he used geopandas.

[Maxxen]

Alright, might be that it was based one of the early drafts of GeoParquet that had not finalized the spec. Either way I'll change it on our end to not throw an error if we can't detect all required geoparquet metadata - in which case we will fall back to reading the file as a normal parquet file.

[Maxxen]

We've now added a setting to control geoparquet conversion, so you can now work around this by doing

SET enable_geoparquet_conversion = false

to keep reading the files as normal parquet, and then perform the WKB conversion manually using ST_GeomFromWKB.