Skip to content

Commit

Permalink
Allow for there to be no valid data in a arrow table string column
Browse files Browse the repository at this point in the history
  • Loading branch information
timj committed Sep 30, 2024
1 parent ca923c0 commit 178fd9d
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion python/lsst/daf/butler/formatters/parquet.py
Original file line number Diff line number Diff line change
Expand Up @@ -1098,7 +1098,8 @@ def _arrow_string_to_numpy_dtype(
# String/bytes length from header.
strlen = int(schema.metadata[encoded])
elif numpy_column is not None and len(numpy_column) > 0:
strlen = max([len(row) for row in numpy_column if row])
lengths = [len(row) for row in numpy_column if row]
strlen = max(lengths) if lengths else 0

dtype = f"U{strlen}" if schema.field(name).type == pa.string() else f"|S{strlen}"

Expand Down

0 comments on commit 178fd9d

Please sign in to comment.