expanduser implementation for geopandas

[ollie-bell]

GeoPandas read_file and to_file attempts a path.expanduser() which raises NotImplementedError when path is a UPath.

Is it intended that UPath.expanduser must raise NotImplementedError, or is it a matter of not getting round to it yet? For context I have had this issue when attempting to read/write on GCS.

In the meantime I am using a workaround to read/write via a io.BytesIO buffer, e.g.:

# write
with io.BytesIO() as buffer:
  gdf.to_file(buffer, driver="GeoJSON")
  buffer.seek(0)
  path.write_bytes(buffer.read())

# read
with io.BytesIO(path.read_bytes()) as buffer:
  gdf = gpd.read_file(buffer)

[ap--]

Hi @ollie-bell

Making expanduser a noop is worth considering. Would you be interested in making a PR?

1. You would have to change the base test here:

   universal_pathlib/upath/tests/cases.py

   Lines 67 to 69 in 5486fa3

   	def test_expanduser(self):
   	with pytest.raises(NotImplementedError):
   	self.path.expanduser()

2. And then change the default implementation to just return self.

Regarding the code example you provided as a workaround for now, is there any reason why the following does not work?

pth = UPath(...)

# write
with pth.open(mode="wb") as buffer:
  gdf.to_file(buffer, driver="GeoJSON")

# read
with pth.open(mode="rb") as buffer:
  gdf = gpd.read_file(buffer)

Cheers,
Andreas 😃

[ollie-bell]

Yeh sure I can look at doing a PR for expanduser!

On the second part, actually yes the read does work! The write does not though, pyogrio (GeoPandas default io backend) gives an error:

NotImplementedError: writing to an open file handle is not yet supported; instead, write to a BytesIO instance and then read bytes from that to write to the file handle

... in fact they suggest doing what I did for writing:

writing the dataset to BytesIO and then writing those bytes to the file handle (regular file handle via open, an fsspec opened file handle, or an open handle within a ZipFile),