Error trying to load local zarr dataset #634
Comments
william-silversmith
added
bug
|
Dang, that's pretty weird! I'll have to fix that. Thanks for reporting. |
|
I looked into this a bit more, and while the error is confusing, what is happening is that there is a missing .zarray file so it's trying different things. |
|
I'm pretty sure there is a Will try and put together a simple reproducible example (that includes writing some data first) when I have a bit more time. |
|
I may be mistaken in my implementation, but in the examples I investigated, there was a For example: |
|
Ah this is a single zarr array, not a zarr group (which would have another level of directories with |
|
Could you send me a zip file containing a small example? I could then debug it more efficiently. |
|
There's been a release of a new cloudvolume with improved Zarr support. Consider giving it a try! |
|
Hmm, still no luck. Here's a self-contained example that creates an array, then trys to load it: from cloudvolume import CloudVolume
import zarr
arr = zarr.open("test.zarr", "w", shape=(10, 10, 10), dtype="u4")
arr[:] = 2
vol = CloudVolume('zarr://file://test.zarr')---------------------------------------------------------------------------
InfoUnavailableError Traceback (most recent call last)
Cell In[7], line 1
----> 1 vol = CloudVolume('zarr://file://test.zarr')
File ~/Data/heart-meshing/.venv/lib/python3.12/site-packages/cloudvolume/cloudvolume.py:255, in CloudVolume.__new__(cls, cloudpath, mip, bounded, autocrop, fill_missing, cache, compress_cache, cdn_cache, progress, info, provenance, compress, compress_level, non_aligned_writes, parallel, delete_black_uploads, background_color, green_threads, use_https, max_redirects, mesh_dir, skel_dir, agglomerate, secrets, spatial_index_db, lru_bytes, cache_locking, lru_encoding)
253 raise err
254 else:
--> 255 raise err
File ~/Data/heart-meshing/.venv/lib/python3.12/site-packages/cloudvolume/cloudvolume.py:247, in CloudVolume.__new__(cls, cloudpath, mip, bounded, autocrop, fill_missing, cache, compress_cache, cdn_cache, progress, info, provenance, compress, compress_level, non_aligned_writes, parallel, delete_black_uploads, background_color, green_threads, use_https, max_redirects, mesh_dir, skel_dir, agglomerate, secrets, spatial_index_db, lru_bytes, cache_locking, lru_encoding)
242 raise UnsupportedFormatError(
243 "Unknown format {}".format(path.format)
244 )
246 try:
--> 247 return init(cloudpath)
248 except InfoUnavailableError as err:
249 if 'precomputed://' not in cloudpath and cloudpath[:4] != 'zarr':
File ~/Data/heart-meshing/.venv/lib/python3.12/site-packages/cloudvolume/cloudvolume.py:240, in CloudVolume.__new__.<locals>.init(cloudpath)
238 path = strict_extract(cloudpath)
239 if path.format in REGISTERED_PLUGINS:
--> 240 return REGISTERED_PLUGINS[path.format](**kwargs)
241 else:
242 raise UnsupportedFormatError(
243 "Unknown format {}".format(path.format)
244 )
File ~/Data/heart-meshing/.venv/lib/python3.12/site-packages/cloudvolume/datasource/zarr/__init__.py:47, in create_zarr(cloudpath, mip, bounded, autocrop, fill_missing, cache, compress_cache, cdn_cache, progress, info, compress, compress_level, non_aligned_writes, delete_black_uploads, parallel, green_threads, secrets, cache_locking, **kwargs)
28 config = SharedConfiguration(
29 cdn_cache=cdn_cache,
30 compress=compress,
(...)
38 cache_locking=cache_locking,
39 )
40 cache = CacheService(
41 cloudpath=get_cache_path(cache, cloudpath),
42 enabled=bool(cache),
43 config=config,
44 compress=compress_cache,
45 )
---> 47 meta = ZarrMetadata(cloudpath, config=config, cache=cache, info=info)
48 imagesrc = ZarrImageSource(
49 config, meta, cache,
50 autocrop=bool(autocrop),
(...)
53 fill_missing=bool(fill_missing),
54 )
56 return CloudVolumePrecomputed(
57 meta, cache, config,
58 imagesrc, mesh=None, skeleton=None,
59 mip=mip
60 )
File ~/Data/heart-meshing/.venv/lib/python3.12/site-packages/cloudvolume/datasource/zarr/metadata.py:50, in ZarrMetadata.__init__(self, cloudpath, config, cache, info)
47 self.zattrs = self.default_zattrs()
49 if orig_info is None:
---> 50 self.info = self.fetch_info()
51 else:
52 self.render_zarr_metadata()
File ~/Data/heart-meshing/.venv/lib/python3.12/site-packages/cloudvolume/datasource/zarr/metadata.py:471, in ZarrMetadata.fetch_info(self)
468 if res is not None:
469 self.zarrays.extend(res)
--> 471 return self.zarr_to_info(self.zarrays, self.zattrs)
File ~/Data/heart-meshing/.venv/lib/python3.12/site-packages/cloudvolume/datasource/zarr/metadata.py:397, in ZarrMetadata.zarr_to_info(self, zarrays, zattrs)
394 num_channels = 1
396 if not zarrays:
--> 397 raise exceptions.InfoUnavailableError()
399 base_res = self.spatial_resolution_in_nm(0, zattrs, zarrays)
401 def extract_chunk_size(chunk_size):
InfoUnavailableError: |
|
A bit more debugging, and this is because the Zarr array doesn't have a |
|
I tried making a zarr array and was able to reproduce the problem with: z1 = zarr.open('data/example.zarr', mode='w', shape=(10000, 10000),
...
... chunks=(1000, 1000), dtype='i4')This doesn't follow the usual pattern for Neuroglancer, but it looks like Neuroglancer can visualize it, so I guess I should try to support it. |
gives me
Somewhere in that message it says
Cloud Path Recieved: zarr2://zarr://file:///Users/dstansby/notebooks/hipct/meshgen/4, which doesn't seem right - there's two protocol identifiers on the front (zarr and zarr2), despite the fact I only passed zarr in the CloudVolume call.The text was updated successfully, but these errors were encountered: