Skip to content
This repository has been archived by the owner on Oct 1, 2024. It is now read-only.

Handle dimensions consistently #64

Open
clbarnes opened this issue Jul 12, 2019 · 3 comments
Open

Handle dimensions consistently #64

clbarnes opened this issue Jul 12, 2019 · 3 comments

Comments

@clbarnes
Copy link
Collaborator

As discussed, N5 stores dimension metadata in java/fortran order. z5py (and presumably zarr) reverses that metadata and transposes the returned arrays into numpy/C order. pyn5 should do the same.

@clbarnes
Copy link
Collaborator Author

Further to discussion on slack, rust-n5's read/write_ndarray sets axis flags correctly, and numpy largely handles the different view orders for you, so really we just need to

  • test reading and writing C and F-ordered stuff
  • implement methods to return dataset metadata in the order the caller would expect

@clbarnes
Copy link
Collaborator Author

The rust/ low-level side is consistent with the java implementation; the h5py-like side manages the conversions as necessary. There is not currently a method for retrieving the dataset metadata in order, but I don't know if it's worth it: if you remember to use that method, you may as well just remember to do the inversion yourself. That would be necessary anyway if the attribute were anything but a 3-tuple.

@d-v-b
Copy link

d-v-b commented Nov 19, 2019

We were thinking about de-ambiguating array axes via tiny arrays which contain axis ID numbers at some of their corners. Indexing into the special corners of this array would return a number that indicates whether metadata should be read in python or java order.

That's probably super confusing. Here's an example: In 2D, we would make the following array: [[-1, 1], [0, -1]]. (The -1s are fill values, only the 1 and 0 matter here). Indexing this array with [-1, 0] would return 0 from python and 1 from java; indexing the array with [0, -1] would return 1 from python and 0 from java. Thus for metadata defined over axes, e.g. resolution : [4, 8], the correct indexing order can be derived from the result of indexing that special array.

tl;dr -- you can solve the axis order ambiguity issue by creating a little array that, when indexed, returns a number that encodes whether metadata are stored in [a,b,c] or [c,b,a] order.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants