You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Following up on #36
We would like to be able to convert columns in a dataset to some type specified by the user. This could for example be done by allowing the user to specify a schema on the cmdline, that is used to convert the dataset to if it is different from the schema in the dataset itself.
One challenge is how to store this modified dataset, because there is no place for this in the directory structure.
The text was updated successfully, but these errors were encountered:
I wonder if we're headed towards using a hash of the metadata as a dirname
Yes, it will become very challenging to distinguish between instances of a certain dataset with different schema's... Besides that, we need to take into account that 1) some properties are optional (but we can fix that with defaults) and 2) users should be able to prune entries and right now, they can just use the directory structure to prune multiple entries underneath a given directory.
Following up on #36
We would like to be able to convert columns in a dataset to some type specified by the user. This could for example be done by allowing the user to specify a schema on the cmdline, that is used to convert the dataset to if it is different from the schema in the dataset itself.
One challenge is how to store this modified dataset, because there is no place for this in the directory structure.
The text was updated successfully, but these errors were encountered: