Implement manifest preloading #649
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
paraseba
force-pushed
the
push-wkkztklqppop
branch
from
d024f8e to
8edba83
Compare
|
Thank you @dcherian for showing me this list of regexes! Exactly what I needed |
This creates a pretty advanced mechanism to configure what manifest should be preloaded on session creation. It can match arrays by name, path or size; and it can create complex conditions using `or` and `and` combinators. By default, we preload manifests that have not more than 1k refs, up to a total of 10k refs, and corresponding to arrays with names matching any of the following regexes: * `\bt\b|(time|min|hour|day|week|month|year)[0-9]*` * `(z|nav_lev|gdep|lv_|[o]*lev|bottom_top|sigma|h(ei)?ght|altitude|depth|isobaric|pres|isotherm)[a-z_]*[0-9]*` * `y|j|nlat|rlat|nj` * `y?(nav_lat|lat|gphi)[a-z0-9]*` * `x?(nav_lon|lon|glam)[a-z0-9]*` * `x|i|nlon|rlon|ni` The regex were taken from: https://github.com/xarray-contrib/cf-xarray/blob/1591ff5ea7664a6bdef24055ef75e242cd5bfc8b/cf_xarray/criteria.py#L149-L160
paraseba
force-pushed
the
push-wkkztklqppop
branch
from
8edba83 to
1a5f17e
Compare
(and some other typing fixes)
paraseba
force-pushed
the
push-wkkztklqppop
branch
from
16fa3b3 to
e7cfb09
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This creates a pretty advanced mechanism to configure what manifest should be preloaded on session creation.
It can match arrays by name, path or size; and it can create complex conditions using
orandandcombinators.By default, we preload manifests that have not more than 1k refs, up to a total of 10k refs, and corresponding to arrays with names matching any of the following regexes:
\bt\b|(time|min|hour|day|week|month|year)[0-9]*(z|nav_lev|gdep|lv_|[o]*lev|bottom_top|sigma|h(ei)?ght|altitude|depth|isobaric|pres|isotherm)[a-z_]*[0-9]*y|j|nlat|rlat|njy?(nav_lat|lat|gphi)[a-z0-9]*x?(nav_lon|lon|glam)[a-z0-9]*x|i|nlon|rlon|niThe regex were taken from:
https://github.com/xarray-contrib/cf-xarray/blob/1591ff5ea7664a6bdef24055ef75e242cd5bfc8b/cf_xarray/criteria.py#L149-L160