You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should return the test for falsy values, otherwise we risk Astropy autocasting the column into a MaskedColumn without asking us. This is a symptom of the fact that the ButlerStandardizer is version dependent and we have data in our datastore that was processed, exported and then re-ingested, from different stack versions.
When the stack processing the data is of a later version than the one we use with ButlerStandardizer, and has removed one or more properties from objects like visit_info or summary_stats (which we use to extract metadata in the Butler), the returned value is sometimes ambiguous and is interpreted as a MaskedConstant (by either pybind or astropy). This auto-casts the entire column into a masked column which causes unexpected behavior.
The solution is either:
set the appropriate obvious, unambiguous, falsy value in the ButlerStandardizer itself (probably the best)
force the masked columns in the ImageCollection itself into a regular column by calling col.filled(default_value) and replacing the masked column
The text was updated successfully, but these errors were encountered:
See #695.
We should return the test for falsy values, otherwise we risk Astropy autocasting the column into a MaskedColumn without asking us. This is a symptom of the fact that the ButlerStandardizer is version dependent and we have data in our datastore that was processed, exported and then re-ingested, from different stack versions.
When the stack processing the data is of a later version than the one we use with ButlerStandardizer, and has removed one or more properties from objects like visit_info or summary_stats (which we use to extract metadata in the Butler), the returned value is sometimes ambiguous and is interpreted as a MaskedConstant (by either pybind or astropy). This auto-casts the entire column into a masked column which causes unexpected behavior.
The solution is either:
col.filled(default_value)
and replacing the masked columnThe text was updated successfully, but these errors were encountered: