Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Casacore related version fixes #170

Merged
merged 29 commits into from
Sep 11, 2024
Merged

Casacore related version fixes #170

merged 29 commits into from
Sep 11, 2024

Conversation

tjgalvin
Copy link
Owner

It appears that there are conditions that can (somehow) corrupt large measurement sets when packages across the host virtual environment (python-casacore) and the casacore in the set of singularity containers differ. It is not entirely understood, but the general failure mode is:

  • a large-ish MS (>15GB)
  • a tool using a new (>3.6) version of casacore is used
  • the aoflagger or wsclean containers are then used

The error presents immediately as an error like:

An unhandled exception occurred: FiledesIO::read /path/to/data.ms/table.mf - read returned a bad value

This means very little to me.

The issue seems to be completely avoided by using a casacore that is somewhat consistent between all packages.

No idea what actually goes wrong with the MS, or whether the error is elsewhere. Inspecting the MS through casacore.tables seems to imply things are OK.

Part of this change is to allow pyhton 3.11 as well.

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 33.33333% with 4 lines in your changes missing coverage. Please review.

Project coverage is 59.17%. Comparing base (d33ebd7) to head (3caf9a8).

Files with missing lines Patch % Lines
flint/ms.py 50.00% 2 Missing ⚠️
flint/flagging.py 0.00% 1 Missing ⚠️
flint/selfcal/casa.py 0.00% 1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #170      +/-   ##
==========================================
- Coverage   59.20%   59.17%   -0.03%     
==========================================
  Files          33       33              
  Lines        5530     5536       +6     
==========================================
+ Hits         3274     3276       +2     
- Misses       2256     2260       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tjgalvin tjgalvin merged commit 78b7592 into main Sep 11, 2024
2 checks passed
@tjgalvin tjgalvin mentioned this pull request Sep 11, 2024
@tjgalvin tjgalvin deleted the python3.11 branch September 11, 2024 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants