24.8.0

What's Changed

• Increase minimum python to 3.9 by @ljgray in #267
• Allow a task to receive the same input for setup and process method by @ljgray in #268
• feat(interferometry): Import from ch_util and driftscan by @ketiltrout in #269
• fix(time): Make Star.names iterable in skyfield_star_from_ra_dec by @ketiltrout in #270
• feat(time): Solar/lunar ephemerides by @ketiltrout in #271
• fix(memh5): fix a bug where shared datasets weren't actually shared by @ljgray in #239
• Convert build to pyproject.toml by @ljgray in #272
• Remove deprecated cached_property in favour of functools.cached_property by @ljgray in #273

Full Changelog: v24.6.0...v24.8.0

v24.6.0

What's Changed

• fix(pyx): ensure distutil directives are read and pin max numpy version by @ljgray in #246
• fix(runner): reset the environment to stop crash at job start by @jrs65 in #248
• fix numpy median by @jrs65 in #249
• Add email notification options for batch jobs by @sjforeman in #250
• Extra ruff rules by @ljgray in #243
• fix: upgrade versioneer and change to non-vendored mode for Python 3.12 by @jrs65 in #257
• parallel load with no attributes by @jrs65 in #253
• Enforce linting rule RUF012 by @ljgray in #259
• Avoid redistribute copy if using a single mpi process by @ljgray in #256
• Runner: module force purge and module use by @ljgray in #255
• Pipeline runner improvements by @ljgray in #258
• feat(pipeline): better handling for key rejection by @ljgray in #265
• Pin numpy < 2 by @ljgray in #266

Full Changelog: v23.6.0...v24.6.0

23.6.0

What's Changed

• perf(moving_weighted_median): transpose 2D array if window size [0] is less than size[1] by @ljgray in #220
• Move cython _invert_no_zero into caput by @ljgray in #221
• Add data directory to gitignore by @rikvl in #228
• fix(mpiarray.MPIArray.gather): Wait for send to complete. by @tristpinsm in #231
• Observer property to get current LSD/CSD by @ljgray in #227
• fix(pipeline): remove erroneous compression options that caused a crash by @jrs65 in #233
• build: Bump minimum supported version to 3.8 by @ljgray in #235
• Reduce memory used by MPIArray.redistribute by @ljgray in #218
• Provide .copy methods for MemDataset and MemDiskGroup by @ljgray in #236
• Allow fileformat to be set by fileformat subclass by @ljgray in #237
• Move invert_no_zero testing from draco repo by @ljgray in #240
• Use ruff for CI linting by @ljgray in #238
• MPIArray reshape fixes by @jrs65 in #241
• Timstamp Alignment (1/4): Add index_attrs property to BasicCont to access attrs of index maps by @ljgray in #234
• feat(mpiarray): helper function to apply a selection to an array by @ljgray in #242

New Contributors

• @rikvl made their first contribution in #228

Full Changelog: v22.10.0...v23.6.0

22.10.0

What's Changed

• fix(time): timescale wasn't returned when pulled from mirror by @jrs65 in #224
• feat(fileformats): make guess_file_formats compatible with pathlib.Pa… by @ljgray in #225

Full Changelog: v22.9.0...v22.10.0

22.9.0

What's Changed

• Skyfield refactor and transit/rise/set time routines by @jrs65 in #146
• fix(transit_times): Handle small intervals better by @ketiltrout in #150
• Remove Python 2 support by @jrs65 in #151
• Pipeline script linting by @nritsche in #147
• Docs build by @nritsche in #140
• Fix sphinx build by @nritsche in #153
• fix(time.Observer.transit_times): Use sidereal day. by @tristpinsm in #154
• Fix issues with line in config by @jrs65 in #158
• Fix linter by @nritsche in #160
• chore(pipeline): add deprecated PipelineConfigError by @nritsche in #156
• fix(scripts.runner): Broken pathlib usage. by @tristpinsm in #161
• CI: Run flake8 and pylint by @nritsche in #157
• fix(OneAndAone): move validation to special function by @nritsche in #163
• Fix pylint to version 2.7.0 and fix new warnings from that version by @nritsche in #166
• fix(runner): change allocation from rpp-krs to rpp-chime by @anjakefala in #170
• build: update versioneer by @anjakefala in #174
• feat(runner): add modules conf variable, specify module environment of a pipeline job by @anjakefala in #168
• feat(memh5): add a __memh5_distributed_axis hint by @ssiegelx in #176
• feat(truncate): a new module for precision truncation of data by @jrs65 in #175
• Allow variable substitution into pipeline scripts by @jrs65 in #172
• fix(runner): need absolute paths in the imports by @jrs65 in #178
• build: build using the oldest supported numpy version by @anjakefala in #182
• Useful enhancements for running the pipeline by @jrs65 in #184
• Merging master into Rick's profiling improvements for caput.pipeline by @anjakefala in #180
• feat(runner): add psutil based profiling to caput-pipeline by @anjakefala in #185
• fix(mpiarray): ensure global_shape for new MPIArray is a tuple by @anjakefala in #187
• Remove Nyquist frequency from PFB by @leungcalvin in #191
• feat(mpiarray.MPIArray): Handle index selections. by @tristpinsm in #192
• feat(runner): add an option to control overwritiing old pipeline runs by @jrs65 in #188
• feat(memh5): zarr support by @nritsche in #169
• feat(mpiarray): ufunc, array_finalize, and getitem handling for MPIArray by @anjakefala in #162
• fix(memh5): ensure contained numpy arrays have a native dtype.byteorder by @anjakefala in #194
• fix(mpiarray): handling Ellipsis and np.newaxis when slicing by @jrs65 in #197
• fix(time): try downloading mirrored data by @jrs65 in #199
• fix(mpiarray): crash testing for an Ellipsis when given an ndarray index by @jrs65 in #200
• fix(tod.concatenate): Convert index map to unicode. by @tristpinsm in #201
• fix(mpiarray): only give warning when compression is disabled by the pipeline by @jrs65 in #202
• HFB container fixes by @jrs65 in #198
• fix(tod): issues copying data from h5py datasets by @jrs65 in #208
• MPIArray cleanups and speedups by @jrs65 in #204
• feat(MPIArray): property for getting global bounds of local array. by @ljgray in #211
• perf(MPIArray): Write redistributed chunks directly into target array by @ljgray in #214
• build(setup.py, .github/workflows/main.yml): un-pin numcodecs version by @sjforeman in #212
• feat: add CFF file for Github/Zenodo by @jrs65 in #215
• fix(memh5): issue with distribution of zarr datasets by @jrs65 in #216
• docs(pipeline): add description of outputs of python-based pipeline by @sjforeman in #219
• feat(memh5): add zipped zarr file support by @jrs65 in #222

New Contributors

• @leungcalvin made their first contribution in #191

Full Changelog: v20.10.0...v22.9.0

20.10.0

Bug Fixes

• config: incorrect exception error message (645b47c)
• memh5: change _make_selections to be a classmethod (9869122)
• memh5: modify BasicCont.redistribute to prevent reference cycles (0b7efaa)
• memh5: pass on detect_subclass and warn when axis selections won't work (f1d2e28)
• misc: change lock_file() behavior to match docstring (5aed55e)
• mpiarray: bug when selecting data during distributed read (3c5392a)
• mpiarray: upper limit incorrectly set in private function _reslice (4e2e1e4)
• setup: load install_requires from the requirements.txt file (9f25a57)
• time: allow zero length arrays as arguments (1a4b324)

Features

• interferometry: add routines use for interferometry (2bf97e1)
• pfb: routines for calculating and correcting the effects of a PFB (a044741)
• pipeline: allow construction and control of pure Python pipelines (81af488)
• runner: support profiling caput-pipeline runs (9b72e7c)

Performance Improvements

• weighted_median: use quickselect to calculate the static median (cfddefb)

20.5.1

Bug Fixes

• memh5: json serialize numpy ndarray (e8156da)

20.5.0

Note: we have switched to calendar versioning for this release.

Bug Fixes

• Fixed circular references in memh5 containers (#60)
• Fixed a race condition when creating output directories (#64)
• Fixed bug in tod.concatenate
• memh5: explicitly set h5py read mode in routines and tests (e5c0016)
• memh5: no comm attribute on group in redistribute (be655b0)
• memh5: allow writing unicode datasets into h5py (25c5b2d)
• memh5: change default convert_attribute_strings for backwards compatibility (bc8261b)
• memh5: comm property was set when distributed=False (a9acd90)
• memh5: detect and convert types annotated with metadata (ef4ebc8)
• memh5: Mapping has moved into collections.abc in Python 3.8 (631412e)
• memh5: serialise datetimes with dict attributes without crashing (5d30194)
• memh5: sort items when copying to/from HDF5 to eliminate MPI errors (7af91fe)
• mpiarray: partition parallel IO to be under the 2GB HDF5 limit (bee591e)
• mpiarray: workaround for h5py collective IO bug (9c68fe3), closes #965
• mpiutil: disable the MPI excepthook (6fba9f6)
• parallel_map: fixed crash (2462a2f)
• pipeline: changed incorrect pro_argspec.keywords to pro_argspec.varkw (906d753), closes #121
• pipeline: use safe_load to avoid warning when loading pipeline conf (8f2d7c3)
• runner: Add call to group at end of file. (e48ef60)
• runner: use copyfile instead of copy (3da9976), closes #102
• time: include required skyfield data in the repo (via Git LFS) (39f357d)
• time: set SkyfieldWrapper default expire=False (#126) (8e84719)
• tod: fixes bug where data variable overwritten (0ed8b2c)

Features

• Context Manager for all memh5 container types (#60). You can now do:

with memh5.BasicCont.from_file("filename.h5"):
   pass

• Support for new CHIME stacked data with reverse_map (#58)
• Better handling of missing config information (#56)
• IO for distributed memh5 containers is now MPI parallel if possible (#66). This should make a big difference to write times of large containers.
• config: add a list Property for validating input lists (b32ad1d)
• BasicCont: make history an h5 attribute (fbc5034)
• caput-pipeline: add option to run job from a temporary dir (90043f5)
• Manager: add metadata options (47432ee), closes #104
• memh5: add compression and chunking of memh5 datasets (c0886c8)
• memh5: sel_* parameters (6743098), closes #108
• misc: context manager for lockfiles (af67279)
• MPIArray: add gather and allgather for collecting the full array (2da95ae)
• pipeline: improve log level config (f418299)
• pipeline: use locking when writing output (2ab01ed)
• setup.py: support building without OpenMP (e478a03)
• tod: allow control of string conversion in tod.concatenate (249face)
• util: moving weighted median (f3e1a2c)
• versioneer: add versioneer for better version naming (fa9cccb)