release/7.3.1

• DeltaDB: Add UNIQUE reduction type (#2641)
• Add worker to blocklist if protocol mismatch (issue2645) #-2648
• eliminate use of python-config #2647
• avoid segfault when cancel_bytag task has no tag #2657
• wq: cancel all tasks in a category, python bindings (#2658)

release/7.3.0

The Cooperative Computing Lab is pleased to announce the release of version 7.3.0 of the Cooperative Computing Tools including Parrot, Chirp, JX, Makeflow, WorkQueue, and other software.

The software may be installed from here: install instructions

This is a minor release with some new features and bug fixes. Among them:

• [WorkQueue] PythonTask to directly execute python functions as WorkQueue tasks. (Barry Sly-Delgado)
• [WorkQueue] Fix max mode allocation to work as a high-water mark when dispatching tasks. (Ben Tovar)
• [WorkQueue] Reworked documentation in https://cctools.readthedocs.io. (Douglas Thain)
• [WorkQueue] API to show summary of workers connected. (David Simonetti)
• [WorkQueue] Adds --wall-time limit to workers. (Thanh Son Phung)
• [Resource Monitor] Time is now reported in seconds, rather than microseconds. (Ben Tovar)
• [JX] jx_repl tool for JX language exploration. (Jack Rundle)

Thanks goes to the contributors for many features, bug fixes, and tests:

• David Rundle
• Barry Sly-Delgado
• Thanh Son Phung
• Tim Shaffer
• David Simonetti
• Douglas Thain
• Ben Tovar

Please send any feedback to the CCTools discussion mailing list:

http://ccl.cse.nd.edu/community/forum

Enjoy!

release/7.3.0RC3

Bump to 7.3.0RC3

release/7.2.15

python_package_run: lock on unpack dir rather than custom file

release/7.2.14

resource_monitor: fix checking core limits at the beginning of the execution.

release/7.2.14RC1

Bump to release/7.2.14

release/7.2.13

• Adds PythonTask to python bindings.
• More efficient work queue submit task.

release/7.2.13RC1

To 7.2.13 (#2621)

* WQ: Add API for Python Function Evaluation #2610

added pytask as well as wrapper script

added Documentation

Update README.md

Update README.md

Updated PyTask and added example script

python result is now method of PyTask. add no result exception class. changed command arguments for task.

added doc for PyTask

added del method

added del condition

 del method changes

* Pytask to PythonTask bindings #2613

move PyTask to PythonTask inside wq bindings

move example to root bindings dir

fix wrapper indent

protect against dill not available

move serialized args to its own function

adds PythonTask.specify_environment

If an environment file is not specified, then the task will run without
python_package_run, and assumes that the worker is running inside a
correct python environment.

document specify_environment in example

document PythonTask in manual

* atexit remove staging directory of PythonTask's (#2615)

* set min workers to max, when min > max (#2616)

Also, error when max < 1

* wq factory: allow abort while sleep (#2618)

This commit checks abort_flag each second the factory sleeps. This
allows a much faster response when using the factory with PythonTask.

The previous behaviour made the factory sleep for a whole period before
checking the abort_flag. Fast executing PythonTask would then have a 30s
overhead when using the 'with workers:' syntax.

* print error messages when setting wq factory (#2619)

* Factory Package Option (#2617)

* Added --python_package to work_queue_factory as an alias for --wrapper "python_package_run X" --wrapper-input X.

* Don't show help on most common error.

* Group options display.

* Cleaned up factory options to be much more readable.

* Update factory manual to match help text.

* Memory leak.

* work queue: improve task submission time (#2620)

* change list_push_priority to O(1)

Before we gave preference to tasks already submitted. The current change
inserts a task as soon as the priority ordering makes sense.

* keep largest resource seen/requested per queue

* update stats in no less time than q->time_last_log_stats

* show overall manually requested in work_queue_status -A

* fix mem leak on queue delete

* Bump to 7.2.13RC1

Co-authored-by: Barry Sly-Delgado <[email protected]>
Co-authored-by: Douglas Thain <[email protected]>

release/7.2.12

Fixes --measure-only in work_queue.

release/7.2.11

• Fix work_queue_hungry.
• Adds --class-ad to condor_submit_workers