Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add user-pluggable block error handling and a new sliding-window error handler #2858

Merged
merged 9 commits into from
Aug 12, 2023
Merged

Add user-pluggable block error handling and a new sliding-window error handler #2858

merged 9 commits into from
Aug 12, 2023

Commits on Aug 10, 2023

  1. Update BlockProviderExecutor to take a callback for block_error_handler 

    * For backward compatibility the block_error_handler is disabled if set to False
* A new block_error_threshold is added for easier user configurability
* Fixed broken logic in handle_errors
    @yadudoc
    yadudoc committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    9b87df7 View commit details
    Browse the repository at this point in the history

  2. New windowed_error_handler to better handle long running workflows 

    * Windowed_error_handler shutsdown the executor IFF the last N jobs all failed where N is configured by block_error_threshold
    @yadudoc
    yadudoc committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    329950b View commit details
    Browse the repository at this point in the history

  3. * Update executor to work with changes to BlockProviderExecutor 

    * Added tests
    @yadudoc
    yadudoc committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    52386a6 View commit details
    Browse the repository at this point in the history

  4. Removing block_error_threshold 

    * Minor mypy fix

`block_error_threshold` is only used by the error_handler, so it is removed from the Executor definitions
    @yadudoc
    yadudoc committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    d20ee05 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. Several fixes from Ben's review 

    * Type cleanups
* Updating tests
* Fixing a sorting error
* Adding a `noop_error_handler`
* Minor fixes to simple_error_handler to match previous logic
    @yadudoc
    yadudoc committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    1e78474 View commit details
    Browse the repository at this point in the history

  2. Renamed parsl.jobs.simple_error_handler to parsl.jobs.error_handlers

    @yadudoc
    yadudoc committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    0216f44 View commit details
    Browse the repository at this point in the history

  3. Remove obsolete mock in test

    @yadudoc
    yadudoc committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    a2bcbc4 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into fix_scale_out

    @yadudoc
    yadudoc authored Aug 11, 2023
    Configuration menu
    Copy the full SHA
    4fad391 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2023

  1. Merge branch 'master' into fix_scale_out

    @benclifford
    benclifford authored Aug 12, 2023
    Configuration menu
    Copy the full SHA
    bc70e87 View commit details
    Browse the repository at this point in the history