[not for merge] DESC release branch #2012
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Without this, the tasks continue to run while subsequent local tests happen. This is at the least confusing in logs - I haven't checked if it introduces any actual testing problems.
The full repr for an executor is usually very long and includes all
of the configuration information for that exception, so previously
a BadStateException looked like this, which is awkwardly large:
parsl.executors.errors.BadStateException: Executor HighThroughputExecutor(
address=None,
address_probe_timeout=None,
cores_per_worker=1,
cpu_affinity='none',
heartbeat_period=30,
heartbeat_threshold=120,
interchange_port_range=(55000, 56000),
label='htex_local',
launch_cmd='executable_that_hopefully_does_not_exist_1030509.py',
managed=True,
max_workers=1,
mem_per_worker=None,
poll_period=1,
prefetch_capacity=0,
provider=LocalProvider(
channel=LocalChannel(
envs={},
script_dir='/home/runner/work/parsl/parsl/runinfo/030/submit_scripts',
userhome='/home/runner/work/parsl/parsl'
),
cmd_timeout=30,
init_blocks=1,
launcher=SimpleLauncher(debug=True),
max_blocks=1,
min_blocks=0,
move_files=None,
nodes_per_block=1,
parallelism=1,
worker_init=''
),
storage_access=None,
worker_debug=True,
worker_logdir_root=None,
worker_port_range=(54000, 55000),
worker_ports=None,
working_dir=None
) failed due to: STDERR: /home/runner/work/parsl/parsl/runinfo/030/submit_scripts/parsl.localprovider.1642801165.5067718.sh: line 3: executable_that_hopefully_does_not_exist_1030509.py: command not found
After this commit, the exception instead looks like this:
parsl.executors.errors.BadStateException: Executor htex_local failed due to: STDERR: /home/benc/parsl/src/parsl/runinfo/000/submit_scripts/parsl.localprovider.1643124620.9775066.sh: line 3: executable_that_hopefully_does_not_exist_1030509.py: command not found
…having shutdown debugging
…mentary and unused parameters
Without this, the interchange process continues running for the duration of the pytest lifetime, overlapping with other tests. This makes CI hangs harder to debug.
…led test suite rather than a hang" This reverts commit 1b4adc5. This commit was causing a different shutdown problem in normal htex operation, so reverting while I debug
…onitoring name tests
… values, rather than repr of unresolved app futures, are recorded in the database
… a node and store results at the end
This gave measurable performance improvement under very high load/shared fs situations, but it is a lot of additional complexity in the code to achieve that. Reducing import cost is probably a good goal, but this way is probably not the way to do it.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This branch is ongoing with various DESC users for using parsl with the LSST DESC workflows. This PR is to get it tested in parsl CI.
This branch is made from an stgit patch stack which lives on @benclifford 's laptop, with regular merges from master. Don't push changes to this branch - rather, get them put on master and poke @benclifford for an update.
I'm migrating patches from this stack over the years onto master and adding new stuff on the end - hopefully eventually the patch stack, and this branch with it, should evaporate entirely.