Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provide a way to detect and clean up jobs which were started but never finished #91

Open
robacarp opened this issue Apr 13, 2022 · 1 comment
Open

Provide a way to detect and clean up jobs which were started but never finished #91

robacarp opened this issue Apr 13, 2022 · 1 comment

Comments

@robacarp
Copy link
Collaborator

robacarp commented Apr 13, 2022 •
edited
Loading

This happens when a long-running job is interrupted (ctrl-c style), or a job crashes so badly that mosquito itself crashes (a crystal runtime bug).

image

image

related to #21
@robacarp robacarp self-assigned this Apr 13, 2022
@robacarp robacarp removed their assignment Oct 4, 2023
@robacarp
Copy link
Collaborator Author

robacarp commented Oct 4, 2023

With the addition of metric check-ins in #118, the back-end has knowledge of the list of workers currently running. If the job run metadata is updated with the id of the worker when the job starts, later workers can easily check to see that worker has expired and emit a warning and reap the keys.

@robacarp robacarp mentioned this issue Jan 6, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@robacarp