[17.0][FIX] Refactor stop signal handling in QueueJobRunner #647
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit refactors the stop signal handling in the `QueueJobRunner` class. Instead of using a pipe, it now uses a socket pair to initialize the stop signals. This change improves the code readability and ensures compatibility with Windows by manually creating a socket pair when the `socketpair` function is not available.
|
Hi @guewen, |
Maralai
changed the title
|
There hasn't been any activity on this pull request in the past 4 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days. |
github-actions
bot
added
the
stale
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit addresses the issue: #564.
Refactor QueueJobRunner to Enhance Performance, Signal Handling, and Stability
This pull request brings substantial improvements and new functionalities to the
QueueJobRunner, which is focused on optimizing performance, signal handling, and stability. The significant changes include the transition to asynchronous HTTP requests, dynamic detection of new databases, and enhancements for robust signal handling.Key Changes:
Async HTTP Requests:
asyncioandaiohttpfor handling HTTP job runs asynchronously. This change enhances concurrency and network-bound operation efficiency.Socket Pair for Signal Handling:
Selectors for Notification Handling:
selectors.DefaultSelector()for handling database and stop signal notifications, improving efficiency and scalability of notification handling.Auto-Detection of New Databases:
queue_jobmodule installed, allowing dynamic adaptation without server restart.Configurable Check Interval:
Database Keep-Alive Mechanism:
SELECT 1queries to keep database connections alive and prevent intermediaries from closing idle connections.Please review the changes and provide feedback or approval for merging this enhanced version of the
QueueJobRunner. These improvements address the aforementioned issue and the todo's included within the original file, increasing robustness, scalability, and overall performance. I have included test cases and attempted to comply with pre-commit tests, with the aim of achieving a green checkmark.Thank you for your consideration.