Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to dill serializer #406

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add support to dill serializer #406

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c7ebbc0
Fix #405
Jun 26, 2024
2dc2441
Adding dill in requirements
Jul 1, 2024
ab98e7c
Adding dill-specific tests
Jul 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions billiard/common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import signal
import sys

import dill
import pickle

from .exceptions import RestartFreqExceeded
Expand All @@ -14,6 +15,9 @@
pickle_load = pickle.load
pickle_loads = pickle.loads

dill_load = dill.load
dill_loads = dill.loads

# cPickle.loads does not support buffer() objects,
# but we can just create a StringIO and use load.
from io import BytesIO
Expand Down
6 changes: 3 additions & 3 deletions billiard/pool.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
from . import cpu_count, get_context
from . import util
from .common import (
TERM_SIGNAL, human_status, pickle_loads, reset_signals, restart_state,
TERM_SIGNAL, human_status, dill_loads, reset_signals, restart_state,
)
from .compat import get_errno, mem_rss, send_offset
from .einfo import ExceptionInfo
Expand Down Expand Up @@ -441,7 +441,7 @@ def _make_recv_method(self, conn):
if hasattr(conn, 'get_payload') and conn.get_payload:
get_payload = conn.get_payload

def _recv(timeout, loads=pickle_loads):
def _recv(timeout, loads=dill_loads):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please update or add some tests to check the dill usage?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you specify what sort of test you would like added? Pool instanciation is already tested, as well as messages sending and receiving (which should handle the tests on whether this would introduce side effects or not). I have added some tests following this review, but I don't know if they meet your expectations.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this resolved?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can I resolve this conversation?

return True, loads(get_payload())
else:
def _recv(timeout): # noqa
Expand All @@ -456,7 +456,7 @@ def _recv(timeout): # noqa
return False, None
return _recv

def _make_child_methods(self, loads=pickle_loads):
def _make_child_methods(self, loads=dill_loads):
self.wait_for_job = self._make_protected_receive(self.inq)
self.wait_for_syn = (self._make_protected_receive(self.synq)
if self.synq else None)
Expand Down
Loading