Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spawnv_passfds() doesn't work on Python 3.9 #309

Closed
vstinner opened this issue Mar 3, 2020 · 7 comments
Closed

spawnv_passfds() doesn't work on Python 3.9 #309

vstinner opened this issue Mar 3, 2020 · 7 comments

Comments

@vstinner
Copy link
Contributor

vstinner commented Mar 3, 2020

spawnv_passfds() function is implemented in billiard/compat.py. On Pytho 3, it calls a private Python function: _posixsubprocess.fork_exec(). The problem is that the API of this function changed in Python 3.9, it got new required parameter.

umask:

user, group, extra_groups:

You can pass default values:

  • umask=-1
  • user=None
  • group=None
  • extra_groups=None

Something like (you may use a different name than args to avoid reusing args name twice):

args = [
    args, [fsencode(path)], True, tuple(passfds), None, None,
    -1, -1, -1, -1, -1, -1, errpipe_read, errpipe_write,
    False, False, None]
if sys.version_info >= (3, 9):
    args.extend((None, None, None, -1))  # group, extra_groups, user, umask
args.append(None)  # preexec_fn
return _posixsubprocess.fork_exec(*args)
@vstinner
Copy link
Contributor Author

vstinner commented Mar 3, 2020

Downstream Fedora issue with Python 3.9: https://bugzilla.redhat.com/show_bug.cgi?id=1792056

@auvipy
Copy link
Member

auvipy commented Mar 4, 2020

ops! thanks for the report! I was planning to update the multiprocessing dependencies of billiard #292 , would you mind sharing your insight about using which version of multiprocessing codes should billiard focused on? python 2.7 or from python3.9/master?

@vstinner
Copy link
Contributor Author

vstinner commented Mar 4, 2020

My patch doesn't touch the Python 2 support which is kept: it only adds support for Python 3.9. It's up to you to continue to support Python 2 or not, but it's no longer supported upstream (will no longer get security fixes).

@auvipy
Copy link
Member

auvipy commented Mar 12, 2020

billiard will get py27 support till Dec 2020[PyPY] and many old enterprise users. and for celery 5 we will not be using billiard. celery 4.x will be python2 compatible with billiard and amqp 1.0 support. celery 5will be based on trio and amqp 1.0 and python 3.6+ only.

@auvipy
Copy link
Member

auvipy commented Mar 12, 2020

btw patch welcome.

@vstinner
Copy link
Contributor Author

btw patch welcome.

I converted my inline patch into a concrete PR: PR #310.

auvipy pushed a commit that referenced this issue May 22, 2020
Python 3.9 added (umask, user, group, extra_groups) parameters to
_posixsubprocess.fork_exec().
@vstinner
Copy link
Contributor Author

vstinner commented Sep 6, 2024

Fixed by #310

@vstinner vstinner closed this as completed Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants