-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FileSystem calls over Atomics.wait instead of service worker #87
Conversation
I'll turn this into draft and leave it open for discussions. But I'll open the proper PR upstream in JupyterLite to introduce the needed APIs for this to work nicely. |
Nice, thanks @martinRenou for looking into this! |
938b873
to
edb8d3b
Compare
edb8d3b
to
4661f9d
Compare
This makes sure to have a working setup with jupyterlite<0.4.0 when using service worker
24c9cd7
to
e62a784
Compare
We should probably document how to enable this. It could be here in the |
We should definitely look at enabling this for jupyterlite-pyodide too :) Agreed we should add it to the docs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested locally with and without the COOP / COEP headers to check it falls back to using the Service Worker 👍
Thanks for working on this!
@martinRenou should this be released in a |
This should be backward compatible if not setting If people were already setting So I believe this is backward incompatible. |
I believe we can't make it backward compatible though, so we should probably make an alpha for |
Yeah I was thinking a Starting a |
Started to document this in jupyterlite/jupyterlite#1405 |
I can have a look at implementing the shared array buffers approach for the pyodide kernel. |
That would be great, thanks! Hopefully the approach will be similar. |
In combination with jupyterlite/jupyterlite#1383
This PR provides an implementation of the Emscripten FileSystem that communicates using Atomics.wait and SharedArrayBuffer instead of the blocking HTTP requests to a service worker.
I open a PR here in jupyterlite/xeus for convenience, but part of the code here should probably go upstream in jupyterlite.Done in jupyterlite/jupyterlite#1383How to test it:
npx static-handler --cors --coop --coep --corp ./
Implementation details
Instead of directly using SharedArrayBuffers and Atomic.waits (which would require defining a proper communication protocol), I decided to go with using https://github.com/WebReflection/coincident which provides high level APIs for making blocking communication between worker <-> main thread.
Coincident comes as a replacement for comlink which we used for wrapping the worker kernel communication.