Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broadcast server awareness to all clients #73

Merged
merged 22 commits into from
Oct 9, 2024

Conversation

brichet
Copy link
Contributor

@brichet brichet commented Oct 1, 2024

This PR remove the Awareness to use the one from pycrdt (jupyter-server/pycrdt#170)

This PR add broadcasting changes of state in the local awareness.

Draft until jupyter-server/pycrdt#170 is merged and released.

@brichet brichet marked this pull request as draft October 1, 2024 13:12
@brichet brichet marked this pull request as ready for review October 1, 2024 20:01
pycrdt_websocket/awareness.py Outdated Show resolved Hide resolved
pycrdt_websocket/awareness.py Outdated Show resolved Hide resolved
pycrdt_websocket/awareness.py Outdated Show resolved Hide resolved
@brichet
Copy link
Contributor Author

brichet commented Oct 3, 2024

Thanks @davidbrochart for the review and live debugging.

As suggested in jupyter-server/jupyter_ydoc#277 (comment) I moved the Awareness to pycrdt in jupyter-server/pycrdt#170.
So I remove it in this PR, but I can also split it in 2 different PRs, (1) remove Awareness and depend on pycrdt's one and (2) make use of Awareness in the YRoom.

pyproject.toml Outdated Show resolved Hide resolved
pycrdt_websocket/yroom.py Outdated Show resolved Hide resolved
pycrdt_websocket/yroom.py Outdated Show resolved Hide resolved
pycrdt_websocket/yroom.py Show resolved Hide resolved
@brichet brichet marked this pull request as draft October 3, 2024 12:52
pycrdt_websocket/yroom.py Outdated Show resolved Hide resolved
@davidbrochart davidbrochart added the enhancement New feature or request label Oct 3, 2024
@brichet brichet changed the title Use the awareness from the server Remove the Awareness in favor of the one from pycrdt Oct 3, 2024
@brichet brichet changed the title Remove the Awareness in favor of the one from pycrdt Broadcast server awareness to all clients Oct 4, 2024
@brichet brichet marked this pull request as ready for review October 4, 2024 08:30
@brichet brichet marked this pull request as draft October 4, 2024 08:32
@brichet
Copy link
Contributor Author

brichet commented Oct 4, 2024

Finally that's a lot of commits and comments for minimal changes 😄

"""
Callback to broadcast the server awareness to clients.
"""
if not changes[1] == "local":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if not changes[1] == "local":
if changes[1] != "local":

self.ready_event = Event()
self.ready = ready
self.ystore = ystore
self.log = log or getLogger(__name__)
self.awareness = Awareness(self.ydoc)
self.awareness.observe(self.local_update_awareness)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about the callback name local_update_awareness, maybe send_server_awareness?

Comment on lines 318 to 321
updated_clients = [
*changes[0].get("added", []),
*changes[0].get("filtered_updated", []),
]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes consist of "added", "updated" and "removed" entries only, not "filtered_updates". Should we include "removed"?

Suggested change
updated_clients = [
*changes[0].get("added", []),
*changes[0].get("filtered_updated", []),
]
updated_clients = [v for value in changes[0].values() for v in value]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also I added a test to broadcast only if there is an effective change (not 'update').

@brichet brichet marked this pull request as ready for review October 9, 2024 11:56
pycrdt_websocket/yroom.py Outdated Show resolved Hide resolved
pycrdt_websocket/yroom.py Outdated Show resolved Hide resolved
pycrdt_websocket/yroom.py Show resolved Hide resolved
"""
Callback to broadcast the server awareness to clients.
"""
if type != "change" or changes[1] != "local":
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we actually broadcast every update?

Suggested change
if type != "change" or changes[1] != "local":
if type != "update" or changes[1] != "local":

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your right, we should broadcast all events.
Currently the update event is triggered only when the change event is triggered.
Should we trigger an update every X sec ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need this logic in pycrdt. I'll work on it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's what I meant.

brichet and others added 2 commits October 9, 2024 14:40
Co-authored-by: David Brochart <[email protected]>
pyproject.toml Outdated
@@ -30,7 +30,7 @@ classifiers = [
dependencies = [
"anyio >=3.6.2,<5",
"sqlite-anyio >=0.2.3,<0.3.0",
"pycrdt >=0.9.16,<0.10.0",
"pycrdt >=0.10.0,<0.11.0",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's take the latest bugfix.
I'm also wondering if we should wait for the implementation of periodic awareness updates in pycrdt?

Suggested change
"pycrdt >=0.10.0,<0.11.0",
"pycrdt >=0.10.1,<0.11.0",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it is required for this change.
But we should probably remove the type != "update" condition to broadcast all the event.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But an "update" change is sent for all events, right? So we should keep it, to deduplicate "change" events. Or am I missing something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, sorry again for the confusion. The change event is only for observers, but not for clients...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll merge then.

Co-authored-by: David Brochart <[email protected]>
Copy link
Collaborator

@davidbrochart davidbrochart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @brichet!

@davidbrochart davidbrochart merged commit 826446f into jupyter-server:main Oct 9, 2024
7 checks passed
@brichet brichet deleted the server_awareness branch October 9, 2024 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants