Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate data into one ephemeral (deltas) and another persisted log (snapshots) #18

Open
adzialocha opened this issue Jan 2, 2025 · 0 comments
Assignees
Labels
backend p2p networking, p2panda, etc. document GObject, Text-CRDT handling, data types, etc.

Comments

@adzialocha
Copy link
Member

adzialocha commented Jan 2, 2025

Automerge gives us both some sort of Operation-Message and Full-State-Messages, that is, interfaces to retrieve both the encoded representations for Operation-based CRDTs or State-based CRDTs (haven't checked the exact internals of Automerge, but I guess that's how it works). Other Text-CRDTs implementations will give us a similar functionality.

To account for the two different natures of these data types we want two separate p2panda append-only logs maintained per document:

  1. "State" log which has a max. depth of 1 (it prunes itself directly on every new appended operation) holding the full State-based-CRDT "snapshot"
  2. "Delta" log which is not persisted (ephemeral) and used to gossip live changes like keystrokes etc. to other peers
@adzialocha adzialocha changed the title Separate data into one ephemeral and another persisted log Separate data into one ephemeral for deltas and another persisted log for snapshots Jan 17, 2025
@adzialocha adzialocha changed the title Separate data into one ephemeral for deltas and another persisted log for snapshots Separate data into one ephemeral (deltas) and another persisted log (snapshots) Jan 17, 2025
@adzialocha adzialocha added the backend p2p networking, p2panda, etc. label Jan 17, 2025
@adzialocha adzialocha added the document GObject, Text-CRDT handling, data types, etc. label Jan 17, 2025
@adzialocha adzialocha self-assigned this Jan 28, 2025
@adzialocha adzialocha added this to the Re-Structuring milestone Jan 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend p2p networking, p2panda, etc. document GObject, Text-CRDT handling, data types, etc.
Projects
None yet
Development

No branches or pull requests

1 participant