feat(sdk): Event cache experimental store: LinkedChunk

[Hywan]

This PR is a work-in-progress. It explores an experimental data structure to store events in an efficient way.

Note: in this comment, I will use the term store to mean database or storage.

The biggest constraint is the following: events can be ordered in multiple ways, either topological order, or sync order. The problem is that, when syncing events (with /sync), or when fetching events (with /messages), we don't know how to order the newly received events compared to the already downloaded events. A reconciliation algorithm must be written (see #3058). However, from the “storage” point of view, events must be read, written and re-ordered efficiently.

Ordering index

The simplest approach would be to use an order_index for example. Every time a new event is inserted, it uses the position of the last event, increments it by one, and done.

However, inserting a new event in the middle of existing events would shift all events on one side of the insertion point: given a, b, c, d, e, f with f being the most recent event, if g needs to be inserted between b and c, then c, d, e, f's ordering positions need to be shifted. That's not optimal at all as it would imply a lot of updates in the store.

Example of a relational database:

ordering_index	event
0	a
1	b
2	g
3	c
…	…

An insertion can be O(n), and it can happen more frequently than one can think of. Let's imagine a permalink to an old message: the user opens it, a couple of events are fetched (with /messages), and these events must be inserted in the store, thus potentially shifting a lot of existing events. Another example: Imagine the SDK has a search API for events; as long as no search result is found, the SDK will back-paginate until reaching the beginning of the room; every time there is a back-pagination, a block of events will be inserted: there is more and more events to shift at each back-pagination.

Linked list

OK, let's forget the order_index. Let's use a linked list then? Each event has a link to the previous and to the next event.

Inserting an event would be at worst O(3) in this case: if the previous event exists, it must be updated, if the next event exists, it must be updated, finally, insert the new event.

Example with a relational database:

previous	id	event	next
null	id(a)	a	id(b)
id(a)	id(b)	b	id(c)
id(b)	id(c)	c	null

This approach ensures a fast writing, but a terribly slow reading. Indeed, reading N events require N queries in the store. Events aren't contiguous in the store, and cannot be ordered by the database engine (e.g. with ORDER BY for SQL-based database). So it really requires one query per event. That's a no-go.

What about gap?

In the two scenarios above, another problem arises. How to represent a gap? Indeed, when new events are synced (via /sync), sometimes the response contains a limited flag, which means that the results are partial.

Let's take the following example: the store contains a, b, c. After a long offline period (during which the room has been pretty active), a sync is started, which provides the following events: x, y, z + the limited flag. The app is killed and reopened later. The event cache store will contain a, b, c, x, y, z. How do we know that there is a hole/a gap between c and x? This is an important information! When z, y and x are displayed, and the user would like to scroll up, the SDK must know that it must back-paginate before providing c, b and a.

So the data structure we use must also represent gaps. This information is also crucial for the events reconciliation algorithm.

Proposal

What about a mix between the two? Here is Linked Chunk.

A linked chunk is like a linked list, except that each node is either a Gap or an Items. A Gap contains nothing, it's just a gap. An Items contains several events. A node is called a Chunk. A chunk has a maximum size, which is called a capacity. When a chunk is full, a new chunk is created and linked appropriately. Inside a chunk, an ordering index is used to order events. At this point, it becomes a trade-off the find the appropriate chunk size to balance the performance between reading and writing. Nonetheless, if the chunk size is 50, then reading events is 50 times more efficient with a linked chunk than with a linked list, and writing events is at worst O(49), compare to the O(n - 1) of the ordering index.

Example with a relational database. First table is events, second table is chunks.

chunk id	index	event
$0	0	a
$0	1	b
$0	2	c
$0	3	d
$2	0	e
$2	1	f
$2	2	g
$2	3	h

chunk id	type	previous	next
$0	items	null	$1
$1	gap	$0	$2
$2	items	$1	null

Reading the last chunk consists of reading all events where the chunk_id is $2 for example, and contains events e, f, g and h. We can sort them easily by using the event_index column. The previous chunk is a gap. The previous chunk contains events a, b, c and d.

Being able to read events by chunk clearly limit the amount of reading and writing in the store. It is also close to what will be really done in real life with this store. It also allows to represent gaps. We can replace a gap by new chunk pretty easily with few writings.

A summary:

Data structure	Reading	Writing
Ordering index	“O(1)”1 (fast)	O(n - 1) (slow)
Linked list	O(n) (slow)	O(3) (fast)
Linked chunk	O(n / capacity)	O(capacity - 1)

Implementation

This PR contains a draft implementation of a linked chunk. It will strictly only contain the required API for the EventCache, understand it is not designed as a generic data structure type.

---

• Address [meta] Event Cache API 🕸 #3058

Footnotes

1. O(1) because it's simply one query to run; the database engine does the sorting for us in a very efficient way, particularly if the ordering_index is an unsigned integer. ↩

[bnjbvr]

Great stuff! A first round of comments here. I'm mostly worried about the naming conventions for the iterators, as I'm 100% sure I'm going to get them wrong; let's see if the renaming proposal below makes sense. Thanks for all the tests <3

[bnjbvr]

Thanks for renaming the iterators 🫶

LGTM, let's ship this so we can move foreward forward with the integration!

[bnjbvr]

(Nothing to action here, just a FYI: note we may need something slightly different, since the back-pagination replaces a gap with the events returned by the back-pagination and a previous gap, if there was another previous-token in the back-pagination response.)

[Hywan]

Well, we can use replace_gap_at + insert_gap_at then?

[codecov]

Codecov Report

Attention: Patch coverage is 77.81570% with 65 lines in your changes are missing coverage. Please review.

Project coverage is 83.69%. Comparing base (7718f90) to head (e8cf6dc).
Report is 2 commits behind head on main.

Files	Patch %	Lines
crates/matrix-sdk/src/event_cache/linked_chunk.rs	86.36%	36 Missing ⚠️
crates/matrix-sdk/src/event_cache/store.rs	0.00%	29 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3166      +/-   ##
==========================================
- Coverage   83.76%   83.69%   -0.08%     
==========================================
  Files         234      235       +1     
  Lines       24174    24467     +293     
==========================================
+ Hits        20249    20477     +228     
- Misses       3925     3990      +65     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.