Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement CLUSTER PUBSUB #866

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from
Draft

Implement CLUSTER PUBSUB #866

wants to merge 24 commits into from

Conversation

vazois
Copy link
Contributor

@vazois vazois commented Dec 9, 2024 •
edited
Loading

This PR adds support for PUBSUB in Cluster mode.

  • Implement SPUBLISH and SSUBSCRIBE APIs
  • Add published message forwarding using the active gossip connections
  • Expose parameters to allow configuring max outstanding PubSub forwarding tasks
  • Augment latency and throughput benchmarks with PUBLISH and SPUBLISH support.

@vazois vazois force-pushed the vazois/cluster-pubsub branch 4 times, most recently from 2f789f5 to 422532b Compare December 12, 2024 01:28
@vazois vazois force-pushed the vazois/cluster-pubsub branch 3 times, most recently from 0f02ac5 to 4945338 Compare January 7, 2025 19:20
vazois added 23 commits January 8, 2025 17:02
@vazois
skip Interlocked.Exchange and FlushConfig when no update detected on …
aac302a 
…merge
@vazois
special case of zero epoch necessitates tracking of explicit slot update
49c0edd
@vazois
version-bump-1.0.50
5b84852
@vazois
ensure src is owner
5526ad8
@vazois
update migrateslotwalk test
952bc14
@vazois
move key spec extraction to clusterProvider
9dd01eb
@vazois
implement CLUSTER PUBLISH api
28a8ab0
@vazois
implement SSUBSCRIBE api
4cf9403
@vazois
implement SPUBLISH api
107896f
@vazois
implement separate code path for CLUSTER SPUBLISH
ff4cc03
@vazois
expose max number of outstanding pubsub forwarding tasks parameter
c759378
@vazois
add API to forward published messages using GarnetClient
a5d97bc
@vazois
add methods to extract nodeIds of all nodes in cluster and all nodes …
c3631bb 
…in a shard
@vazois
hook up published message forwarding
7e1cd63
@vazois
PUBLISH and SPUBLISH benchmark addition
c1f1516
@vazois
fix check for getting most recent config instance
27866bf
@vazois
optimization 1: calculate candidates for published message forwarding…
a7fac22 
… only when config gets updated
@vazois
optimization2: ignore response from cluster publish
44f4d60
@vazois
add publish BDN
bf3cb8b
@vazois
minimize task creation
88defb1
@vazois
load no response task
98985fc
@vazois
change to pinned array
7195e07
@vazois
simplify config update for publish forward
e8a1f1a
@vazois vazois force-pushed the vazois/cluster-pubsub branch 4 times, most recently from 05088f5 to b29ef07 Compare January 11, 2025 01:02
@vazois vazois force-pushed the vazois/cluster-pubsub branch from b29ef07 to 27282b1 Compare January 11, 2025 01:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@vazois