swim-protocol

SWIM protocol implementation for exchanging cluster membership status and metadata.

This library is intended to fit into an asyncio event loop to help synchronize a distributed group of processes.

Introduction

API Documentation

Install and Usage

$ pip install swim-protocol

File Sync Tool

A basic tool for reporting cluster membership and synchronizing metadata as files is provided:

$ swim-protocol-sync --name 127.0.0.1:2001 --peer 127.0.0.1:2002 ~/node1
$ swim-protocol-sync --name 127.0.0.1:2002 --peer 127.0.0.1:2001 ~/node2

While running, the state of the cluster and the metadata of each member are visible on the filesystem.

$ tree -a ~/node1
node1
├── .available
│   └── 127.0.0.1:2002 -> ../127.0.0.1:2002
├── .local -> 127.0.0.1:2001
├── .offline
├── .online
│   └── 127.0.0.1:2002 -> ../127.0.0.1:2002
├── .suspect
├── .unavailable
├── 127.0.0.1:2001
│   └── file-one.txt
└── 127.0.0.1:2002
    └── file-two.txt

To change the metadata of the local cluster member, edit the files and issue a SIGHUP to the process:

$ vim ~/node1/.local/file-one.txt
$ pkill -HUP -f swim-protocol-sync

Running the Demo

There is a demo application included as a reference implementation. Try it out by running the following, each from a new terminal window, and use Ctrl-C to exit:

$ swim-protocol-demo --name 127.0.0.1:2001 --peer 127.0.0.1:2003
$ swim-protocol-demo --name 127.0.0.1:2002 --peer 127.0.0.1:2001
$ swim-protocol-demo --name 127.0.0.1:2003 --peer 127.0.0.1:2001
$ swim-protocol-demo --name 127.0.0.1:2004 --peer 127.0.0.1:2003

Typing in any window will disseminate what has been typed across the cluster with eventual consistency.

Getting Started

First you should create a new UdpConfig object:

from swimprotocol.udp import UdpConfig

config = UdpConfig(local_name='127.0.0.1:2001',
                   local_metadata={'name': b'one'},
                   peers=['127.0.0.1:2002'],
                   secret='my secret')

All other config arguments have default values, which are tuned somewhat arbitrarily with a small cluster of 3-4 members in mind.

Now you can create the cluster members manager and transport layer, and enter the event loop:

from contextlib import AsyncExitStack
from swimprotocol.members import Member, Members
from swimprotocol.udp import UdpTransport
from swimprotocol.worker import Worker

members = Members(config)
worker = Worker(config, members)
transport = UdpTransport(config, worker)

async def run() -> None:
    async with AsyncExitStack() as stack:
        await stack.enter_async_context(transport)
        await stack.enter_async_context(worker)
        await stack.enter_async_context(
            members.listener.on_notify(on_member_change))
        await ...  # run your application

async def on_member_change(member: Member) -> None:
    ...  # handle a change in member status or metadata

These snippets demonstrate the UDP transport layer directly. For a more generic approach that uses argparse and load_transport, check out the demo or the sync tool.

If your application is deployed as a Docker Service, the UdpConfig discovery=True keyword argument can be used to discover configuration based on the service name. See the documentation for more comprehensive usage.

Checking Members

The Members object provides a few ways to check on the cluster and its members:

for member in members.non_local:
    # all other known cluster members
    print(member.name, member.status, member.metadata)

from swimprotocol.status import Status
for member in members.get_status(Status.AVAILABLE):
    # all cluster members except offline
    print(member.name, member.status, member.metadata)

Alternatively, listen for status or metadata changes on all members:

from swimprotocol.member import Member

async def _updated(member: Member) -> None:
    print('updated:', member.name, member.status, member.metadata)

async with AsyncExitStack() as stack:
    # ...
    stack.enter_context(members.listener.on_notify(_updated))

UDP Transport Security

The UdpTransport transport layer (the only included transport implementation) uses salted hmac digests to sign each UDP packet payload. Any UDP packets received that are malformed or have an invalid signature are silently ignored. The eventual consistency model should recover from packet loss.

The signatures rely on a shared secret between all cluster members, given as the secret=b'...' argument to the UdpConfig constructor. If secret=None is used, it defaults to uuid.getnode() but this is not secure for production setups unless all sockets are bound to a local loopback interface.

The cluster member metadata is not encrypted during transmission, so only private networks should be used if metadata includes any secret data, or that secret data should be encrypted separately by the application.

If member metadata is larger than can be transmitted in a single UDP packet (hard-coded at 1500 bytes due to MTU sizes on public networks), a TCP connection is used instead. There is no additional protocol for TCP; the connection is opened, the oversized packet is transmitted, and then the connection is closed without waiting for a response.

Development

You will need to do some additional setup to develop and test plugins. Install Hatch to use the CLI examples below.

Run all tests and linters:

$ hatch run check

Because this project supports several versions of Python, you can use the following to run the checks on all versions:

$ hatch run all:check

Type Hinting

This project makes heavy use of Python's type hinting system, with the intention of a clean run of mypy:

$ mypy

No code contribution will be accepted unless it makes every effort to use type hinting to the extent possible and common in the rest of the codebase.