New Broker WebSocket Python bindings

[ckreibich]

We'd like to retire the existing Python bindings for Broker and replace them with a more lightweight implementation that uses the WebSocket transport. Benefits include:

• The new bindings can be a standalone Python package, installable via pip
• We no longer need a Broker .so in every component that wants to use the bindings
• It should make it easier to avoid pitfalls in data type incompatibilities, for example in terms of hashable types.

The main use case for the bindings is easy Python-level interaction with Zeek events, both to receive and transmit. The goal is not to provide a direct API to Broker's WebSocket data model, but to provide an API for event pub/sub. The data model and its types can of course be accessible to the user if that's useful.

It may be desirable to add an RPC-style abstraction that captures the fact that transmission of an event type X will usually be responded to with an instance of another event type Y, since that's a common pattern in existing uses of the Python bindings.

The zeek-client codebase currently has a prototypical internal implementation of these bindings and will become an early adopter once the standalone package takes shape. In particular:

• It has a Python module that implements Broker's data model.
• It uses the websocket-client socket abstraction. There are (at least) two existing packages to choose from: the former, and the async-based websockets package.
• It has an event registry concept that I'd also consider prototypical — entirely possible that there's something more Pythonic out there.
• It has a TLS abstraction to provide variations on encrypted transports.
• It does not have a real event loop, but integration with existing event loops would be a nice feature for the new bindings. These are usually based on async so that's something to keep in mind.

A topic that's come up repeatedly for the bindings is resilience to changes in the data format. In the past we exposed users of the old bindings to incompatibilities in the event format, to changes in the layout of record types, etc. We have a couple of options here — since the data format itself is not relevant to the API of the package, we can commit to recognizing and abstracting away such incompatibilities under the hood (as users had to in the past), or we could extend the websocket handshake to convey versioning information (for Zeek itself, for the application being built with bindings, etc). I'd suggest the former to start with, and keeping the latter in mind for future work.

This will be a separate repo, but I'm keeping it in the zeek/broker project for the time being.

This was originally slated for 6.1 but slipped. Relates to #303.