Support running kernels in multiple locations (locally or remote) simultaneously.

[ojarjur]

Problem

The Jupyter Server currently only supports having a single place to run kernels; either locally or via a single remote Kernel Gateway server.

This means that a user who wants to switch locations needs to stop their server, reconfigure it, and then restart it.

It would be much better if instead all of the kernelspecs from all of the available locations were presented to the user as a combined set, and then the user could switch between different locations by simply changing kernelspecs.

Proposed Solution

The proposal is to provide extensions of the AsyncMappingKernelManager class and the KernelSpecManager class that provide unified views of the local/remote kernels and kernelspecs, respectively.

Since local and remote kernels use different websocket connections (ZMQ based ones vs. Gateway based ones), we would also need to provide an extension of BaseKernelWebsocketConnection that chooses which of those to use based on the type of kernel (local vs. remote).

The ServerApp could then be configured with this extended mapping kernel manager class for its kernel_manager_class trait, the extended kernelspec manager class for its kernel_spec_manager_class trait, and the extended websocket connection class for its kernel_websocket_connection_class trait.

The session_manager_class would be configured to just be the usual SessionManager class.

The extended classes would each route to the appropriate instance of the existing local or remote class based on whether the corresponding kernelspec was a local or remote one. For listing kernelspecs and kernels, the extended classes would fan out the request to all the corresponding local and remote classes, and then combine the results together into a unified view.

To support configuring of multiple gateway clients, I propose that we add a new base class called something like GatewayClientProvider. This would be called during request handling and return a map from name to a gateway client. Users could then configure multiple clients by providing their own implementation of this class. That also allows the set of clients to change dynamically.

For example, a cloud hosting provider might implement a class that enumerates all of the VMs they have running with Jupyter, and then new gateway clients could appear or disappear as VMs are created or deleted.

Additional context

This was originally proposed in the Jupyter Server repo, but this grew in scope to the point where we need a more in-depth discussion before trying to move forward with the changes.

Both @kevin-bates and @Zsailer expressed interest in this and were heavily involved in discussions about how to do it.

Kevin pointed out that the problem should be generalized to also supporting multiple remote servers side-by-side. However, he also recognized that traitlets does not provide a good way of configuring multiple separate instances of the same class. This meant that there was an open question about how the multiple remote gateway clients can be configured.

In the meantime, I implemented a proof-of-concept version of the routing described above in my fork of the jupyter-server repo.

I then rewrote my proof of concept and released it as an external server extension. That extension implements a very minimal implementation of this proposal. It multiplexes between local and remote kernels, but does not support more than one remote gateway and does not do anything to prevent kernel naming collisions (instead, it just favors the local kernel over a remote one when there is a conflict).