Federated Topology

[CiraciNicolo]

Hi,

I'm having some kind of issues understanding the topology for federated deployment; in my scenario, based on Kubernetes, I've this "runners":

• One GPU instance with a GPU model
• One GPU instance with a different GPU model

This is necessary since we run different models on different hardware.

I would like to instantiate a federated network; currently I'm deploying 3 instances of LocalAI:

• On each GPU instance (launching with --p2p --federated)
• One "master" instance (laucning with federated)

All instances share the same P2P token. The master instance is needed for service/ingress exposition via Kubernetes. I'm seeing the following issues:

• On LocalAI dashboard we see zombie workers
• Sometimes the request is routed to the wrong worker

Am I doing something wrong?