Add an option to consider Not Ready but Is Serving Endpoints as not healthy

[hedgieinsocks]

Is your feature request related to a problem?

With the current configuration options, I can't express a case where my Service with externalTrafficPolicy: Local must cease sending traffic to the Node that has only a single Pod replica in Terminating state with the following Endpoint conditions:

"conditions": {
    "ready": false,
    "serving": true,
    "terminating": true
}

In our (simplified for clarity) usage case, I have 2 Nodes that host a haproxy-ingress Deployment with 2 replicas. The podAntiAffinity rule ensures that each Node is allowed to host only 1 replica. And during a deployment rollout, when one of the replicas enters Terminating state (preStop hook with sleep for 30 seconds), I expect the traffic to the Node with this Terminating replica to stop as explained in the k8s docs: https://kubernetes.io/blog/2022/12/30/advancements-in-kubernetes-traffic-engineering/

But since, MetalLB considers endpoints in the aforementioned conditions healthy, the traffic to the Node with the Terminated replica does not cease. And this makes it impossible for us to perform a perfect, seamless rollout.

Describe the solution you'd like

We want to have an option to instruct MetalLB to consider the Endpoints with the following conditions as not healthy:

"conditions": {
    "ready": false,
    "serving": true,
    "terminating": true
}

Additional context

The draft of the suggested changes to implement the feature is in the following PR: #2578

I've read and agree with the following

• I've checked all open and closed issues and my request is not there.
• I've checked all open and closed pull requests and my request is not there.