Multi deployment: Blocks on first replica if other replica fails to start

[flouthoc]

Hi Maintainers,

I found this one issue and just leaving it here as a note for future developers and if someone else stumbles upon this. Please free to close this

When doing waver multi deploy , the entire deployment blocks on the first replica if second replica is on a blocking state due to any error and logs from second replica are not propagated to stdout , making it hard to debug since as a developer I am stuck with a blocked deployment and no logs from my failing replicas.

I am not sure but maybe a potential solution could be to always collect and report logs from all replicas in a polling manner ?

[rgrandl]

@flouthoc, can you elaborate on this? You deployed an app with weaver multi. However, one of the replica is not able to start, hence nothing works?

Can you show an example app that triggers this issue?

[flouthoc]

@rgrandl My current application is in a bit complicated state to open-source, but sure I can try creating a minimal reproducer and report back here.

Once the issue is acknowledged as generally reproduce-able I can take a stab at fixing the issue as well.