Returns are not delivered always before ACKs (but they should)

[lowmanb]

I'd like to bump this issue raised in streadway/amqp:

streadway/amqp#351

It's unfortunate that it is not possible to determine if a specific message was/wasn't returned. Is there a workaround that I am missing?

[michaelklishin]

Clients are generally not supposed to deal with returns often. Tying process of returns to that of acks [publisher confirms] is therefore usually unnecessary.

Just like in other clients, if a basic.return frame is received before a basic.ack, it would be dispatched first. However, how exactly
both are exposed to the user varies greatly between clients. There is no such thing as a "generic client" which you are referring to.

In most clients, e.g. Java, .NET, Ruby, Python ones, these events result in user-provided function calls on the dispatching thread.
In this client, many operations are Go channel operations. Since separate Go channels are used for different server-sent methods,
you have a race condition and a general concurrent code coordination problem if you expect to process returns before acks.

I don't see a solution to this without drastically changing the API of this client to use Go channels less. I strongly disagree that
such change would be justified.

Instead, deal with acks first, and if a returned message comes around, re-add ("requeue") it for re-publishing, log it, or do whatever
you need to do with returns. Treat returns as a completely separate, rare occurrence, and publisher confirms as the main indicator
of successful publishing. Also, if you deal with returns often, perhaps your topology is a tad too dynamic.

[lowmanb]

Thanks for the discussion.

I don't see a solution to this without drastically changing the API of this client to use Go channels less. I strongly disagree that
such change would be justified.

I think this sums it up (and don't disagree). Perhaps it's worth documenting how the library design (cleanly implemented with go channels) is fundamentally incompatible with this protocol feature?

[michaelklishin]

It's not necessarily incompatible, it just puts all the responsibilities for tracking both returns and confirms on the application developer.

If we have docs on publisher confirms, I'd be happy to add a note. Can you point me at where this belongs (if anywhere)?

[lowmanb]

As is, it's not possible to leverage the ordering of returns and acks as described by the rabbitmq docs:

For unroutable messages, the broker will issue a confirm once the exchange verifies a message won't route to any queue (returns an empty list of queues). If the message is also published as mandatory, the basic.return is sent to the client before basic.ack. The same is true for negative acknowledgements (basic.nack).

Without exposing this ordering, it's not possible to know that a message -- in general -- was acked and not returned (the library is incompatible with this use-case). It's not just a matter of tracking returns and confirms. This doesn't invalidate your point that this use-case is uncommon and does not justify a re-architecture of the library.

If there are other instances like this one (where ordering between protocol messages is important but lost), I think it makes sense to add a note in the "Non Goals" section of the README. Otherwise, I don't think additional docs are warranted.

[michaelklishin]

Done in b896c73