Redelivery counts in classic queues

[michaelklishin]

This is a continuation to #502, which was addressed for quorum queues. It remains an open question whether the same feature for classic queues is necessary/the effort can be justified given that quorum queues support it already.

[tobydickenson]

#1020 describes how a nack loop can cause a DOS.

#502 mitigates that DOS risk using redelivery counts, however the DOS risk can be mitigated with something much more simple. At least I hope it is more simple. A queue policy to prevent all re-queuing following a nack would be really useful for me. That means basic.nack(requeue=True) gets treated the same as basic.nack(requeue=False). I guess this is effectively the same thing as delivery-limit=1, but without actually needing per-message storage for x-delivery-count

[michaelklishin]

Sorry but we won't be overriding the meaning of basic.nack fields based on a policy.

[barmic]

This feature can be really useful for me. In my understand, I can't implement it by plugin.
For me I really need a counter and the delivery limit will be a nice to have.

[michaelklishin]

Use quorum queues and you can use it today.

On Mon, 2 Mar 2020 at 16:06, Michel ***@***.***> wrote: This feature can be really useful for me. In my understand, I can't implement it by plugin. For me I really need a counter and the delivery limit will be a nice to have. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2013?email_source=notifications&email_token=AAAAIQURZNYJX6BSKVK64STRFOVM5A5CNFSM4HO4VCMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPHDGQ#issuecomment-593392026>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAIQXGZMRCWWVIISRRBA3RFOVM5ANCNFSM4HO4VCMA> .

-- Staff Software Engineer, Pivotal/RabbitMQ

[philomory]

I'd like to see x-delivery-count and delivery-limit available on classic queues, if for no other reason than that it's not possible to convert a classic queue into a quorum queue. More broadly, this is a generally useful feature that isn't directly related to a queue's HA strategy (I mean, it's needed to make quorum queues work properly, but it's not itself an HA feature). So, unless quorum queues are actually intended to replace classic queues entirely, it'd be unfortunate to restrict access to a non-HA-related feature based on the HA-model chosen.

Obviously, none of that is a comment on the effort required to implement this for classic queues; it may indeed be that it's simply not feasible to do this, or that it shouldn't be a priority. But I'd love to be able to simply turn on delivery-limiting via policy for some of our legacy applications, without having to alter those applications so they declare quorum queues.

[michaelklishin]

It is possible to move messages from a classic queue to a different queue, quorum or not.

It is increasingly difficult to justify time and effort investment (and adding to an already extensive feature matrix) into classic queues for our team. This issue has a low probability of being addressed. The future of RabbitMQ is more specialized, optimized, use case-focussed queue types.
Adding more and more and more features to (currently) 13-year-old classic queues only can go so far and we are near the practical limit in my opinion. Specifically, the fundamental problem with poison message handling can be solved on the consumer end.

I'm locking this issue as this is turning into an upvoting fest. Those who really want to see classic queues support redelivery limits are welcome to contribute this feature. Contributions are not guaranteed to be accepted but they will be considered. This is open-source software after all.