Open
Description
- Use
rabbit_queue_type
API directly if shovel source or destination is in the local cluster. - Use credit API v2 of the
rabbit_queue_type
API. This ensures that the shovel shovels as many messages as can be handled by the destination. For example, if the destination grants lots of link credit to the shovel, the shovel can also pass a high number of link credit to the source, therefore enabling high throughput shovelling. If the destination grants few link credit to the shovel, the shovel passes only few credits to the source. This ensures that the shovel itself isn’t flooded with messages. - If the source queue is in the local cluster, the shovel should run co-located on a node with a queue member to reduce intra cluster network traffic. (This is currently being worked on separately by @michaelklishin.)
- Likewise, if the destination is a queue in the local cluster, the shovel should ideally run the queue leader node.
- If both source and destination are in the local cluster, no message cross protocol translation should take place. This ensures no message metadata gets lost and guarantees the immutability of the AMQP 1.0 bare message.
- If a message needs to be re-published with a protocol that’s different in how the message was stored in the local source queue, a translation takes place using the
mc
API according to our well documented conversion rules. This solves AMQP 1.0 Application Properties lost when shovelling to an AMQP 0.9.1 broker #13957 - If a message gets shovelled only within the local cluster, the message should be neither encoded/serialised nor decoded/deserialised.
- If a message gets shovelled from a local queue to a remote broker, the message should get encoded/serialised once (as opposed to multiple times as for example done in Shovel: use message containers #14096).
- If a message gets shovelled from a remote broker to a local destination, the message should be decoded/deserialised once.
- The shovel should use AMQP address format v2.
- The shovel should guarantee the immutability of the AMQP bare message. For example if a message is shovelled from a 3rd party AMQP 1.0 broker to RabbitMQ, the integrity of the bare message must not be violated. This solves ActiveMQ to RabbitMQ shovel results in amqp header data being inserted in message data #12455