The problem

The stock service consumes the stock.events queue. An order.order.placed.v1 event mentions a product that stock does not know: its handler fails on every attempt.

Without a limit, this message loops forever. It keeps the worker busy and delays the messages queued behind it. With a limit but no quarantine, it disappears after the last attempt.

So we need a finite number of attempts, then a quarantine where the message stays readable and replayable. That is the job of a dead letter queue (DLQ).

The solution

Retries with a growing delay, then a failure transport

For a Symfony Messenger consumer, we configure the transport's retries and a failure transport (Symfony 7.4): 

# config/packages/messenger.yaml (stock service)
framework:
  messenger:
    failure_transport: failed
    transports:
      events:
        dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
        # options: exchange app.events, queue stock.events
        retry_strategy:
          max_retries: 3
          delay: 1000
          multiplier: 2
      failed: 'doctrine://default?queue_name=failed'

Messenger retries the message after about 1 s, 2 s then 4 s. Each retry goes through a delay queue that sends it back to stock.events only, as described in our article on the RabbitMQ topology. After the last one, the message goes to failed, stored in the database.

The failure_transport key can also be set per transport. For an error known to be permanent, the handler can throw UnrecoverableMessageHandlingException: the message then goes to quarantine without any retry.

What the quarantine keeps

The failure transport keeps the message and its stamps. ErrorDetailsStamp holds the class, code and message of the last exception. SentToFailureTransportStamp names the original transport, and the RedeliveryStamp entries date the attempts the serializer kept.

A custom serializer must at least keep the retry count, in a header read back as a RedeliveryStamp. Without it, the counter would start again from zero and the message would never leave the loop. 

php bin/console messenger:failed:show           # list the quarantine
php bin/console messenger:failed:show 42 -vv    # details, exception included
php bin/console messenger:failed:retry 42       # handle the message again
php bin/console messenger:failed:remove 42      # discard it for good

A replayed message that fails again goes back to failed, three times at most by default. After that, Messenger deletes it for good.

Replaying assumes an idempotent handler, the subject of our article on the idempotent consumer.

Other languages: the DLX

Say shipping is written in another language. The quarantine then lives in RabbitMQ: a dead letter exchange (DLX) receives the messages the queue discards.

We declare it with a policy rather than with the x-dead-letter-exchange argument. A policy can be changed without redeploying or recreating the queue. 

rabbitmqctl set_policy shipping-dlx '^shipping\.events$' \
  '{"dead-letter-exchange":"app.dead-letters","dead-letter-routing-key":"shipping.events"}' \
  --apply-to queues

The app.dead-letters exchange and the shipping.events.dlq queue, bound with the shipping.events key, must exist. Otherwise, RabbitMQ silently drops the message. After its own attempts, the consumer rejects the message with requeue=false.

RabbitMQ then adds the x-death header: original queue, reason (rejected, expired, maxlen, delivery_limit), count and time. The x-first-death-queue and x-first-death-reason headers keep the first occurrence.

Since RabbitMQ 4.0, a quorum queue sets delivery-limit to 20 by default. A message redelivered more often, for instance after worker crashes, is dropped, or sent to the DLX if there is one.

The pitfall

Two quarantine levels: Messenger in the application, the DLX in the broker Application level: stock service, Symfony Messenger consumer stock.events queue Messenger worker the handler fails retries left delay queue 1 s, 2 s then 4 s new attempt after max_retries failed failure transport message, exception and failure dates messenger:failed:* Broker level: shipping service, consumer written in another language shipping.events queue consumer rejects, requeue=false dead-letter-exchange policy app.dead-letters dead letter exchange shipping.events.dlq message + x-death header The exception is not there: the consumer logs it. Avoid: a DLX on stock.events on top of the failure transport Messenger rejects the original after each failure: the DLQ would get one copy per attempt.

RabbitMQ DLX Messenger failure transport
Level broker application
Trigger rejection, TTL, delivery-limit failure after max_retries
Content message and x-death header message, exception, failure dates
Replay needs tooling messenger:failed:retry
Consumers any language Symfony Messenger

Stacking both levels on the same queue creates duplicates. After each failure, Messenger republishes a copy, then rejects the original without requeueing it. A DLX on stock.events would get one copy per attempt: four with max_retries: 3.

Conversely, in Symfony 7.4, a message the serializer cannot decode is rejected before any retry. It never reaches failed: without a DLX, RabbitMQ drops it. Since Symfony 8.1, it follows the normal path, retries then failure transport.

Our recommendation: a single quarantine level per queue. The failure transport for Messenger consumers, the DLX for other languages.

On Symfony 7.4, this choice loses unreadable messages, which schema validation in our producers reduces. To keep them: Symfony 8.1 (same serializer on failed) or an archive DLX on stock.events, never replayed in bulk.

Sources