[CONF] Apache Camel: Dead Letter Channel (page edited)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[CONF] Apache Camel: Dead Letter Channel (page edited)

Dhiraj Bokde (Confluence)

Dead Letter Channel has been edited by Claus Ibsen (Feb 22, 2009).

(View changes)

Content:

Dead Letter Channel

Camel supports the Dead Letter Channel from the EIP patterns using the DeadLetterChannel processor which is an Error Handler.

Redelivery

It is common for a temporary outage or database deadlock to cause a message to fail to process; but the chances are if its tried a few more times with some time delay then it will complete fine. So we typically wish to use some kind of redelivery policy to decide how many times to try redeliver a message and how long to wait before redelivery attempts.

The RedeliveryPolicy defines how the message is to be redelivered. You can customize things like

  • how many times a message is attempted to be redelivered before it is considered a failure and sent to the dead letter channel
  • the initial redelivery timeout
  • whether or not exponential backoff is used (i.e. the time between retries increases using a backoff multiplier)
  • whether to use collision avoidance to add some randomness to the timings
  • delay pattern a new option in Camel 2.0, see below for details.

Once all attempts at redelivering the message fails then the message is forwarded to the dead letter queue.

OnRedelivery

Available in Camel 1.5.1 onwards

When Dead Letter Channel is doing redeliver its possible to configure a Processor that is executed just before every redelivery attempt. This can be used for the situations where you need to alter the message before its redelivered. See below for sample.

onException and onRedeliver

In Camel 2.0 we also added support for per onException to set a onRedeliver. That means you can do special on redelivery for different exceptions, as opposed to onRedelivery set on Dead Letter Channel can be viewed as a global scope.

Redelivery default values

The default redeliver policy will use the following values:

  • maximumRedeliveries=5
  • delay=1000L (1 second, new as of Camel 2.0)
    • use initialRedeliveryDelay for previous versions
  • maximumRedeliveryDelay = 60 * 1000L (60 seconds, new option in Camel 1.4)
  • And the exponential backoff and collision avoidance is turned off.
  • The retriesExhaustedLogLevel and retryAttemptedLogLevel are set to LoggingLevel.ERROR

The maximum redeliver delay ensures that a delay is never longer than the value, default 1 minute. This can happen if you turn on the exponential backoff.

The maximum redeliveries is the number of re delivery attempts. By default Camel will try to process the exchange 1 + 5 times. 1 time for the normal attempt and then 5 attempts as redeliveries.
Setting the maximumRedeliveries to a negative value such as -1 will then always redelivery (unlimited).
Setting the maximumRedeliveries to 0 will disable any re delivery attempt.

Camel will log delivery failures at the ERROR logging level by default. You can change this by specifying retriesExhaustedLogLevel and/or retryAttemptedLogLevel. See ExceptionBuilderWithRetryLoggingLevelSetTest for an example.

Redeliver Delay Pattern

Available as of Camel 2.0
Delay pattern is used as a single option to set a range pattern for delays. If used then the following options does not apply: (delay, backOffMultiplier, useExponentialBackOff, useCollisionAvoidance, maximumRedeliveryDelay).

The idea is to set groups of ranges using the following syntax: limit:delay;limit 2:delay 2;limit 3:delay 3;...;limit N:delay N

Each group has two values separated with colon

  • limit = upper limit
  • delay = delay in millis
    And the groups is again separated with semi colon.
    The rule of thumb is that the next groups should have a higher limit than the previous group.

Lets clarify this with an example:
delayPattern=5:1000;10:5000;20:20000

That gives us 3 groups:

  • 5:1000
  • 10:5000
  • 20:20000

Resulting in these delays for redelivery attempt:

  • Attempt number 0..4 = 0 millis (as the first group start with 5)
  • Attempt number 5..9 = 1000 millis (the first group)
  • Attempt number 10..19 = 5000 millis (the second group)
  • Attempt number 20.. = 20000 millis (the last group)

You can start a group with limit 0 to eg have a starting delay: delayPattern=0:1000;5:5000

  • Attempt number 0..4 = 1000 millis (the first group)
  • Attempt number 5.. = 5000 millis (the last group)

There is no requirement that the next delay should be higher than the previous. You can use any delay value you like. For example with delayPattern=0:5000;3:1000 we start with 5 sec delay and then later reduce that to 1 second.

Redelivery header

When a message is redelivered the DeadLetterChannel will append a customizable header to the message to indicate how many times its been redelivered.
In Camel 1.x: The header is org.apache.camel.redeliveryCount.
In Camel 2.0: The header is CamelRedeliveryCount.

And a boolean flag whether it is being redelivered or not (first attempt)
In Camel 1.x: The header org.apache.camel.Redelivered contains a boolean if the message is redelivered or not.
In Camel 2.0: The header CamelRedelivered contains a boolean if the message is redelivered or not.

Samples

The following example shows how to configure the Dead Letter Channel configuration using the DSL

RouteBuilder builder = new RouteBuilder() {
    public void configure() {
        // using dead letter channel with a seda queue for errors
        errorHandler(deadLetterChannel("seda:errors"));

        // here is our route
        from("seda:a").to("seda:b");
    }
};

You can also configure the RedeliveryPolicy as this example shows

RouteBuilder builder = new RouteBuilder() {
    public void configure() {
        // configures dead letter channel to use seda queue for errors and use at most 2 redelveries
        // and exponential backoff
        errorHandler(deadLetterChannel("seda:errors").maximumRedeliveries(2).useExponentialBackOff());

        // here is our route
        from("seda:a").to("seda:b");
    }
};

How can I modify the Exchange before redelivery?

In Camel 1.5.1 we added support directly in Dead Letter Channel to set a Processor that is executed before each redelivery attempt. In older releases you have to sort to other solutions that isn't as solid as this new feature.

Camel 1.5.0 or older

When Dead Letter Channel is doing redeliver it redeliveries immediately with the original Exchange that caused the error in the first place. However in some situations you might want to be able to alter the message before its redelivered. As Camel at this time of writing doesn't have a nice DSL syntax or configuration on the Dead Letter Channel to allow custom processing before redeliver we are gonna show a different solution, that actually also pin points the flexibility Camel has.

We are going to use an interceptor that gets triggered when an Exchange is being redelivered. We use the fact that interceptors by default will proceed from the point of interceptor. This is the Detour EIP pattern we are using.

The code below demonstrates this.

// we configure an interceptor that is triggered when the redelivery flag
// has been set TRUE on an exchange
intercept().when(header("CamelRedelivered").isEqualTo(Boolean.TRUE)).
        process(new Processor() {
            public void process(Exchange exchange) throws Exception {
                // the message is being redelivered so we can alter it

                // we just append the redelivery counter to the body
                // you can of course do all kind of stuff instead
                String body = exchange.getIn().getBody(String.class);
                int count = exchange.getIn().getHeader("CamelRedeliveryCounter", Integer.class);

                exchange.getIn().setBody(body + count);
            }
        });
Code uses wrong header key

As the code above is based on an unit test its based on Camel 2.x or newer. The header to use in Camel 1.x should be "org.apache.camel.redeliveryCount" instead of "CamelRedeliveryCount".

However you should notice as Camel will keep the redeliver flag on the Exchange for the remainder of its routing this interceptor will kick in for subsequence processing. So you should keep track if you already have altered the message before redelivery.

Camel 1.5.1 or newer

When Dead Letter Channel is doing redeliver its possible to configure a Processor that is executed just before every redelivery attempt. This can be used for the situations where you need to alter the message before its redelivered.

Here we configure the Dead Letter Channel to use our processor MyRedeliveryProcessor to be executed before each redelivery.

// we configure our Dead Letter Channel to invoke
// MyRedeliveryProcessor before a redelivery is
// attempted. This allows us to alter the message before
errorHandler(deadLetterChannel("mock:error")
        .onRedelivery(new MyRedeliverPrcessor())
        // setting delay to zero is just to make unit teting faster
        .delay(0L));

And this is the processor MyRedeliveryProcessor where we alter the message.

// This is our processor that is executed before every redelivery attempt
// here we can do what we want in the java code, such as altering the message
public class MyRedeliverPrcessor implements Processor {

    public void process(Exchange exchange) throws Exception {
        // the message is being redelivered so we can alter it

        // we just append the redelivery counter to the body
        // you can of course do all kind of stuff instead
        String body = exchange.getIn().getBody(String.class);
        int count = exchange.getIn().getHeader("CamelRedeliveryCounter", Integer.class);

        exchange.getIn().setBody(body + count);
    }
}

Using This Pattern

If you would like to use this EIP Pattern then please read the Getting Started, you may also find the Architecture useful particularly the description of Endpoint and URIs. Then you could try out some of the Examples first before trying this pattern out.