Redelivery with multiple transacted routes

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Redelivery with multiple transacted routes

kafe
Hi,

I have some problem to have a correct redelivery policy in my environment.
I work with transacted routes and i'd like to have 3 redelivery attempts if any exception is raised.

I have the following routes :

<route id="routeA">
   <from uri="activemq:queue:traiterReponse" />
   <onException useOriginalMessage="true">
      <exception>java.lang.Exception</exception>
      <redeliveryPolicy maximumRedeliveries="3" redeliveryDelay="5000" />
      <handled>
         <constant>true</constant>
      </handled>
      <to uri="activemq:queue:pendingMessage" />
   </onException>
   <transacted />
   <bean ref="processor1" />
   <to uri="direct:traiterReponse" />
</route>

<route id="routeB">
   <from uri="direct:traiterReponse" />
   <bean ref="processor2" />
   <to uri="log:output" />
</route>

If an exception is raised by the processor1, it works as expected, there are 3 attempts to redeliver the message and if it fails, the message is then redirected to the 'pendingMessage' queue.

But the problem is if an exception is raised in processor2, then the message is directly redirected in the 'ActiveMQ.DLQ' queue, there is no redelivery attempt at all.

I expected the exception to be "catched" by the 'onException' clause of routeA (if i'm right the 2 routes share the same transaction).

What is the problem ?
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Babak Vahdat
Hi

sharing the same transaction boundary doesn't necessarily mean sharing the same onException clause by two different routes.

In this case defining the onException clause at the global scope should help. Check also [1] for the global & route specific scope definitions.

[1] http://camel.apache.org/exception-clause.html

Babak
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

kafe
The problem is that if i put an onException clause on the second route, when there is a redelivery (after an exception raised by 'processor2'), it directly starts at the point the exception was raised not at the beginning of routeA.
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

kafe
Hum... actually this seems to be a normal behaviour :

"Point of entry for redelivery attempts

All redelivery attempts start at the point of the failure. So the route:

.onException(ConnectException.class)
.from("direct:start")
 .process("processor1")
 .process("processor2") // <--- throws a ConnectException
.to("mock:theEnd")

Will retry from processor2 - not the complete route. "

(http://camel.apache.org/exception-clause.html)

Thank you for your help.
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Babak Vahdat
In reply to this post by kafe
I didn't mean to put an onException() on the second route *but* on the global scope, so something like:

<onException useOriginalMessage="true">
   <exception>java.lang.Exception</exception>
   <redeliveryPolicy maximumRedeliveries="3" redeliveryDelay="5000" />
   <handled>
     <constant>true</constant>
   </handled>
   <to uri="activemq:queue:pendingMessage" />
</onException>


<route id="routeA">
   <from uri="activemq:queue:traiterReponse" />
   <transacted />
   <bean ref="processor1" />
   <to uri="direct:traiterReponse" />
</route>

<route id="routeB">
   <from uri="direct:traiterReponse" />
   <bean ref="processor2" />
   <to uri="log:output" />
</route>

Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Babak Vahdat
In reply to this post by kafe
Yeah exactly, all redelivery attempts start at the point of the failure...

BTW this behaviour seems to me in synergy with the plain java where you catch an Exception some where in your code and then you continue with the logic right *after* the catch block and not necessarily from the first line of your Main method :-)

Babak
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

kafe
The remaining problem is that the thread that makes the redeliveries is blocked until all redeliveries have been done.

Potentially, if the redeliveryPolicy is configured to redeliver undefinitely (which will probably be my case in the future) and lot of messages arrive in my initial ActiveMQ queue, the system could be down !?

Isn't it possible to have a redelivery policy in a transacted mode without having the thread (ie an activeMQ consumer) blocked ?
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Babak Vahdat
Hi,

I just copy&paste what the documentation says about this second question of yours:

From Camel 2.4 onwards Camel has a feature to not block while waiting for a delayed redelivery to occur. However if you use transacted routes then Camel will block as its mandated by the transaction manager to execute all the work in the same thread context. You can enable the non blocking asynchronous behavior by the asyncDelayedRedelivery option. This option can be set on the errorHandler, onException or the redelivery policies.
By default the error handler will create and use a scheduled thread pool to trigger redelivery in the future. From Camel 2.8 onwards you can configure the executorServiceRef on the Error Handler to indicate a reference to either a shared thread pool you can enlist in the registry, or a thread pool profile in case you want to be able to control pool settings.

Babak
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Raul Kripalani-2
If you are using JMS transactions, how about rolling back the
transaction when an exception occurs, and letting AMQ do the
redelivery for you? That way the message will again enter the route at
the beginning (from endpoint).

You do this by configuring a redelivery policy on the connection
factory. Check out: http://activemq.apache.org/redelivery-policy.html.

On 19 Jan 2012, at 18:27, Babak Vahdat <[hidden email]> wrote:

> Hi,
>
> I just copy&paste what the documentation says about this second question of
> yours:
>
> From Camel 2.4 onwards Camel has a feature to not block while waiting for a
> delayed redelivery to occur. However if you use transacted routes then Camel
> will block as its mandated by the transaction manager to execute all the
> work in the same thread context. You can enable the non blocking
> asynchronous behavior by the asyncDelayedRedelivery option. This option can
> be set on the errorHandler, onException or the redelivery policies.
> By default the error handler will create and use a scheduled thread pool to
> trigger redelivery in the future. From Camel 2.8 onwards you can configure
> the executorServiceRef on the Error Handler to indicate a reference to
> either a shared thread pool you can enlist in the registry, or a thread pool
> profile in case you want to be able to control pool settings.
>
> Babak
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Redelivery-with-multiple-transacted-routes-tp5158209p5158521.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

kafe
@Babak : it seems that the asyncDelayedRedelivery option does not apply to transacted routes ("However if you use transacted routes then Camel will block as its mandated by the transaction manager to execute all the work in the same thread context"). But i'll try that to be sure.

@Raul : actually, that could be the solution to my problem, but i did not succeed in having ActiveMQ perform a correct redelivery. I'll post my jms configuration tomorrow, which probably has some errors.

Thank you for your help.
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Babak Vahdat
Hi Kafe,

that was exactly my intention to make it clear to you that this will NOT be possible as long as your routing logic is transacted. For instance different implementations of Spring's PlatformTransactionManager keep the state of the currently running transaction into a ThreadLocal so that one cannot simply spawn another thread to continue routing in another thread for the SAME transaction.

BTW, Raul's suggestion sounds good to me.

Babak
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

kafe
Thank you for these precisions.
I'll have to make native activemq redelivery work tomorow then !
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

kafe
I'm reading again and again the several documentations about transaction and error handling, and i have one more question:
when working with the transactionErrorHandler (http://camel.apache.org/transactionerrorhandler.html), and especially when configuring redelivery (maximumRedeliveries(6)...), is it Camel or ActiveMQ that performs the redelivery ?
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Babak Vahdat
Digging into Camel code is the best way to find it out.

It's Camel doing the redelivery, as the redelivery facility should generally work for all other kinds of components as well and not just ActiveMQ.

Babak
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Claus Ibsen-2
In reply to this post by kafe
ActiveMQ 5.6 will support non blocking redelivery. You would need to
enable this on the AMQ side.

You can have both Camel + AMQ do redelivery. Camel will do redelivery
at the point of problem. So that could be in the middle of a route, if
a processor fails etc. AMQ is always redelivering from the beginning
again, eg its basically just re-submitting the message.

If you use them both you end up having x^2 redelivery. AMQ is by
default configured to do 6 redelivery attempts.


On Thu, Jan 19, 2012 at 10:40 PM, kafe <[hidden email]> wrote:

> I'm reading again and again the several documentations about transaction and
> error handling, and i have one more question:
> when working with the transactionErrorHandler
> (http://camel.apache.org/transactionerrorhandler.html), and especially when
> configuring redelivery (maximumRedeliveries(6)...), is it Camel or ActiveMQ
> that performs the redelivery ?
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Redelivery-with-multiple-transacted-routes-tp5158209p5158983.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



--
Claus Ibsen
-----------------
FuseSource
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

kafe
Hi,

It's good to know that both Camel & AMQ redelivery can be used.

Currently, i'm trying to have AMQ do the redelivery but i have a strange behaviour : the redelivery delay is not taken into account. here is my Camel/JMS config :

<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
        <property name="brokerURL" value="tcp://localhost:61616" />
        <property name="redeliveryPolicy" ref="redeliveryPolicy" />
</bean>
       
<bean id="redeliveryPolicy" class="org.apache.activemq.RedeliveryPolicy">
        <property name="maximumRedeliveries" value="2" />
        <property name="initialRedeliveryDelay" value="5000" />
        <property name="redeliveryDelay" value="5000" />
</bean>

<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
        <property name="maxConnections" value="10" />
        <property name="maximumActive" value="10" />
        <property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>

<bean id="jmsTxManager" class="org.springframework.jms.connection.JmsTransactionManager">
        <property name="connectionFactory" ref="pooledConnectionFactory" />
</bean>
       
<bean id="PROPAGATION_REQUIRED" class="org.apache.camel.spring.spi.SpringTransactionPolicy">
        <property name="transactionManager" ref="jmsTxManager"/>
        <property name="propagationBehaviorName" value="PROPAGATION_REQUIRED"/>
</bean>

<bean id="activemq" class="org.apache.camel.component.jms.JmsComponent">
        <property name="connectionFactory" ref="pooledConnectionFactory" />
        <property name="transacted" value="true" />
        <property name="transactionManager" ref="jmsTxManager" />
</bean>

I modified my routes to have a NoErrorHandler :

<errorHandler id="noErrorHandler" type="NoErrorHandler" />

<route id="routeA" errorHandlerRef="noErrorHandler">
   <from uri="activemq:queue:traiterReponse" />
   <transacted />
   <bean ref="processor1" />
   <to uri="direct:traiterReponse" />
</route>

<route id="routeB" errorHandlerRef="noErrorHandler">
   <from uri="direct:traiterReponse" />
   <bean ref="processor2" />
   <to uri="log:output" />
</route>

With this config, i have 2 redeliveries but there is not a 5 seconds delay beetween the initial attempt and the 2 redeliveries (it seems that AMQ redelivers immediatly the message).

Am i missing something ?
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Babak Vahdat
Hi,

the redeliveryDelay [1] will be taken into the account *if* initialRedeliveryDelay is 0 which you've set it to be 5000.

[1] http://activemq.apache.org/redelivery-policy.html

Babak
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

kafe
Hi Babak,

It did not resolve the problem.
Messages are still being redelivered immediatly.

Eric
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

Babak Vahdat
Hi,

in case nobody else has a better idea I recommand you to ask for help in ActiveMQ user-forum [1] as you decided to use ActiveMQ redelivery policy and not the one Camel provides.

[1] http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Babak
Reply | Threaded
Open this post in threaded view
|

Re: Redelivery with multiple transacted routes

anoordover
I have the same problem: http://camel.465427.n5.nabble.com/Container-managed-redelivery-tp5159934p5159934.html

Up until version 2.7.5 redelivery delay is taken into account.
From version 2.8.0 and onwards redelivery on the container isn't taken into account anymore and redelivery in immediate.
12