How does camel dead letter channel work?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

How does camel dead letter channel work?

pratibhaG
Suppose I am sending a message from app1 to app2.
If app2 is down, at servicemix logs I get java.net.ConnectException.

I think if I am using deadletter the error should be caught and after some retries it should be given to dead letter channel.
Here is the configuration:

errorHandler(deadLetterChannel(
                "jbi:service:http://servicemix.in2m.com/samples/http/JMSServiceCamelErrorHandler").maximumRedeliveries(2));
                from("jbi:service:http://servicemix.in2m.com/samples/http/httpConsumer")
    .to("jbi:service:http://servicemix.in2m.com/samples/http/MyProviderService");

So if i am not able to connect to  my "MyProviderService", my message should be sent to "JMSServiceCamelErrorHandler" right?

Or my understanding is wrong?

Pratibha
Reply | Threaded
Open this post in threaded view
|

Re: How does camel dead letter channel work?

sakkew
Looks right to me. You could test it manually too by simply shutting app2 down and see what happens ;)

Consider using the exponential backoff setting. It's handy where network problems might occur http://activemq.apache.org/camel/dead-letter-channel.html.



pratibhaG wrote
Suppose I am sending a message from app1 to app2.
If app2 is down, at servicemix logs I get java.net.ConnectException.

I think if I am using deadletter the error should be caught and after some retries it should be given to dead letter channel.
Here is the configuration:

errorHandler(deadLetterChannel(
                "jbi:service:http://servicemix.in2m.com/samples/http/JMSServiceCamelErrorHandler").maximumRedeliveries(2));
                from("jbi:service:http://servicemix.in2m.com/samples/http/httpConsumer")
    .to("jbi:service:http://servicemix.in2m.com/samples/http/MyProviderService");

So if i am not able to connect to  my "MyProviderService", my message should be sent to "JMSServiceCamelErrorHandler" right?

Or my understanding is wrong?

Pratibha
Reply | Threaded
Open this post in threaded view
|

Re: How does camel dead letter channel work?

pratibhaG
I tried it by manually shutting down app2. But I never get a message in
"http://servicemix.in2m.com/samples/http/JMSServiceCamelErrorHandler". What can be a problem?

Pratibha
Reply | Threaded
Open this post in threaded view
|

Re: How does camel dead letter channel work?

gertv
Prathiba,

Could you enable DEBUG logging on org.apache.camel inside ServiceMix'
conf/log4j.xml.  After that, could you post a full trace of a single
MessageExchange (both the JBI and Camel messages in the log) so we can
figure out what's going on?

Regards,

Gert

pratibhaG wrote:
> I tried it by manually shutting down app2. But I never get a message in
> "http://servicemix.in2m.com/samples/http/JMSServiceCamelErrorHandler". What
> can be a problem?
>
> Pratibha
>  

Reply | Threaded
Open this post in threaded view
|

Re: How does camel dead letter channel work?

pratibhaG
It works. i get the message in deadletterchannel after the retries I specify.
I use this configuration:
errorHandler(deadLetterChannel(
                "jbi:service:http://servicemix.in2m.com/samples/http/JMSServiceCamelErrorHandler?mep=in-only")
                .maximumRedeliveries(2).useExponentialBackOff());

    from("jbi:service:http://servicemix.in2m.com/samples/http/httpConsumer")
            .to("jbi:service:http://servicemix.in2m.com/samples/http/MyProviderService")
            .to("jbi:service:http://servicemix.in2m.com/samples/http/bean2Service");

It retries the message delivery 2 times. OK. works great.

If I want to specify that my first attempt should be after 10 minutes then second after say 20 minutes from the  first attempt. and like wise the next attempts should happen with the timespan between two attempts always increased. That is it should be exponential.
Where can I specify it. How useExponentialBackOff() works? I think this is going to help me. i am using camel inside servicemix.

Pratibha
Reply | Threaded
Open this post in threaded view
|

Re: How does camel dead letter channel work?

pratibhaG
Got few new things.
1)By default after first attempt it waits for 1000 milli seconds.
2)after that for second attempt it waits for 2000 milli seconds.
3)third attempt after 4000 milli seconds. and so on.

Now how can I change this interval? Lets say I want to wait for 10 minutes after first attempt then for 20 minutes before next attempt and then 30 minutes?

Pratibha

Reply | Threaded
Open this post in threaded view
|

Re: How does camel dead letter channel work?

pratibhaG
Class DeadLetterChannelBuilder answers few of my questions.

Pratibha
Reply | Threaded
Open this post in threaded view
|

Re: How does camel dead letter channel work?

pratibhaG
Now I have following configuration:
public void configure() {
    errorHandler(deadLetterChannel(
                "jbi:service:http://servicemix.in2m.com/samples/http/JMSServiceCamelErrorHandler?mep=in-only")
                .maximumRedeliveries(4).initialRedeliveryDelay(300000).backOffMultiplier(2.0));

    from("jbi:service:http://servicemix.in2m.com/samples/http/httpConsumer")
            .to("jbi:service:http://servicemix.in2m.com/samples/http/MyProviderService")
            .to("jbi:service:http://servicemix.in2m.com/samples/http/bean2Service");
    }

I think the flow should be like this:
1)message is consumed from httpConsumer.
2)messge is sent to MyProviderService.
3)If MyProviderService is up and running, everything goes fine and the message is sent to bean2 service.
4)If MyProviderService is down, the messsage will be redelivered after 300000 ms that is 5 minutes. After 5 minutes if the application is again down, message will be redelivered after 10 minutes, then after 20 minutes and then 40 minutes. If still  MyProviderService is not up, message will be sent to deadletterchannel.

But I am getting something different:
After first attempt as per my servicemix logs it wits for 300000 ms, but then I get
java.lang.IllegalStateException: Exchange not found

Also if my client's request time is 2008-06-24 16:11:41,112, then it gets response at 2008-06-24 16:12:51,125.

What should ideally happen?

Here are my servicemix logs:
ERROR - DeadLetterChannel              - Failed delivery for exchangeId: ID-gpratibha.site/38645-1214286518597/0-14. On delivery attempt: 0 caught: java.net.ConnectException: Connection refused
java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:519)
        at java.net.Socket.connect(Socket.java:469)
        at java.net.Socket.<init>(Socket.java:366)
        at java.net.Socket.<init>(Socket.java:239)
        at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:79)
        at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:121)
        at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:706)
        at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1321)
        at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:386)
        at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
        at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:346)
        at org.apache.servicemix.http.processors.ProviderProcessor.process(ProviderProcessor.java:159)
        at org.apache.servicemix.common.AsyncBaseLifeCycle.doProcess(AsyncBaseLifeCycle.java:538)
        at org.apache.servicemix.common.AsyncBaseLifeCycle.processExchange(AsyncBaseLifeCycle.java:490)
        at org.apache.servicemix.common.BaseLifeCycle.onMessageExchange(BaseLifeCycle.java:46)
        at org.apache.servicemix.jbi.messaging.DeliveryChannelImpl.processInBound(DeliveryChannelImpl.java:610)
        at org.apache.servicemix.jbi.nmr.flow.AbstractFlow.doRouting(AbstractFlow.java:170)
        at org.apache.servicemix.jbi.nmr.flow.seda.SedaFlow.doRouting(SedaFlow.java:167)
        at org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
        at java.lang.Thread.run(Thread.java:595)
DEBUG - DefaultExceptionPolicyStrategy - Finding best suited exception policy for thrown exception java.net.ConnectException
DEBUG - DefaultExceptionPolicyStrategy - No candidate found to be used as exception policy
DEBUG - DeadLetterChannel              - Sleeping for: 300000 millis until attempting redelivery
DEBUG - JmsTemplate                    - Executing callback on JMS Session [ActiveMQSession {id=ID:gpratibha.site-40436-1214286465411-3:15554:1,started=false}]





DEBUG - JettyContextManager            - Dispatching job: RetryContinuation@25144440,pending,expired
DEBUG - ConsumerProcessor              - Receiving HTTP request: POST /example/ HTTP/1.1
Authorization: Basic c214OnNteA==
User-Agent: Jakarta Commons-HttpClient/3.1
Host: localhost:8192
Content-Length: 149


WARN  - jetty                          - /example/
java.lang.IllegalStateException: Exchange not found
        at org.apache.servicemix.http.processors.ConsumerProcessor.process(ConsumerProcessor.java:186)
        at org.apache.servicemix.http.HttpBridgeServlet.doPost(HttpBridgeServlet.java:71)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.Server.handle(Server.java:313)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:375)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
        at org.mortbay.jetty.nio.SelectChannelConnector$RetryContinuation.run(SelectChannelConnector.java:511)
        at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
DEBUG - JettyContextManager            - Dispatching job: SCEP@24886946[d=true,io=1,w=true,b=false|false]


Pratibha
Reply | Threaded
Open this post in threaded view
|

Re: How does camel dead letter channel work?

pratibhaG
Now the dead letter channel works fine. It retries at the interval that i have given. If my application is up inbetween the retries it gets proper response. The only problem is I get the warning
 
WARN  - jetty                          - /example/
java.lang.IllegalStateException: Exchange not found

after about 1 minute of the first try and hence my client gets the same message that is
DEBUG - JettyContextManager            - Dispatching job: RetryContinuation@28799325,pending,expired
DEBUG - ConsumerProcessor              - Receiving HTTP request: POST /example/ HTTP/1.1
Authorization: Basic c214OnNteA==
User-Agent: Jakarta Commons-HttpClient/3.1
Host: localhost:8192
Content-Length: 149


WARN  - jetty                          - /example/
java.lang.IllegalStateException: Exchange not found
        at org.apache.servicemix.http.processors.ConsumerProcessor.process(ConsumerProcessor.java:186)
        at org.apache.servicemix.http.HttpBridgeServlet.doPost(HttpBridgeServlet.java:71)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.Server.handle(Server.java:313)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:375)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
        at org.mortbay.jetty.nio.SelectChannelConnector$RetryContinuation.run(SelectChannelConnector.java:511)
        at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
DEBUG - JettyContextManager    


Now if application is up in the mean time and everything goes fine. Then i want to send success message to client. How can i do that? Is this a servicemix problem? Should I ask it at servicemix forum?

pratibha
Reply | Threaded
Open this post in threaded view
|

RE: How does camel dead letter channel work?

Claus Ibsen
Hi

> Is this a servicemix problem? Should I ask it at servicemix forum?
Yes please try there also, maybe they have the answer ready.


Med venlig hilsen
 
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk

-----Original Message-----
From: pratibhaG [mailto:[hidden email]]
Sent: 26. juni 2008 07:35
To: [hidden email]
Subject: Re: How does camel dead letter channel work?


Now the dead letter channel works fine. It retries at the interval that i
have given. If my application is up inbetween the retries it gets proper
response. The only problem is I get the warning
 
WARN  - jetty                          - /example/
java.lang.IllegalStateException: Exchange not found

after about 1 minute of the first try and hence my client gets the same
message that is
DEBUG - JettyContextManager            - Dispatching job:
RetryContinuation@28799325,pending,expired
DEBUG - ConsumerProcessor              - Receiving HTTP request: POST
/example/ HTTP/1.1
Authorization: Basic c214OnNteA==
User-Agent: Jakarta Commons-HttpClient/3.1
Host: localhost:8192
Content-Length: 149


WARN  - jetty                          - /example/
java.lang.IllegalStateException: Exchange not found
        at
org.apache.servicemix.http.processors.ConsumerProcessor.process(ConsumerProcessor.java:186)
        at
org.apache.servicemix.http.HttpBridgeServlet.doPost(HttpBridgeServlet.java:71)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
        at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
        at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:362)
        at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
        at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:211)
        at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
        at org.mortbay.jetty.Server.handle(Server.java:313)
        at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:375)
        at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396)
        at
org.mortbay.jetty.nio.SelectChannelConnector$RetryContinuation.run(SelectChannelConnector.java:511)
        at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442)
DEBUG - JettyContextManager    

Is this a servicemix problem? Should I ask it at servicemix forum?

pratibha

--
View this message in context: http://www.nabble.com/How-does-camel-dead-letter-channel-work--tp18026494s22882p18126869.html
Sent from the Camel - Users mailing list archive at Nabble.com.