Router Blocked by full Destination Route Queue

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Router Blocked by full Destination Route Queue

yesnid
Hello All,

My router is becoming blocked by sending messages to a queue on which no one is taking messages from the queue, when the queue becomes full, the router is blocked waiting for more space in the queue to become free, any ideas about how I could get around this? In my application it is possible that applications listening to queues may come up and down for a shortwhile so I can't just simply purge a queue when there are no consumers. Also If I wait for the queue to become completely full before taking action it causes a sort of cascade effect causing others to rapidly fill up as well.

Thank you for the input, in advance.
Reply | Threaded
Open this post in threaded view
|

Re: Router Blocked by full Destination Route Queue

huntc
Can you please provide the route's specification in code or XML. This will give us a better chance of answering your question.

Having said that, sounds like you've got an internal seda queue... I would have thought that an exception would be thrown once the queue becomes full. By default a seda queue can only hold 1000 items.

If it is a JMS queue, then would setting the time to live option be useful to you?
Reply | Threaded
Open this post in threaded view
|

Re: Router Blocked by full Destination Route Queue

Bruno Borges
In reply to this post by yesnid
Maybe you should use a persistent queue, for later processing.


Bruno Borges
blog.brunoborges.com.br
+55 21 76727099

"The glory of great men should always be
measured by the means they have used to
acquire it."
- Francois de La Rochefoucauld


On Thu, Apr 9, 2009 at 10:47 PM, yesnid <[hidden email]> wrote:

>
> Hello All,
>
> My router is becoming blocked by sending messages to a queue on which no
> one
> is taking messages from the queue, when the queue becomes full, the router
> is blocked waiting for more space in the queue to become free, any ideas
> about how I could get around this? In my application it is possible that
> applications listening to queues may come up and down for a shortwhile so I
> can't just simply purge a queue when there are no consumers. Also If I wait
> for the queue to become completely full before taking action it causes a
> sort of cascade effect causing others to rapidly fill up as well.
>
> Thank you for the input, in advance.
> --
> View this message in context:
> http://www.nabble.com/Router-Blocked-by-full-Destination-Route-Queue-tp22981898p22981898.html
> Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Router Blocked by full Destination Route Queue

Ashwin Karpe
In reply to this post by yesnid
Hi,

In case you are averse to changing your Camel route, you can set up your current queue as a virtual queue to where all your JMS producers send their messages. The virtual queue could send messages to several real queues at the back.

The JMS Consumers could then feed of these real queues when they come up.

This will require a rule to be added to the ActiveMQ Broker via the activemq.xml file. Check this out...

http://activemq.apache.org/virtual-destinations.html

Cheers,

Ashwin...

yesnid wrote
Hello All,

My router is becoming blocked by sending messages to a queue on which no one is taking messages from the queue, when the queue becomes full, the router is blocked waiting for more space in the queue to become free, any ideas about how I could get around this? In my application it is possible that applications listening to queues may come up and down for a shortwhile so I can't just simply purge a queue when there are no consumers. Also If I wait for the queue to become completely full before taking action it causes a sort of cascade effect causing others to rapidly fill up as well.

Thank you for the input, in advance.
---------------------------------------------------------
Ashwin Karpe
Apache Camel Committer & Sr Principal Consultant
Red Hat
http://www.redhat.com

Blog:http://opensourceknowledge.blogspot.com
---------------------------------------------------------
Reply | Threaded
Open this post in threaded view
|

Re: Router Blocked by full Destination Route Queue

yesnid
In reply to this post by Bruno Borges
Can you extrapolate on what you mean by this? I thought I was using persistent queues, but what makes you think that, that should solve this issue?

Thank you.
Bruno Borges wrote
Maybe you should use a persistent queue, for later processing.


Bruno Borges
blog.brunoborges.com.br
+55 21 76727099

"The glory of great men should always be
measured by the means they have used to
acquire it."
- Francois de La Rochefoucauld


On Thu, Apr 9, 2009 at 10:47 PM, yesnid <nsimi@pelco.com> wrote:

>
> Hello All,
>
> My router is becoming blocked by sending messages to a queue on which no
> one
> is taking messages from the queue, when the queue becomes full, the router
> is blocked waiting for more space in the queue to become free, any ideas
> about how I could get around this? In my application it is possible that
> applications listening to queues may come up and down for a shortwhile so I
> can't just simply purge a queue when there are no consumers. Also If I wait
> for the queue to become completely full before taking action it causes a
> sort of cascade effect causing others to rapidly fill up as well.
>
> Thank you for the input, in advance.
> --
> View this message in context:
> http://www.nabble.com/Router-Blocked-by-full-Destination-Route-Queue-tp22981898p22981898.html
> Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
>
>


-----
Bruno Borges
blog.brunoborges.com.br
+55 21 76727099

"The glory of great men should always be
measured by the means they have used to
acquire it."
- Francois de La Rochefoucauld
Reply | Threaded
Open this post in threaded view
|

Re: Router Blocked by full Destination Route Queue

yesnid
In reply to this post by huntc
Hello,

Below you will find a simplified example, of what my router is doing, basically recieving on one queue and then resending on another in this example, now to get the test working properly, have a producer send messages to the recieving queue and no one listening to the destination queue eventually the router will hang.

My producers are connecting with the following, settings to the broker; Delivery Mode = persistent, Message TTL = 0 (Essentially infinite), transacted = false, session = auto acknowledge.

To see the problem faster you can set the memory in the policy entry for you queue to lower, I suggest really low so that the problem appears really fast and you don't have to wait to see it appear.

public class TestCamelRouter extends RouteBuilder
{
        final static String BEAN_NAME = "TestCamelRouter";
        final static String BEAN_METHOD = "route";

        @RecipientList
        public List<String> route(Exchange exchange)
        {
                ArrayList<String> returnSet = new ArrayList<String>();
                returnSet.add("JMS-BROKER" + ":queue:" + "TestDeliveryQueue");
                return returnSet;
        }

        @Override
        public void configure() throws Exception
        {
                from("JMS-BROKER" + ":queue:" + "RecievingQueue").beanRef(TestCamelRouter.BEAN_NAME, TestCamelRouter.BEAN_METHOD);
        }

        public static void main(String[] args)
        {
                JndiContext jndiContext = null;
                try
                {
                        jndiContext = new JndiContext();
                        jndiContext.bind(TestCamelRouter.BEAN_NAME, new TestCamelRouter());
                }
                catch (Exception e)
                {
                        e.printStackTrace();
                        return;
                }
                CamelContext context = new DefaultCamelContext(jndiContext);
                ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("tcp://localhost:61616");
                factory.setDispatchAsync(true);
                context.addComponent("JMS-BROKER", JmsComponent.jmsComponentAutoAcknowledge(factory));
                try
                {
                        context.addRoutes(new TestCamelRouter());
                        context.start();
                        while (true)
                                Thread.sleep(60000);
                }
                catch (Exception e)
                {
                        e.printStackTrace();
                        return;
                }
        }

}


 



huntc wrote
Can you please provide the route's specification in code or XML. This will give us a better chance of answering your question.

Having said that, sounds like you've got an internal seda queue... I would have thought that an exception would be thrown once the queue becomes full. By default a seda queue can only hold 1000 items.

If it is a JMS queue, then would setting the time to live option be useful to you?
Reply | Threaded
Open this post in threaded view
|

Re: Router Blocked by full Destination Route Queue

Claus Ibsen-2
Hi

I was looking into another matter and the problem with the sending to
JMS hangs is explained a bit here:
http://www.nabble.com/Do-the-send-timeouts-actually-work--td20583221.html

The other matter is this one:
http://fusesource.com/issues/browse/MR-170

On Sun, Apr 12, 2009 at 6:47 PM, yesnid <[hidden email]> wrote:

>
> Hello,
>
> Below you will find a simplified example, of what my router is doing,
> basically recieving on one queue and then resending on another in this
> example, now to get the test working properly, have a producer send messages
> to the recieving queue and no one listening to the destination queue
> eventually the router will hang.
>
> My producers are connecting with the following, settings to the broker;
> Delivery Mode = persistent, Message TTL = 0 (Essentially infinite),
> transacted = false, session = auto acknowledge.
>
> To see the problem faster you can set the memory in the policy entry for you
> queue to lower, I suggest really low so that the problem appears really fast
> and you don't have to wait to see it appear.
>
> public class TestCamelRouter extends RouteBuilder
> {
>        final static String BEAN_NAME = "TestCamelRouter";
>        final static String BEAN_METHOD = "route";
>
>        @RecipientList
>        public List<String> route(Exchange exchange)
>        {
>                ArrayList<String> returnSet = new ArrayList<String>();
>                returnSet.add("JMS-BROKER" + ":queue:" + "TestDeliveryQueue");
>                return returnSet;
>        }
>
>        @Override
>        public void configure() throws Exception
>        {
>                from("JMS-BROKER" + ":queue:" +
> "RecievingQueue").beanRef(TestCamelRouter.BEAN_NAME,
> TestCamelRouter.BEAN_METHOD);
>        }
>
>        public static void main(String[] args)
>        {
>                JndiContext jndiContext = null;
>                try
>                {
>                        jndiContext = new JndiContext();
>                        jndiContext.bind(TestCamelRouter.BEAN_NAME, new TestCamelRouter());
>                }
>                catch (Exception e)
>                {
>                        e.printStackTrace();
>                        return;
>                }
>                CamelContext context = new DefaultCamelContext(jndiContext);
>                ActiveMQConnectionFactory factory = new
> ActiveMQConnectionFactory("tcp://localhost:61616");
>                factory.setDispatchAsync(true);
>                context.addComponent("JMS-BROKER",
> JmsComponent.jmsComponentAutoAcknowledge(factory));
>                try
>                {
>                        context.addRoutes(new TestCamelRouter());
>                        context.start();
>                        while (true)
>                                Thread.sleep(60000);
>                }
>                catch (Exception e)
>                {
>                        e.printStackTrace();
>                        return;
>                }
>        }
>
> }
>
>
>
>
>
>
>
> huntc wrote:
>>
>> Can you please provide the route's specification in code or XML. This will
>> give us a better chance of answering your question.
>>
>> Having said that, sounds like you've got an internal seda queue... I would
>> have thought that an exception would be thrown once the queue becomes
>> full. By default a seda queue can only hold 1000 items.
>>
>> If it is a JMS queue, then would setting the time to live option be useful
>> to you?
>>
>
> --
> View this message in context: http://www.nabble.com/Router-Blocked-by-full-Destination-Route-Queue-tp22981898p23012582.html
> Sent from the Camel - Users (activemq) mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Apache Camel Reference Card:
http://refcardz.dzone.com/refcardz/enterprise-integration