camel - Activemq performance suggestions

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

camel - Activemq performance suggestions

kiranreddykasa
Hi,

Can anyone suggest any performance improvement techniques for camel-activemq

We have tried connection pooling mentioned in this page http://camel.apache.org/activemq.html

Here is my xml and route configuration

from("netty:tcp://10.44.71.187:7000?textline=true").threads(800,800)
.bean(MainDummyProcessor.class)
.to("activemq:CAMEL_ONE_QUEUE?testConnectionOnStartup=true&replyTo=CAMEL_ONE_QUEUE_REP&replyToType=Exclusive");
                               
from("activemq:CAMEL_ONE_QUEUE?testConnectionOnStartup=true&concurrentConsumers=500&maxConcurrentConsumers=500&asyncConsumer=true")
.to("netty:tcp://10.44.71.67:7004?textline=true");



        <bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
                <property name="brokerURL" value="tcp://10.44.71.85:61616" />
        </bean>
        <bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory"
                init-method="start" destroy-method="stop">
                <property name="maxConnections" value="100" />
                <property name="connectionFactory" ref="jmsConnectionFactory" />
        </bean>

        <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
                <property name="connectionFactory" ref="pooledConnectionFactory" />
                <property name="concurrentConsumers" value="100" />
        </bean>

        <bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
                <property name="configuration" ref="jmsConfig" />
        </bean>



In the same route Without using activemq we are getting much high tps(~2500), whereas with activemq its very less (~200) .

Should we use any extra properties while configuring activemq?

camel version : 2.10.3
activemq : 5.7.0
Regards

kiran Reddy
Reply | Threaded
Open this post in threaded view
|

Re: camel - Activemq performance suggestions

ceposta
When you say "without using activemq" what do you use in its place? the
"direct" component?
Have you debugged a little to see where the processing is slowing down? ie,
where are the bottlenecks? On the camel side? On the broker side?


On Fri, Jun 21, 2013 at 2:39 AM, kiranreddykasa <[hidden email]>wrote:

> Hi,
>
> Can anyone suggest any performance improvement techniques for
> camel-activemq
>
> We have tried connection pooling mentioned in this page
> http://camel.apache.org/activemq.html
> <http://camel.apache.org/activemq.html>
>
> Here is my xml and route configuration
>
> from("netty:tcp://10.44.71.187:7000?textline=true").threads(800,800)
> .bean(MainDummyProcessor.class)
>
> .to("activemq:CAMEL_ONE_QUEUE?testConnectionOnStartup=true&replyTo=CAMEL_ONE_QUEUE_REP&replyToType=Exclusive");
>
>
> from("activemq:CAMEL_ONE_QUEUE?testConnectionOnStartup=true&concurrentConsumers=500&maxConcurrentConsumers=500&asyncConsumer=true")
> .to("netty:tcp://10.44.71.67:7004?textline=true");
>
>
>
>         <bean id="jmsConnectionFactory"
> class="org.apache.activemq.ActiveMQConnectionFactory">
>                 <property name="brokerURL" value="tcp://10.44.71.85:61616"
> />
>         </bean>
>         <bean id="pooledConnectionFactory"
> class="org.apache.activemq.pool.PooledConnectionFactory"
>                 init-method="start" destroy-method="stop">
>                 <property name="maxConnections" value="100" />
>                 <property name="connectionFactory"
> ref="jmsConnectionFactory" />
>         </bean>
>
>         <bean id="jmsConfig"
> class="org.apache.camel.component.jms.JmsConfiguration">
>                 <property name="connectionFactory"
> ref="pooledConnectionFactory" />
>                 <property name="concurrentConsumers" value="100" />
>         </bean>
>
>         <bean id="activemq"
> class="org.apache.activemq.camel.component.ActiveMQComponent">
>                 <property name="configuration" ref="jmsConfig" />
>         </bean>
>
>
>
> In the same route Without using activemq we are getting much high
> tps(~2500), whereas with activemq its very less (~200) .
>
> Should we use any extra properties while configuring activemq?
>
> camel version : 2.10.3
> activemq : 5.7.0
>
>
>
>
> -----
> Regards
>
> kiran Reddy
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/camel-Activemq-performance-suggestions-tp5734508.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>



--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re: camel - Activemq performance suggestions

kiranreddykasa
Hi

Without activemq in the sense direct tcp to tcp.

No other queue in the middle.

from("netty:tcp://10.44.71.187:7000?textline=true").threads(800,800) .bean(MainDummyProcessor.class)
.to("netty:tcp://10.44.71.67:7004?textline=true");

Regards

kiran Reddy
Reply | Threaded
Open this post in threaded view
|

Re: camel - Activemq performance suggestions

ceposta
JMS default is to do persistent messaging. Will add quite a bit more
overhead than just straight sockets especially if you have a slow disk.
You'll have to consider your use case needs (level of message durability,
message size, throughput, etc, etc)... take a look at tuning activemq:

http://activemq.apache.org/performance-tuning.html


On Fri, Jun 21, 2013 at 9:13 AM, kiranreddykasa <[hidden email]>wrote:

> Hi
>
> Without activemq in the sense direct tcp to tcp.
>
> No other queue in the middle.
>
> from("netty:tcp://10.44.71.187:7000?textline=true").threads(800,800)
> .bean(MainDummyProcessor.class)
> .to("netty:tcp://10.44.71.67:7004?textline=true");
>
>
>
>
>
> -----
> Regards
>
> kiran Reddy
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/camel-Activemq-performance-suggestions-tp5734508p5734533.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>



--
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re: camel - Activemq performance suggestions

kiranreddykasa
Ya I agree jms will add little overhead,
But based on the results I got TCP is 10x faster.
Generally will there be this much difference??
Regards

kiran Reddy
Reply | Threaded
Open this post in threaded view
|

Re: camel - Activemq performance suggestions

Claus Ibsen-2
In reply to this post by kiranreddykasa
Hi

You should not really use threads(800,800) as the netty consumer has
its thread pool already. So you just end up with too many thread
pools.


And you can use concurrent consumer on the reply "side" when doing
request/reply, eg

.to("activemq:CAMEL_ONE_QUEUE?testConnectionOnStartup=true&replyTo=CAMEL_ONE_QUEUE_REP&replyToType=Exclusive");

Can have these options as well

.to("activemq:CAMEL_ONE_QUEUE?testConnectionOnStartup=true&replyTo=CAMEL_ONE_QUEUE_REP&replyToType=Exclusive&concurrentConsumers=100");

On Fri, Jun 21, 2013 at 8:39 AM, kiranreddykasa <[hidden email]> wrote:

> Hi,
>
> Can anyone suggest any performance improvement techniques for camel-activemq
>
> We have tried connection pooling mentioned in this page
> http://camel.apache.org/activemq.html
> <http://camel.apache.org/activemq.html>
>
> Here is my xml and route configuration
>
> from("netty:tcp://10.44.71.187:7000?textline=true").threads(800,800)
> .bean(MainDummyProcessor.class)
> .to("activemq:CAMEL_ONE_QUEUE?testConnectionOnStartup=true&replyTo=CAMEL_ONE_QUEUE_REP&replyToType=Exclusive");
>
> from("activemq:CAMEL_ONE_QUEUE?testConnectionOnStartup=true&concurrentConsumers=500&maxConcurrentConsumers=500&asyncConsumer=true")
> .to("netty:tcp://10.44.71.67:7004?textline=true");
>
>
>
>         <bean id="jmsConnectionFactory"
> class="org.apache.activemq.ActiveMQConnectionFactory">
>                 <property name="brokerURL" value="tcp://10.44.71.85:61616" />
>         </bean>
>         <bean id="pooledConnectionFactory"
> class="org.apache.activemq.pool.PooledConnectionFactory"
>                 init-method="start" destroy-method="stop">
>                 <property name="maxConnections" value="100" />
>                 <property name="connectionFactory" ref="jmsConnectionFactory" />
>         </bean>
>
>         <bean id="jmsConfig"
> class="org.apache.camel.component.jms.JmsConfiguration">
>                 <property name="connectionFactory" ref="pooledConnectionFactory" />
>                 <property name="concurrentConsumers" value="100" />
>         </bean>
>
>         <bean id="activemq"
> class="org.apache.activemq.camel.component.ActiveMQComponent">
>                 <property name="configuration" ref="jmsConfig" />
>         </bean>
>
>
>
> In the same route Without using activemq we are getting much high
> tps(~2500), whereas with activemq its very less (~200) .
>
> Should we use any extra properties while configuring activemq?
>
> camel version : 2.10.3
> activemq : 5.7.0
>
>
>
>
> -----
> Regards
>
> kiran Reddy
> --
> View this message in context: http://camel.465427.n5.nabble.com/camel-Activemq-performance-suggestions-tp5734508.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



--
Claus Ibsen
-----------------
www.camelone.org: The open source integration conference.

Red Hat, Inc.
FuseSource is now part of Red Hat
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Reply | Threaded
Open this post in threaded view
|

Re: camel - Activemq performance suggestions

kiranreddykasa
In reply to this post by kiranreddykasa
Hi,

We should not use threads options at all or we should have some limited number of threads  on netty??

And in my xml configuration i have already given concurrentConsumers as 100.

 <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
                <property name="connectionFactory" ref="pooledConnectionFactory" />
                <property name="concurrentConsumers" value="100" />
  </bean>

So on reply queue also i can see 100 consumers are listening by the above config.
Are there any other properties ??
Regards

kiran Reddy
Reply | Threaded
Open this post in threaded view
|

Re: camel - Activemq performance suggestions

kiranreddykasa
Hi ,

By using the following property 'useAsyncSend' activemq performance  has been improved.

Regards

kiran Reddy