SJMS2 vs JMS components for transfer messages from and to ActiveMQ Artemis

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

SJMS2 vs JMS components for transfer messages from and to ActiveMQ Artemis

Mikhail Lukyanov
I am trying to find the fastest way to transfer messages from one ActiveMQ
queue to another Artemis. And I thought that the SJMS2 component would be
faster than traditional JMS, but routing with JMS is 2.5 times faster (20
000 vs 8000 msg/s). I use Camel version 2.20.2 and Artemis version 2.11.0.

Route with JMS

import org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.JndiRegistry;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.junit.Test;
import org.messaginghub.pooled.jms.JmsPoolConnectionFactory;

import javax.jms.ConnectionFactory;
import java.util.concurrent.TimeUnit;

public class JMSTransferTest extends CamelTestSupport {
    @Test
    public void testArtemis() throws Exception {
        TimeUnit.SECONDS.sleep(100);
    }

    @Override
    protected RouteBuilder createRouteBuilder() throws Exception {
        return new RouteBuilder() {
            public void configure() {

from("jms://TEST.IN?connectionFactory=#artemisCF&concurrentConsumers=200")
                        .to("jms://TEST.OUT?connectionFactory=#artemisCF");
            }
        };
    }

    @Override
    protected JndiRegistry createRegistry() throws Exception {
        JndiRegistry registry = super.createRegistry();

        final ConnectionFactory connFactory = new
ActiveMQConnectionFactory("tcp://localhost:61622");
        final ConnectionFactory connFactoryDeadLeatter = new
ActiveMQConnectionFactory("tcp://localhost:61622");
        JmsPoolConnectionFactory pooledConnectionFactory = new
JmsPoolConnectionFactory();

        pooledConnectionFactory.setConnectionFactory(connFactory);
        pooledConnectionFactory.setMaxConnections(20);
        pooledConnectionFactory.setMaxSessionsPerConnection(100);
        registry.bind("artemisCF", pooledConnectionFactory);
        registry.bind("deadLetterCF", connFactoryDeadLeatter);
        return registry;
    }
}

Route with SJMS2, other settings as in the code above

 @Override
    protected RouteBuilder createRouteBuilder() throws Exception {
        return new RouteBuilder() {
            public void configure() {

from("sjms2://SJMS.CONSUMER.TEST?connectionFactory=#artemisCF&consumerCount=200&asyncStartListener=true")

.to("sjms2://OUT.SJMS.CONSUMER.TEST?connectionFactory=#artemisCF");
            }
        };
    }

How can I use the SJMS2 component to get the same speeds as JMS component?
Reply | Threaded
Open this post in threaded view
|

Re: SJMS2 vs JMS components for transfer messages from and to ActiveMQ Artemis

Andrea Cosentino-3
2.20.2 is really old. You can try with newer versions.

Il giorno mer 26 feb 2020 alle ore 13:59 Mikhail Lukyanov <
[hidden email]> ha scritto:

> I am trying to find the fastest way to transfer messages from one ActiveMQ
> queue to another Artemis. And I thought that the SJMS2 component would be
> faster than traditional JMS, but routing with JMS is 2.5 times faster (20
> 000 vs 8000 msg/s). I use Camel version 2.20.2 and Artemis version 2.11.0.
>
> Route with JMS
>
> import org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.JndiRegistry;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import org.messaginghub.pooled.jms.JmsPoolConnectionFactory;
>
> import javax.jms.ConnectionFactory;
> import java.util.concurrent.TimeUnit;
>
> public class JMSTransferTest extends CamelTestSupport {
>     @Test
>     public void testArtemis() throws Exception {
>         TimeUnit.SECONDS.sleep(100);
>     }
>
>     @Override
>     protected RouteBuilder createRouteBuilder() throws Exception {
>         return new RouteBuilder() {
>             public void configure() {
>
> from("jms://TEST.IN?connectionFactory=#artemisCF&concurrentConsumers=200")
>                         .to("jms://TEST.OUT?connectionFactory=#artemisCF");
>             }
>         };
>     }
>
>     @Override
>     protected JndiRegistry createRegistry() throws Exception {
>         JndiRegistry registry = super.createRegistry();
>
>         final ConnectionFactory connFactory = new
> ActiveMQConnectionFactory("tcp://localhost:61622");
>         final ConnectionFactory connFactoryDeadLeatter = new
> ActiveMQConnectionFactory("tcp://localhost:61622");
>         JmsPoolConnectionFactory pooledConnectionFactory = new
> JmsPoolConnectionFactory();
>
>         pooledConnectionFactory.setConnectionFactory(connFactory);
>         pooledConnectionFactory.setMaxConnections(20);
>         pooledConnectionFactory.setMaxSessionsPerConnection(100);
>         registry.bind("artemisCF", pooledConnectionFactory);
>         registry.bind("deadLetterCF", connFactoryDeadLeatter);
>         return registry;
>     }
> }
>
> Route with SJMS2, other settings as in the code above
>
>  @Override
>     protected RouteBuilder createRouteBuilder() throws Exception {
>         return new RouteBuilder() {
>             public void configure() {
>
>
> from("sjms2://SJMS.CONSUMER.TEST?connectionFactory=#artemisCF&consumerCount=200&asyncStartListener=true")
>
> .to("sjms2://OUT.SJMS.CONSUMER.TEST?connectionFactory=#artemisCF");
>             }
>         };
>     }
>
> How can I use the SJMS2 component to get the same speeds as JMS component?
>
Reply | Threaded
Open this post in threaded view
|

Re: SJMS2 vs JMS components for transfer messages from and to ActiveMQ Artemis

Mikhail Lukyanov
I updated Camel to the new version 2.25.0 and ran the tests described. The speeds did not change
For SJMS2 ~8 000 msg/s

For common JMS ~20 000 msg/s


 

ср, 26 февр. 2020 г. в 16:05, Andrea Cosentino <[hidden email]>:
2.20.2 is really old. You can try with newer versions.

Il giorno mer 26 feb 2020 alle ore 13:59 Mikhail Lukyanov <
[hidden email]> ha scritto:

> I am trying to find the fastest way to transfer messages from one ActiveMQ
> queue to another Artemis. And I thought that the SJMS2 component would be
> faster than traditional JMS, but routing with JMS is 2.5 times faster (20
> 000 vs 8000 msg/s). I use Camel version 2.20.2 and Artemis version 2.11.0.
>
> Route with JMS
>
> import org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.JndiRegistry;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import org.messaginghub.pooled.jms.JmsPoolConnectionFactory;
>
> import javax.jms.ConnectionFactory;
> import java.util.concurrent.TimeUnit;
>
> public class JMSTransferTest extends CamelTestSupport {
>     @Test
>     public void testArtemis() throws Exception {
>         TimeUnit.SECONDS.sleep(100);
>     }
>
>     @Override
>     protected RouteBuilder createRouteBuilder() throws Exception {
>         return new RouteBuilder() {
>             public void configure() {
>
> from("jms://TEST.IN?connectionFactory=#artemisCF&concurrentConsumers=200")
>                         .to("jms://TEST.OUT?connectionFactory=#artemisCF");
>             }
>         };
>     }
>
>     @Override
>     protected JndiRegistry createRegistry() throws Exception {
>         JndiRegistry registry = super.createRegistry();
>
>         final ConnectionFactory connFactory = new
> ActiveMQConnectionFactory("tcp://localhost:61622");
>         final ConnectionFactory connFactoryDeadLeatter = new
> ActiveMQConnectionFactory("tcp://localhost:61622");
>         JmsPoolConnectionFactory pooledConnectionFactory = new
> JmsPoolConnectionFactory();
>
>         pooledConnectionFactory.setConnectionFactory(connFactory);
>         pooledConnectionFactory.setMaxConnections(20);
>         pooledConnectionFactory.setMaxSessionsPerConnection(100);
>         registry.bind("artemisCF", pooledConnectionFactory);
>         registry.bind("deadLetterCF", connFactoryDeadLeatter);
>         return registry;
>     }
> }
>
> Route with SJMS2, other settings as in the code above
>
>  @Override
>     protected RouteBuilder createRouteBuilder() throws Exception {
>         return new RouteBuilder() {
>             public void configure() {
>
>
> from("sjms2://SJMS.CONSUMER.TEST?connectionFactory=#artemisCF&consumerCount=200&asyncStartListener=true")
>
> .to("sjms2://OUT.SJMS.CONSUMER.TEST?connectionFactory=#artemisCF");
>             }
>         };
>     }
>
> How can I use the SJMS2 component to get the same speeds as JMS component?
>


--
С наилучшими пожеланиями, Лукьянов Михаил
Моб: +7-909-69-71-547​
Reply | Threaded
Open this post in threaded view
|

Re: SJMS2 vs JMS components for transfer messages from and to ActiveMQ Artemis

Claus Ibsen-2
In reply to this post by Mikhail Lukyanov
Hi

200 consumers is too much. That instead makes it slower as you have
200 consumers racing for messages. Instead try to find a lower balance
that is closer to cpu cores etc.

Also often a JMS client has a prefetch buffer (or some concept like
this) which means a consumer may pre-download 1000 messages and then
the other 199 consumers cant process these messages. So you need to
tweak this option too.

Also if you have too many consumers and remote network connections
then you get too chatty over IO etc. So its all about tuning depending
on use-cases.

spring-jms has a thread pool built in that can automatic grow/shrink
depending on load, and this can explain why its out of the box without
tuning can appear to be faster.

Writing such logic is a bit more complex and this hasnt been added to
sjms. I created a ticket about this
https://issues.apache.org/jira/browse/CAMEL-14637

You can get in touch with commercial Camel supports as there are
companies and consultants that has great experience with JMS brokers
and Camel and to get them tuned to very high performance. The settings
for JMV and OS and hardware can all make a big difference.


On Wed, Feb 26, 2020 at 1:59 PM Mikhail Lukyanov <[hidden email]> wrote:

>
> I am trying to find the fastest way to transfer messages from one ActiveMQ
> queue to another Artemis. And I thought that the SJMS2 component would be
> faster than traditional JMS, but routing with JMS is 2.5 times faster (20
> 000 vs 8000 msg/s). I use Camel version 2.20.2 and Artemis version 2.11.0.
>
> Route with JMS
>
> import org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory;
> import org.apache.camel.builder.RouteBuilder;
> import org.apache.camel.impl.JndiRegistry;
> import org.apache.camel.test.junit4.CamelTestSupport;
> import org.junit.Test;
> import org.messaginghub.pooled.jms.JmsPoolConnectionFactory;
>
> import javax.jms.ConnectionFactory;
> import java.util.concurrent.TimeUnit;
>
> public class JMSTransferTest extends CamelTestSupport {
>     @Test
>     public void testArtemis() throws Exception {
>         TimeUnit.SECONDS.sleep(100);
>     }
>
>     @Override
>     protected RouteBuilder createRouteBuilder() throws Exception {
>         return new RouteBuilder() {
>             public void configure() {
>
> from("jms://TEST.IN?connectionFactory=#artemisCF&concurrentConsumers=200")
>                         .to("jms://TEST.OUT?connectionFactory=#artemisCF");
>             }
>         };
>     }
>
>     @Override
>     protected JndiRegistry createRegistry() throws Exception {
>         JndiRegistry registry = super.createRegistry();
>
>         final ConnectionFactory connFactory = new
> ActiveMQConnectionFactory("tcp://localhost:61622");
>         final ConnectionFactory connFactoryDeadLeatter = new
> ActiveMQConnectionFactory("tcp://localhost:61622");
>         JmsPoolConnectionFactory pooledConnectionFactory = new
> JmsPoolConnectionFactory();
>
>         pooledConnectionFactory.setConnectionFactory(connFactory);
>         pooledConnectionFactory.setMaxConnections(20);
>         pooledConnectionFactory.setMaxSessionsPerConnection(100);
>         registry.bind("artemisCF", pooledConnectionFactory);
>         registry.bind("deadLetterCF", connFactoryDeadLeatter);
>         return registry;
>     }
> }
>
> Route with SJMS2, other settings as in the code above
>
>  @Override
>     protected RouteBuilder createRouteBuilder() throws Exception {
>         return new RouteBuilder() {
>             public void configure() {
>
> from("sjms2://SJMS.CONSUMER.TEST?connectionFactory=#artemisCF&consumerCount=200&asyncStartListener=true")
>
> .to("sjms2://OUT.SJMS.CONSUMER.TEST?connectionFactory=#artemisCF");
>             }
>         };
>     }
>
> How can I use the SJMS2 component to get the same speeds as JMS component?



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: SJMS2 vs JMS components for transfer messages from and to ActiveMQ Artemis

Mikhail Lukyanov
Hello Claus, thanks for the information, I will continue to search and
select the optimal parameters.

вс, 1 мар. 2020 г., 13:18 Claus Ibsen <[hidden email]>:

> Hi
>
> 200 consumers is too much. That instead makes it slower as you have
> 200 consumers racing for messages. Instead try to find a lower balance
> that is closer to cpu cores etc.
>
> Also often a JMS client has a prefetch buffer (or some concept like
> this) which means a consumer may pre-download 1000 messages and then
> the other 199 consumers cant process these messages. So you need to
> tweak this option too.
>
> Also if you have too many consumers and remote network connections
> then you get too chatty over IO etc. So its all about tuning depending
> on use-cases.
>
> spring-jms has a thread pool built in that can automatic grow/shrink
> depending on load, and this can explain why its out of the box without
> tuning can appear to be faster.
>
> Writing such logic is a bit more complex and this hasnt been added to
> sjms. I created a ticket about this
> https://issues.apache.org/jira/browse/CAMEL-14637
>
> You can get in touch with commercial Camel supports as there are
> companies and consultants that has great experience with JMS brokers
> and Camel and to get them tuned to very high performance. The settings
> for JMV and OS and hardware can all make a big difference.
>
>
> On Wed, Feb 26, 2020 at 1:59 PM Mikhail Lukyanov <[hidden email]>
> wrote:
> >
> > I am trying to find the fastest way to transfer messages from one
> ActiveMQ
> > queue to another Artemis. And I thought that the SJMS2 component would be
> > faster than traditional JMS, but routing with JMS is 2.5 times faster (20
> > 000 vs 8000 msg/s). I use Camel version 2.20.2 and Artemis version
> 2.11.0.
> >
> > Route with JMS
> >
> > import org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory;
> > import org.apache.camel.builder.RouteBuilder;
> > import org.apache.camel.impl.JndiRegistry;
> > import org.apache.camel.test.junit4.CamelTestSupport;
> > import org.junit.Test;
> > import org.messaginghub.pooled.jms.JmsPoolConnectionFactory;
> >
> > import javax.jms.ConnectionFactory;
> > import java.util.concurrent.TimeUnit;
> >
> > public class JMSTransferTest extends CamelTestSupport {
> >     @Test
> >     public void testArtemis() throws Exception {
> >         TimeUnit.SECONDS.sleep(100);
> >     }
> >
> >     @Override
> >     protected RouteBuilder createRouteBuilder() throws Exception {
> >         return new RouteBuilder() {
> >             public void configure() {
> >
> > from("jms://TEST.IN?connectionFactory=#artemisCF&concurrentConsumers=200
> ")
> >
>  .to("jms://TEST.OUT?connectionFactory=#artemisCF");
> >             }
> >         };
> >     }
> >
> >     @Override
> >     protected JndiRegistry createRegistry() throws Exception {
> >         JndiRegistry registry = super.createRegistry();
> >
> >         final ConnectionFactory connFactory = new
> > ActiveMQConnectionFactory("tcp://localhost:61622");
> >         final ConnectionFactory connFactoryDeadLeatter = new
> > ActiveMQConnectionFactory("tcp://localhost:61622");
> >         JmsPoolConnectionFactory pooledConnectionFactory = new
> > JmsPoolConnectionFactory();
> >
> >         pooledConnectionFactory.setConnectionFactory(connFactory);
> >         pooledConnectionFactory.setMaxConnections(20);
> >         pooledConnectionFactory.setMaxSessionsPerConnection(100);
> >         registry.bind("artemisCF", pooledConnectionFactory);
> >         registry.bind("deadLetterCF", connFactoryDeadLeatter);
> >         return registry;
> >     }
> > }
> >
> > Route with SJMS2, other settings as in the code above
> >
> >  @Override
> >     protected RouteBuilder createRouteBuilder() throws Exception {
> >         return new RouteBuilder() {
> >             public void configure() {
> >
> >
> from("sjms2://SJMS.CONSUMER.TEST?connectionFactory=#artemisCF&consumerCount=200&asyncStartListener=true")
> >
> > .to("sjms2://OUT.SJMS.CONSUMER.TEST?connectionFactory=#artemisCF");
> >             }
> >         };
> >     }
> >
> > How can I use the SJMS2 component to get the same speeds as JMS
> component?
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>