Quantcast

scalable bus with multiple Camel instances

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

scalable bus with multiple Camel instances

unludo
This post was updated on .
My idea is to use camel to decouple modules. In order to support scalability and failover, I am wondering if the following architecture is adviced?

I have two applications with Camel embedded AppCamel1 and AppCamel2. Then I have standalone camel nodes Camel1 and Camel2.

AppCamel1 would have a route with fail-over/load balancing to Camel1 and Camel2. This way, if Camel1 crashes for example, Camel2 is used for failover.

Camel1 and 2 would do a REST call with the http component for example. Also there would be a request-reply from AppCamel1 up to camel1  or 2.

Is it a valid scenario?

What should I use to interconnect the different Camel instances (AppCamel1 to Camel1 or 2)? (I would like to know if it's possible to avoid another component like a jms server in the middle)

Thank you!

Also there: http://stackoverflow.com/questions/9943507/a-scalable-bus-with-multiple-camel-instances
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: scalable bus with multiple Camel instances

Christian Mueller
You have many possibilities. And the best solution depends on multiple
criteria:
- What's your payload?
- How big is it?
- Do you have ambitious performance requirements?
- Do you have ambitious throughput requirements?
- Do you also have to process the requests (later) if Camel1 and Camel2 is
not up?
- Is loosing messages acceptable?
- ...

In my company, we use ActiveMQ to decouple different services/applications.
We use persistent messaging to be sure we do not loose messages. It's fast
(we have "normal" performance requirements) and fits our requirements.

Best,
Christian

On Fri, Mar 30, 2012 at 3:02 PM, unludo <[hidden email]> wrote:

> My idea is to use camel to decouple modules. In order to support
> scalability
> and failover, I am wondering if the following architecture is adviced?
>
> I have two applications with Camel embedded AppCamel1 and AppCamel2. Then I
> have standalone camel nodes Camel1 and Camel2.
>
> AppCamel1 would have a route with fail-over/load balancing to Camel1 and
> AppCamel2. This way, if Camel1 crashes for example, Camel2 is used for
> failover.
>
> Camel1 and 2 would do a REST call with the http component for example. Also
> there would be a request-reply from AppCamel1 up to camel1  or 2.
>
> Is it a valid scenario?
>
> What should I use to interconnect the different Camel instances (AppCamel1
> to Camel1 or 2)? (I would like to know if it's possible to avoid another
> component like a jms server in the middle)
>
> Thank you!
>
> Also there:
>
> http://stackoverflow.com/questions/9943507/a-scalable-bus-with-multiple-camel-instances
>
> http://stackoverflow.com/questions/9943507/a-scalable-bus-with-multiple-camel-instances
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/scalable-bus-with-multiple-Camel-instances-tp5606593p5606593.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: scalable bus with multiple Camel instances

unludo
This post was updated on .
Thanks for your answer Christian.

I'd like to have a response as fast as possible and can afford loosing messages. Also the payload may be big (<100MB)

Actually I would like to see if I can avoid activemq in the middle.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: scalable bus with multiple Camel instances

Christian Mueller
If performance is the most important requirement, I would go for
camel-netty, camel-mina2 or camel-mina.

One remark: I really ofter hear "as fast as possible" but  in most cases,
it doesn't matter whether it takes 10ms or 50ms longer for one request. If
this is the same for your requirement, have a look at some other protocols
you may are more familiar with, e.g.: camel-http(4), camel-cxf (for JAX-RS
or JAX-WS), ...

But in all this cases, your applications/services are not so loosely
couplet (as possible). If Camel1 and/or Camel2 is down, AppCamel1 and/or
AppCamel2 will recognize it. This is not the case with a message broker in
between.
What's your strategy if you have to scale CamelX? Adding Camel3, Camel4,
...and so on, I asume. But in this case, you have to touch AppCamel1 and
AppCamel2 again. With an message broker, simple add more consumers onCamel1
and Camel2 . And if this is not enough, add Camel3 and Camel4. But in this
case, you don't have to touch AppCamel1 or AppCamel2.

Best,
Christian

On Fri, Mar 30, 2012 at 10:04 PM, unludo <[hidden email]> wrote:

> Thanks for your answer.
>
> I'd like to have a response as fast as possible and can afford loosing
> messages.
>
> Actually I would like to see if I can avoid activemq in the middle.
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/scalable-bus-with-multiple-Camel-instances-tp5606593p5607615.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: scalable bus with multiple Camel instances

unludo
I see that Camel mainly makes sense with a broker like ActiveMQ, as a core for a scalable/decoupled system.

Thanks for helping me understanding this!
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: scalable bus with multiple Camel instances

Christian Mueller
It's not "Camel mainly makes sense with a broker like ActiveMQ". There are
many other easons why you should use Camel also without a message broker.
I want only let you know
1) what the benefit of a message broker is and
2) what customers says is sometimes not what they really want... ;-)

Best,
Christian

On Mon, Apr 2, 2012 at 4:25 PM, unludo <[hidden email]> wrote:

> I see that Camel mainly makes sense with a broker like ActiveMQ, as a core
> for a scalable/decoupled system.
>
> Thanks for helping me understanding this!
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/scalable-bus-with-multiple-Camel-instances-tp5606593p5612599.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: scalable bus with multiple Camel instances

Claus Ibsen-2
Hi

You can use in-memory only communication across JVMs such as hazelcast

Jean wrote a blog about this
http://blog.nanthrax.net/2012/02/communication-between-two-remote-camel-routes-using-karaf-cellar/

There is also DOSGi - distributed osgi. But that of course requires OSGi.

There is a really fast implementation in Fuse Fabric:
http://gnodet.blogspot.se/2011/06/distributed-osgi-in-fabric.html

And I think there is a DOSGi implementation in Apache CXF.

And speaking of Fuse Fabric, then it will offer more easy to
configure, manage, clusters of JVMs.
Which discovery and failover, and all the stuff like that.
https://github.com/fusesource/fuse

At Apache we have many links to various blogs and articles, there is
some great ones among them.
http://camel.apache.org/articles


On Mon, Apr 2, 2012 at 7:00 PM, Christian Müller
<[hidden email]> wrote:

> It's not "Camel mainly makes sense with a broker like ActiveMQ". There are
> many other easons why you should use Camel also without a message broker.
> I want only let you know
> 1) what the benefit of a message broker is and
> 2) what customers says is sometimes not what they really want... ;-)
>
> Best,
> Christian
>
> On Mon, Apr 2, 2012 at 4:25 PM, unludo <[hidden email]> wrote:
>
>> I see that Camel mainly makes sense with a broker like ActiveMQ, as a core
>> for a scalable/decoupled system.
>>
>> Thanks for helping me understanding this!
>>
>> --
>> View this message in context:
>> http://camel.465427.n5.nabble.com/scalable-bus-with-multiple-Camel-instances-tp5606593p5612599.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>



--
Claus Ibsen
-----------------
CamelOne 2012 Conference, May 15-16, 2012: http://camelone.com
FuseSource
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: scalable bus with multiple Camel instances

unludo
Thank you Claus, that's quite inspiring! :)
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: scalable bus with multiple Camel instances

whisky
could also look on using akka for the scaling and camel for the
request/routing considered as a 'unit of work' (i.e a black box for
individual operations like pulling from queues..)

akka and camel blend perfectly well together.


On Tue, Apr 3, 2012 at 2:42 PM, unludo <[hidden email]> wrote:

> Thank you Claus, that's quite inspiring! :)
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/scalable-bus-with-multiple-Camel-instances-tp5606593p5614993.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Loading...