Distributing routes on multiple camel instances

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Distributing routes on multiple camel instances

Sherwin Pinto
Hi All,

I am currently using the camel FTP component to download files and upload
them to s3. The number of routes are growing at an alarming rate i.e. each
route is reading from a separate FTP server and the servers are polled
multiple times a day. I currently have only one camel instance running,  so
I was wondering how I could add more instances, but distribute the routes
across multiple instances so that no one instance is running all routes.

Thanks In advance

Regards

Sherwin
Reply | Threaded
Open this post in threaded view
|

Re: Distributing routes on multiple camel instances

skin27
Hi Sherwin,

Since Camel 3.x multiple CamelContexts are not supported. See migration
guide:
https://github.com/apache/camel/blob/master/docs/user-manual/modules/ROOT/pages/camel-3-migration-guide.adoc

However it's possible to run multiple Camel applications. It depends on
your runtime how to implement it. Some examples

1) Standalone

When running Camel as standalone application just start two JVM's. So for
example route 1 to 10 is running in JVM proces 1 and route 11 to 20 in JVM
proces 2. Both can run on the same server or on different servers. You can
even have some kind of failover:

https://camel.apache.org/components/latest/eips/loadBalance-eip.html

2) Karaf

When you have for example Karaf (https://karaf.apache.org/) containers. You
can put some routes in one Karaf container and other routes in another.

3) Assimbly

I developed and use Assimbly (which runs in Spring boot) as a runtime for
some FTP --> S3 routes. There you can bundle routes per gateway (
https://github.com/assimbly/gateway).

4) Wildfly/EAP

In Wildfly you can pack camel routes as war files, so each war file run
within the Wildfly application server.

You might also want to check: https://tomd.xyz/multiple-camel-contexts/

As a sidenote: when you need more flexible and scalable solution you might
try to use message queueing like ActiveMQ (http://activemq.apache.org/) or
build a REST API with Camel (
https://camel.apache.org/components/latest/rest-component.html).

Regards,

Raymond










Op zo 17 mei 2020 om 02:59 schreef Sherwin Pinto <[hidden email]>:

> Hi All,
>
> I am currently using the camel FTP component to download files and upload
> them to s3. The number of routes are growing at an alarming rate i.e. each
> route is reading from a separate FTP server and the servers are polled
> multiple times a day. I currently have only one camel instance running,  so
> I was wondering how I could add more instances, but distribute the routes
> across multiple instances so that no one instance is running all routes.
>
> Thanks In advance
>
> Regards
>
> Sherwin
>
Reply | Threaded
Open this post in threaded view
|

Re: Distributing routes on multiple camel instances

Sherwin Pinto
Thanks! will look into the different methods.

Regards

Sherwin

On Mon, May 18, 2020 at 5:59 AM ski n <[hidden email]> wrote:

> Hi Sherwin,
>
> Since Camel 3.x multiple CamelContexts are not supported. See migration
> guide:
>
> https://github.com/apache/camel/blob/master/docs/user-manual/modules/ROOT/pages/camel-3-migration-guide.adoc
>
> However it's possible to run multiple Camel applications. It depends on
> your runtime how to implement it. Some examples
>
> 1) Standalone
>
> When running Camel as standalone application just start two JVM's. So for
> example route 1 to 10 is running in JVM proces 1 and route 11 to 20 in JVM
> proces 2. Both can run on the same server or on different servers. You can
> even have some kind of failover:
>
> https://camel.apache.org/components/latest/eips/loadBalance-eip.html
>
> 2) Karaf
>
> When you have for example Karaf (https://karaf.apache.org/) containers.
> You
> can put some routes in one Karaf container and other routes in another.
>
> 3) Assimbly
>
> I developed and use Assimbly (which runs in Spring boot) as a runtime for
> some FTP --> S3 routes. There you can bundle routes per gateway (
> https://github.com/assimbly/gateway).
>
> 4) Wildfly/EAP
>
> In Wildfly you can pack camel routes as war files, so each war file run
> within the Wildfly application server.
>
> You might also want to check: https://tomd.xyz/multiple-camel-contexts/
>
> As a sidenote: when you need more flexible and scalable solution you might
> try to use message queueing like ActiveMQ (http://activemq.apache.org/) or
> build a REST API with Camel (
> https://camel.apache.org/components/latest/rest-component.html).
>
> Regards,
>
> Raymond
>
>
>
>
>
>
>
>
>
>
> Op zo 17 mei 2020 om 02:59 schreef Sherwin Pinto <[hidden email]>:
>
> > Hi All,
> >
> > I am currently using the camel FTP component to download files and upload
> > them to s3. The number of routes are growing at an alarming rate i.e.
> each
> > route is reading from a separate FTP server and the servers are polled
> > multiple times a day. I currently have only one camel instance running,
> so
> > I was wondering how I could add more instances, but distribute the routes
> > across multiple instances so that no one instance is running all routes.
> >
> > Thanks In advance
> >
> > Regards
> >
> > Sherwin
> >
>