Running Same set of Camel routes (Camel Context) on multiple nodes

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Running Same set of Camel routes (Camel Context) on multiple nodes

Sherwin Pinto
Hi,

I am currently running a single instance of Camel. Routes are set up to pull files from ftp servers.

Camel Version: 2.15.3

Running as a standalone application using spring

I am using an Idempotent Consumer (JpaMessageIdRepository) to prevent downloading duplicate files.

We are looking to scale (run multiple nodes) and also provide fail over (incase there is an issue with the instance)

What would be the best method to accomplish this ?

Would starting another instance work ? Basically both instances compete for the routes and if one finds that the other has already downloaded the file (message present in the database) then it will try to download another file from the ftp server ?

Thanks

Regards,

Sherwin

Reply | Threaded
Open this post in threaded view
|

Re: Running Same set of Camel routes (Camel Context) on multiple nodes

yogu13
if you are using timer to trigger the poll for file transfer then quartz could be used, quartz has cluster support.

Regards,
-Yogesh
Reply | Threaded
Open this post in threaded view
|

Re: Running Same set of Camel routes (Camel Context) on multiple nodes

Sherwin Pinto
Hi Yogesh,

Thanks for your response. I am evaluating our options . Using quartz in a clustered environment is definitely an option . Are there any other configurations you would recommend ?

Thanks

Sherwin

> On May 17, 2016, at 7:25 AM, yogu13 <[hidden email]> wrote:
>
> if you are using timer to trigger the poll for file transfer then quartz
> could be used, quartz has cluster support.
>
> Regards,
> -Yogesh
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Running-Same-set-of-Camel-routes-Camel-Context-on-multiple-nodes-tp5782680p5782702.html
> Sent from the Camel - Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Running Same set of Camel routes (Camel Context) on multiple nodes

Karel Bernolet
Have you looked into the leader election integration pattern?
I think camel-zookeeper and camel-hazelcast support it.

On Tue, May 17, 2016 at 4:56 PM, Sherwin Pinto <[hidden email]> wrote:

> Hi Yogesh,
>
> Thanks for your response. I am evaluating our options . Using quartz in a
> clustered environment is definitely an option . Are there any other
> configurations you would recommend ?
>
> Thanks
>
> Sherwin
>
> > On May 17, 2016, at 7:25 AM, yogu13 <[hidden email]> wrote:
> >
> > if you are using timer to trigger the poll for file transfer then quartz
> > could be used, quartz has cluster support.
> >
> > Regards,
> > -Yogesh
> >
> >
> >
> > --
> > View this message in context:
> http://camel.465427.n5.nabble.com/Running-Same-set-of-Camel-routes-Camel-Context-on-multiple-nodes-tp5782680p5782702.html
> > Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Running Same set of Camel routes (Camel Context) on multiple nodes

Sherwin Pinto
Thanks Karel … I had taken a look at it. Will research it in more detail

> On May 17, 2016, at 11:25 AM, Karel Bernolet <[hidden email]> wrote:
>
> Have you looked into the leader election integration pattern?
> I think camel-zookeeper and camel-hazelcast support it.
>
> On Tue, May 17, 2016 at 4:56 PM, Sherwin Pinto <[hidden email]> wrote:
>
>> Hi Yogesh,
>>
>> Thanks for your response. I am evaluating our options . Using quartz in a
>> clustered environment is definitely an option . Are there any other
>> configurations you would recommend ?
>>
>> Thanks
>>
>> Sherwin
>>
>>> On May 17, 2016, at 7:25 AM, yogu13 <[hidden email]> wrote:
>>>
>>> if you are using timer to trigger the poll for file transfer then quartz
>>> could be used, quartz has cluster support.
>>>
>>> Regards,
>>> -Yogesh
>>>
>>>
>>>
>>> --
>>> View this message in context:
>> http://camel.465427.n5.nabble.com/Running-Same-set-of-Camel-routes-Camel-Context-on-multiple-nodes-tp5782680p5782702.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>