Apache camel running on multiple nodes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Apache camel running on multiple nodes

SRAVAN KUMAR
Hi Team,

 I want to run the apache camel file route on multiple nodes and the
source  folder is same for both nodes and expecting route to process files
parallelly

> when I tried the running camel on multiple nodes, files are processing in
a round-robin fashion,
> Example:- Node1 will pick 1 file (Node2 is waiting for node1 to process)
> after node1 processing, node2 will pick a new file (Node1 will wait for
node2 to process file)


MY FILE ROUTE :

from("file:\\\folder?maxMessagesPerPoll=1&delete=true&
readLockMinLength=0&readLock=changed&readLockTimeout=1000&r
eadLockCheckInterval=1000)
Reply | Threaded
Open this post in threaded view
|

Re: Apache camel running on multiple nodes

Claus Ibsen-2
Hi

You have multiple nodes compete for the same files, and the changed
read lock will "look at one file at a time" and therefore there a big
chance the compete for the same file.
You can sort by random to let the read lock not clash for same files.

Also to process concurrently you can use a clustered read lock also,
there is a chapter on this in the CiA2 book.

On Mon, Sep 28, 2020 at 1:21 PM SRAVAN KUMAR <[hidden email]> wrote:

>
> Hi Team,
>
>  I want to run the apache camel file route on multiple nodes and the
> source  folder is same for both nodes and expecting route to process files
> parallelly
>
> > when I tried the running camel on multiple nodes, files are processing in
> a round-robin fashion,
> > Example:- Node1 will pick 1 file (Node2 is waiting for node1 to process)
> > after node1 processing, node2 will pick a new file (Node1 will wait for
> node2 to process file)
>
>
> MY FILE ROUTE :
>
> from("file:\\\folder?maxMessagesPerPoll=1&delete=true&
> readLockMinLength=0&readLock=changed&readLockTimeout=1000&r
> eadLockCheckInterval=1000)



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2