FileAsyncStressReadLockLockFileTest and maxMessagePerPoll

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

FileAsyncStressReadLockLockFileTest and maxMessagePerPoll

ltsallas

Can you explain a little bit more this  comment  in org.apache.camel.component.file.stress.FileAsyncStressReadLockLockFileTest?
                // leverage the fact that we can limit to max 50 files per poll
                // this will result in polling again and potentially picking up files
                // that already are in progress

 Is it safer to have a bigger or smaller  maxMessagePerPoll in order to avoid picking files that are already in process?  Is there a way to  guaranty that a file will never be picked twice?  
Reply | Threaded
Open this post in threaded view
|

Re: FileAsyncStressReadLockLockFileTest and maxMessagePerPoll. picking twice

ltsallas
I ran various org.apache.camel.component.file.stress.FileAsyncStressTest tests and when the initial files in the inbox folder are 2000 and maxMessagesPerPoll is 50, I always get files being processed twice with all readLock options, except rename.Mind that  I am using an additional move option to move processed files to an archive folder.

My route looks like

 from("file:target/filestress?maxMessagesPerPoll=50&readLock=markerFile&move=archive")
.routeId("foo").noAutoStartup().threads(2)

Should these test considered successful when having files processed twice? Should I open a jira for this?




Reply | Threaded
Open this post in threaded view
|

Re: FileAsyncStressReadLockLockFileTest and maxMessagePerPoll. picking twice

saltnlight5
It works for me when I use "preMove" into a staging folder first. Try this:

from("file:target/filestress?maxMessagesPerPoll=50&preMove=processing&move=archive")
Reply | Threaded
Open this post in threaded view
|

Re: FileAsyncStressReadLockLockFileTest and maxMessagePerPoll. picking twice

Afroze
Hi,

I want my camel file method to pick 5 files simultaneously and process .
using  maxMessagesPerPoll its only pick one file .Please can you help me on this.
Can you please explain me how it works
Reply | Threaded
Open this post in threaded view
|

Re: FileAsyncStressReadLockLockFileTest and maxMessagePerPoll. picking twice

ltsallas
This post was updated on .
In reply to this post by saltnlight5