Restricting the duplicate files to be added into idempotent repository irrespective of cacheSize value

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Restricting the duplicate files to be added into idempotent repository irrespective of cacheSize value

arvind
Hello,

I am using camel version 2.9.2 . Can anyone help me to know that, How to restrict duplicate files adding into idempotent repository irrespective of cache size , means  any number of files in source directory but no files should be added twice to the repository .

<bean id="fileStore"
                class="org.apache.camel.processor.idempotent.FileIdempotentRepository">               
                <property name="fileStore" value="D:/filestore/eclFileStore.dat" />

                <property name="cacheSize" value="5000" />
         </bean>
====================================================================
<route id="ResFiles">
                        <from uri="ftp://user@***.**.**.*:**/xyz?filter=#ResFilter&amp;readLock=changed&amp;idempotent=true&amp;idempotentRepository=#fileStore&amp;delay=2000&amp;readLockCheckInterval=1000&amp;localWorkDirectory=C:/temp&amp;passiveMode=true&amp;disconnect=true&amp;password=****" />
                        <to uri="file:\\*****" />
        </route>


In the above scenario I am only able to restrict the number of entries depending on cacheSize value but not above it . Kindly help .
Reply | Threaded
Open this post in threaded view
|

Re: Restricting the duplicate files to be added into idempotent repository irrespective of cacheSize value

arvind
Please comment on this.
Reply | Threaded
Open this post in threaded view
|

Re: Restricting the duplicate files to be added into idempotent repository irrespective of cacheSize value

ceposta
Maybe you can provide more detail so we can respond intelligently.. or
give an example of what you're doing and what's happening...

<quote>How to
restrict duplicate files adding into idempotent repository irrespective of
cache size , means  any number of files in source directory but no files
should be added twice to the repository .</quote>

Does this mean you're seeing multiple entries of the same file
instance in your idempotent repository? How is the cache size
currently affecting your results?

On Mon, May 12, 2014 at 4:16 AM, arvind <[hidden email]> wrote:
> Please comment on this.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Restricting-the-duplicate-files-to-be-added-into-idempotent-repository-irrespective-of-cacheSize-vale-tp5750853p5751022.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



--
Christian Posta
http://www.christianposta.com/blog
twitter: @christianposta
Reply | Threaded
Open this post in threaded view
|

Re: Restricting the duplicate files to be added into idempotent repository irrespective of cacheSize value

arvind
I am trying to transfer the files from FTP to local directory .Currently the cache size value is 5000 .

Yes I am seeing multiple entries of same file in the idempotent repository when the file entries crosses 5000.

Is there any way that we can restrict to move the same file which is already being transferred as well as restricting the multiple entries of same file in the idempotent repository, irrespective of cache size value.

Please let me know if you need more details.
Reply | Threaded
Open this post in threaded view
|

Re: Restricting the duplicate files to be added into idempotent repository irrespective of cacheSize value

Joe Shevland
Its intriguing. I need more details.

On 14/05/14 15:33, arvind wrote:

> I am trying to transfer the files from FTP to local directory .Currently the
> cache size value is 5000 .
>
> Yes I am seeing multiple entries of same file in the idempotent repository
> when the file entries crosses 5000.
>
> Is there any way that we can restrict to move the same file which is already
> being transferred as well as restricting the multiple entries of same file
> in the idempotent repository, irrespective of cache size value.
>
> Please let me know if you need more details.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Restricting-the-duplicate-files-to-be-added-into-idempotent-repository-irrespective-of-cacheSize-vale-tp5750853p5751106.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>

Reply | Threaded
Open this post in threaded view
|

Re: Restricting the duplicate files to be added into idempotent repository irrespective of cacheSize value

arvind
Thanks for showing the interest. The first post of mine consisting the xml file which I am using to transfer the file from FTP to local directory. It is working fine but when it crosses the number of files more than 5000 i.e the cacheSize value mentioned in the xml , the files are getting added multiple times to Idempotent repository so the files are transferred more than once from FTP to Local directory.
 So please let me know if there is any way to restrict the file to be transferred more than once(from FTP to local directory) means to restrict the the files to add into the repository multiple times.
Reply | Threaded
Open this post in threaded view
|

Re: Restricting the duplicate files to be added into idempotent repository irrespective of cacheSize value

Claus Ibsen-2
Hi

Set a higher cache size, such as 9999999999 or implement your own
idempotent repo.
For example without a first level cache, so it checks the persistent
file each time etc.

Or contribute to the existing file idempotent repo, to let it check
the file if there was a cache miss.
http://camel.apache.org/contributing

On Thu, May 15, 2014 at 12:19 PM, arvind <[hidden email]> wrote:

> Thanks for showing the interest. The first post of mine consisting the xml
> file which I am using to transfer the file from FTP to local directory. It
> is working fine but when it crosses the number of files more than 5000 i.e
> the cacheSize value mentioned in the xml , the files are getting added
> multiple times to Idempotent repository so the files are transferred more
> than once from FTP to Local directory.
>  So please let me know if there is any way to restrict the file to be
> transferred more than once(from FTP to local directory) means to restrict
> the the files to add into the repository multiple times.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Restricting-the-duplicate-files-to-be-added-into-idempotent-repository-irrespective-of-cacheSize-vale-tp5750853p5751154.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



--
Claus Ibsen
-----------------
Red Hat, Inc.
Email: [hidden email]
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
hawtio: http://hawt.io/
fabric8: http://fabric8.io/