File based idempotent Repository

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

File based idempotent Repository

Gv, Shivakumar
Hi Team,

We are seeing few ID's  missing in the file repository  when using File based idempotent Consumer , when we run multiple instance of same springboot jar ..
Out 300 successful unique transactions 30 to 40 id's were missing in file repository.

Please let us know if this known issue with file based repo  for clustered environment or  should we use   JDBC IdempotentRepository ?

.idempotentConsumer(simple("${headers.idempotentId}"), org.apache.camel.processor.idempotent.FileIdempotentRepository.fileIdempotentRepository(new java.io.File(idempotentFile), 500))
.eager(true)
.skipDuplicate(true)
.removeOnFailure(true)

Thanks,
Shiva

________________________________
NOTICE: All information in and attached to the e-mails below may be proprietary, confidential, privileged and otherwise protected from improper or erroneous disclosure. If you are not the sender's intended recipient, you are not authorized to intercept, read, print, retain, copy, forward, or disseminate this message. If you have erroneously received this communication, please notify the sender immediately by phone (704-758-1000) or by e-mail and destroy all copies of this message electronic, paper, or otherwise. By transmitting documents via this email: Users, Customers, Suppliers and Vendors collectively acknowledge and agree the transmittal of information via email is voluntary, is offered as a convenience, and is not a secured method of communication; Not to transmit any payment information E.G. credit card, debit card, checking account, wire transfer information, passwords, or sensitive and personal information E.G. Driver's license, DOB, social security, or any other information the user wishes to remain confidential; To transmit only non-confidential information such as plans, pictures and drawings and to assume all risk and liability for and indemnify Lowe's from any claims, losses or damages that may arise from the transmittal of documents or including non-confidential information in the body of an email transmittal. Thank you.
Reply | Threaded
Open this post in threaded view
|

Re: File based idempotent Repository

Claus Ibsen-2
Hi

The file idempotent repository is not clusterable with concurrent consumers.
If you have a copy of the CiA2 book then see the clustering chapter.

On Thu, Jul 30, 2020 at 12:00 PM Gv, Shivakumar <[hidden email]> wrote:

>
> Hi Team,
>
> We are seeing few ID's  missing in the file repository  when using File based idempotent Consumer , when we run multiple instance of same springboot jar ..
> Out 300 successful unique transactions 30 to 40 id's were missing in file repository.
>
> Please let us know if this known issue with file based repo  for clustered environment or  should we use   JDBC IdempotentRepository ?
>
> .idempotentConsumer(simple("${headers.idempotentId}"), org.apache.camel.processor.idempotent.FileIdempotentRepository.fileIdempotentRepository(new java.io.File(idempotentFile), 500))
> .eager(true)
> .skipDuplicate(true)
> .removeOnFailure(true)
>
> Thanks,
> Shiva
>
> ________________________________
> NOTICE: All information in and attached to the e-mails below may be proprietary, confidential, privileged and otherwise protected from improper or erroneous disclosure. If you are not the sender's intended recipient, you are not authorized to intercept, read, print, retain, copy, forward, or disseminate this message. If you have erroneously received this communication, please notify the sender immediately by phone (704-758-1000) or by e-mail and destroy all copies of this message electronic, paper, or otherwise. By transmitting documents via this email: Users, Customers, Suppliers and Vendors collectively acknowledge and agree the transmittal of information via email is voluntary, is offered as a convenience, and is not a secured method of communication; Not to transmit any payment information E.G. credit card, debit card, checking account, wire transfer information, passwords, or sensitive and personal information E.G. Driver's license, DOB, social security, or any other information the user wishes to remain confidential; To transmit only non-confidential information such as plans, pictures and drawings and to assume all risk and liability for and indemnify Lowe's from any claims, losses or damages that may arise from the transmittal of documents or including non-confidential information in the body of an email transmittal. Thank you.



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

RE: [EXTERNAL] Re: File based idempotent Repository

Gv, Shivakumar
Thanks Claus. Sure I will have a look into that chapter.

We were also checking  on  HazelCast Idempotent Repository as an option . Does that have any drawbacks  or any other recommendation  for clustered setup because we want to avoid using DataBase for unique ID repo.

Thanks,
Shiva


-----Original Message-----
From: Claus Ibsen <[hidden email]>
Sent: Thursday, July 30, 2020 3:33 PM
To: [hidden email]
Subject: [EXTERNAL] Re: File based idempotent Repository

*EXTERNAL SENDER*


Hi

The file idempotent repository is not clusterable with concurrent consumers.
If you have a copy of the CiA2 book then see the clustering chapter.

On Thu, Jul 30, 2020 at 12:00 PM Gv, Shivakumar <[hidden email]> wrote:

>
> Hi Team,
>
> We are seeing few ID's  missing in the file repository  when using File based idempotent Consumer , when we run multiple instance of same springboot jar ..
> Out 300 successful unique transactions 30 to 40 id's were missing in file repository.
>
> Please let us know if this known issue with file based repo  for clustered environment or  should we use   JDBC IdempotentRepository ?
>
> .idempotentConsumer(simple("${headers.idempotentId}"),
> org.apache.camel.processor.idempotent.FileIdempotentRepository.fileIde
> mpotentRepository(new java.io.File(idempotentFile), 500))
> .eager(true)
> .skipDuplicate(true)
> .removeOnFailure(true)
>
> Thanks,
> Shiva
>
> ________________________________
> NOTICE: All information in and attached to the e-mails below may be proprietary, confidential, privileged and otherwise protected from improper or erroneous disclosure. If you are not the sender's intended recipient, you are not authorized to intercept, read, print, retain, copy, forward, or disseminate this message. If you have erroneously received this communication, please notify the sender immediately by phone (704-758-1000) or by e-mail and destroy all copies of this message electronic, paper, or otherwise. By transmitting documents via this email: Users, Customers, Suppliers and Vendors collectively acknowledge and agree the transmittal of information via email is voluntary, is offered as a convenience, and is not a secured method of communication; Not to transmit any payment information E.G. credit card, debit card, checking account, wire transfer information, passwords, or sensitive and personal information E.G. Driver's license, DOB, social security, or any other information the user wishes to remain confidential; To transmit only non-confidential information such as plans, pictures and drawings and to assume all risk and liability for and indemnify Lowe's from any claims, losses or damages that may arise from the transmittal of documents or including non-confidential information in the body of an email transmittal. Thank you.



--
Claus Ibsen
-----------------
https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdavsclaus.com%2F&amp;data=02%7C01%7Cshivakumar.gv%40lowes.com%7C08a8dcc2dc2646f3b62e08d8346fbfce%7Cbcfa3e87841e48c7983b584159dd1a69%7C0%7C0%7C637317001855503477&amp;sdata=TeB5O%2FMIk2BolLuTjTz1MOzmpNyz%2B5WhqTA85Z5eNVc%3D&amp;reserved=0 @davsclaus Camel in Action 2: https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.manning.com%2Fibsen2&amp;data=02%7C01%7Cshivakumar.gv%40lowes.com%7C08a8dcc2dc2646f3b62e08d8346fbfce%7Cbcfa3e87841e48c7983b584159dd1a69%7C0%7C0%7C637317001855503477&amp;sdata=2KR98sYqgVZj3xX6CnI3K6zmAWbBSYx29jlOYXBVa08%3D&amp;reserved=0
________________________________
NOTICE: All information in and attached to the e-mails below may be proprietary, confidential, privileged and otherwise protected from improper or erroneous disclosure. If you are not the sender's intended recipient, you are not authorized to intercept, read, print, retain, copy, forward, or disseminate this message. If you have erroneously received this communication, please notify the sender immediately by phone (704-758-1000) or by e-mail and destroy all copies of this message electronic, paper, or otherwise. By transmitting documents via this email: Users, Customers, Suppliers and Vendors collectively acknowledge and agree the transmittal of information via email is voluntary, is offered as a convenience, and is not a secured method of communication; Not to transmit any payment information E.G. credit card, debit card, checking account, wire transfer information, passwords, or sensitive and personal information E.G. Driver's license, DOB, social security, or any other information the user wishes to remain confidential; To transmit only non-confidential information such as plans, pictures and drawings and to assume all risk and liability for and indemnify Lowe's from any claims, losses or damages that may arise from the transmittal of documents or including non-confidential information in the body of an email transmittal. Thank you.