File component gotchas

Previous Topic Next Topic
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

File component gotchas

When reading from a file and writing it to queue, are there any corner
cases where we could end up loosing the contents?

If there's an exception during the route, GenericFileRenameProcessStrategy
does a rollback and the original file is not deleted.
But I'm wondering if there could be a corner case where we may mark the
file as processed?
(I understand we move the file to .camel folder or explicitly move it
somewhere else so we have a copy. But that means we have to manually copy
the file back into the main folder).

Additionally it seems like a challenge in a containerized environment.
If I have n replica's and all of them reading from the same folder will be
an issue. The documentation mentions that multiple consumers on the same
folder is an issue.

If we each replica creates it's own directory off the shared folder, and
when the replica dies and a new one is spawned by kubernetes, it will get a
new folder and the files in the folder than belong to the dead folder is
now orphaned.

I would be interested to hear how others solved these issues?


Rajith <>