Strategy for deleting/moving files

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Strategy for deleting/moving files

jpcook01
Hello,

I am using the file endpoint with delete=true. My routes pick up files
from a directory and then pass through a number of steps, so are
reasonably complex. I have a dead letter route where in the event of an
error I want to move the original file to a kind of failed directory area.

But the problem is that when an error occurs, by the time the code runs
inside the dead letter route, the original file has already been deleted
by the FileComponent because of delete=true. If I set this to false it
works OK but the file stays at the original folder which I don't want.

Do I have to manage this myself, i.e not use delete=true and have
another step in my route (bean or processor) which would delete or move
the file from the original directory at the end of all the processing
rather than rely on the file component?

Thanks for any suggestions!

Paul

Reply | Threaded
Open this post in threaded view
|

Re: Strategy for deleting/moving files

Alex Dettinger
Hi Jonathan,

   Depending on the situation, you could try to use
.to("file:kind-of-failed-directory") in the dead letter channel.
   But, it sounds to me that you should not use a dlc and look first at the
moveFailed option of the file-component*:*
*
https://github.com/apache/camel/blob/master/components/camel-file/src/main/docs/file-component.adoc
<https://github.com/apache/camel/blob/master/components/camel-file/src/main/docs/file-component.adoc>*

Enjoy :)
Alex

On Fri, May 17, 2019 at 7:03 PM Jonathan Cook <[hidden email]>
wrote:

> Hello,
>
> I am using the file endpoint with delete=true. My routes pick up files
> from a directory and then pass through a number of steps, so are
> reasonably complex. I have a dead letter route where in the event of an
> error I want to move the original file to a kind of failed directory area.
>
> But the problem is that when an error occurs, by the time the code runs
> inside the dead letter route, the original file has already been deleted
> by the FileComponent because of delete=true. If I set this to false it
> works OK but the file stays at the original folder which I don't want.
>
> Do I have to manage this myself, i.e not use delete=true and have
> another step in my route (bean or processor) which would delete or move
> the file from the original directory at the end of all the processing
> rather than rely on the file component?
>
> Thanks for any suggestions!
>
> Paul
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Strategy for deleting/moving files

dpravin
In reply to this post by jpcook01
You can write the file to the queue and then process it. This will simplify
the logic.

Thanks,
Pravin

On Fri, May 17, 2019 at 1:03 PM Jonathan Cook <[hidden email]>
wrote:

> Hello,
>
> I am using the file endpoint with delete=true. My routes pick up files
> from a directory and then pass through a number of steps, so are
> reasonably complex. I have a dead letter route where in the event of an
> error I want to move the original file to a kind of failed directory area.
>
> But the problem is that when an error occurs, by the time the code runs
> inside the dead letter route, the original file has already been deleted
> by the FileComponent because of delete=true. If I set this to false it
> works OK but the file stays at the original folder which I don't want.
>
> Do I have to manage this myself, i.e not use delete=true and have
> another step in my route (bean or processor) which would delete or move
> the file from the original directory at the end of all the processing
> rather than rely on the file component?
>
> Thanks for any suggestions!
>
> Paul
>
>
- Pravin
Reply | Threaded
Open this post in threaded view
|

RE: Strategy for deleting/moving files

Hart, James W.
In reply to this post by jpcook01
The file: endpoint supports what you want.

Set as below and on failure it will move the file to the .error directory.  You can also use simple language to make it more dynamic and move to anywhere you want.

&moveFailed=.error

-----Original Message-----
From: Jonathan Cook [mailto:[hidden email]]
Sent: Friday, May 17, 2019 1:04 PM
To: [hidden email]
Subject: Strategy for deleting/moving files

[[ SEI WARNING *** This email was sent from an external source. Do not open attachments or click on links from unknown or suspicious senders. *** ]]


Hello,

I am using the file endpoint with delete=true. My routes pick up files
from a directory and then pass through a number of steps, so are
reasonably complex. I have a dead letter route where in the event of an
error I want to move the original file to a kind of failed directory area.

But the problem is that when an error occurs, by the time the code runs
inside the dead letter route, the original file has already been deleted
by the FileComponent because of delete=true. If I set this to false it
works OK but the file stays at the original folder which I don't want.

Do I have to manage this myself, i.e not use delete=true and have
another step in my route (bean or processor) which would delete or move
the file from the original directory at the end of all the processing
rather than rely on the file component?

Thanks for any suggestions!

Paul

Reply | Threaded
Open this post in threaded view
|

Re: Strategy for deleting/moving files

jpcook01
In reply to this post by jpcook01
Thanks for the replies

The moveFailed does do what I want but only when I remove the configured
errorHandler from my routes. In my error handler I have other logic such
as sending an email notification. So I don't think I can sue the
moveFailed unless there is a way to be able to use both?

Thanks

On 17/05/2019 19:03, Jonathan Cook wrote:

> Hello,
>
> I am using the file endpoint with delete=true. My routes pick up files
> from a directory and then pass through a number of steps, so are
> reasonably complex. I have a dead letter route where in the event of
> an error I want to move the original file to a kind of failed
> directory area.
>
> But the problem is that when an error occurs, by the time the code
> runs inside the dead letter route, the original file has already been
> deleted by the FileComponent because of delete=true. If I set this to
> false it works OK but the file stays at the original folder which I
> don't want.
>
> Do I have to manage this myself, i.e not use delete=true and have
> another step in my route (bean or processor) which would delete or
> move the file from the original directory at the end of all the
> processing rather than rely on the file component?
>
> Thanks for any suggestions!
>
> Paul
>

Reply | Threaded
Open this post in threaded view
|

Re: Strategy for deleting/moving files

Alex Dettinger
In this case, chances are that adding .to("file:kind-of-failed-directory")
at some point in the dead letter channel logic helps.
useOriginalMessage() may be of interest too.

Alex

On Sat, May 18, 2019 at 10:14 AM Jonathan Cook <[hidden email]>
wrote:

> Thanks for the replies
>
> The moveFailed does do what I want but only when I remove the configured
> errorHandler from my routes. In my error handler I have other logic such
> as sending an email notification. So I don't think I can sue the
> moveFailed unless there is a way to be able to use both?
>
> Thanks
>
> On 17/05/2019 19:03, Jonathan Cook wrote:
> > Hello,
> >
> > I am using the file endpoint with delete=true. My routes pick up files
> > from a directory and then pass through a number of steps, so are
> > reasonably complex. I have a dead letter route where in the event of
> > an error I want to move the original file to a kind of failed
> > directory area.
> >
> > But the problem is that when an error occurs, by the time the code
> > runs inside the dead letter route, the original file has already been
> > deleted by the FileComponent because of delete=true. If I set this to
> > false it works OK but the file stays at the original folder which I
> > don't want.
> >
> > Do I have to manage this myself, i.e not use delete=true and have
> > another step in my route (bean or processor) which would delete or
> > move the file from the original directory at the end of all the
> > processing rather than rely on the file component?
> >
> > Thanks for any suggestions!
> >
> > Paul
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Strategy for deleting/moving files

bjoern.ohm
Hi,

remove the errorHandler and just use onException. For example use

this.onException(Throwable.class)
.maximumRedeliveries(0)
.to(ENDPOINT_DEAD_LETTER_CHANN)
.stop();

instead of

 this.errorHandler(this.deadLetterChannel(ENDPOINT_DEAD_LETTER_CHANNEL));

Regards...
 



Von:    Alex Dettinger <[hidden email]>
An:     [hidden email]
Datum:  18.05.2019 11:50
Betreff:        Re: Strategy for deleting/moving files



In this case, chances are that adding .to("file:kind-of-failed-directory")
at some point in the dead letter channel logic helps.
useOriginalMessage() may be of interest too.

Alex

On Sat, May 18, 2019 at 10:14 AM Jonathan Cook
<[hidden email]>
wrote:

> Thanks for the replies
>
> The moveFailed does do what I want but only when I remove the configured
> errorHandler from my routes. In my error handler I have other logic such
> as sending an email notification. So I don't think I can sue the
> moveFailed unless there is a way to be able to use both?
>
> Thanks
>
> On 17/05/2019 19:03, Jonathan Cook wrote:
> > Hello,
> >
> > I am using the file endpoint with delete=true. My routes pick up files
> > from a directory and then pass through a number of steps, so are
> > reasonably complex. I have a dead letter route where in the event of
> > an error I want to move the original file to a kind of failed
> > directory area.
> >
> > But the problem is that when an error occurs, by the time the code
> > runs inside the dead letter route, the original file has already been
> > deleted by the FileComponent because of delete=true. If I set this to
> > false it works OK but the file stays at the original folder which I
> > don't want.
> >
> > Do I have to manage this myself, i.e not use delete=true and have
> > another step in my route (bean or processor) which would delete or
> > move the file from the original directory at the end of all the
> > processing rather than rely on the file component?
> >
> > Thanks for any suggestions!
> >
> > Paul
> >
>
>



 
Diese E-Mail könnte vertrauliche und / oder rechtlich geschützte Informationen enthalten. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Mail sind nicht gestattet.

This e-mail may contain confidential and / or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
Reply | Threaded
Open this post in threaded view
|

Re: Strategy for deleting/moving files

jpcook01
In reply to this post by jpcook01
Hi Alex,

Great, that did the trick. Why is it that onException works and
errorHandler doesn't for the file moveFailed specifically? Just to
understand..

Thanks again
Jonathan

On 18/05/2019 10:14, Jonathan Cook wrote:

> Thanks for the replies
>
> The moveFailed does do what I want but only when I remove the
> configured errorHandler from my routes. In my error handler I have
> other logic such as sending an email notification. So I don't think I
> can sue the moveFailed unless there is a way to be able to use both?
>
> Thanks
>
> On 17/05/2019 19:03, Jonathan Cook wrote:
>> Hello,
>>
>> I am using the file endpoint with delete=true. My routes pick up
>> files from a directory and then pass through a number of steps, so
>> are reasonably complex. I have a dead letter route where in the event
>> of an error I want to move the original file to a kind of failed
>> directory area.
>>
>> But the problem is that when an error occurs, by the time the code
>> runs inside the dead letter route, the original file has already been
>> deleted by the FileComponent because of delete=true. If I set this to
>> false it works OK but the file stays at the original folder which I
>> don't want.
>>
>> Do I have to manage this myself, i.e not use delete=true and have
>> another step in my route (bean or processor) which would delete or
>> move the file from the original directory at the end of all the
>> processing rather than rely on the file component?
>>
>> Thanks for any suggestions!
>>
>> Paul
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Strategy for deleting/moving files

Claus Ibsen-2
Hi

On Sun, May 19, 2019 at 5:09 PM Jonathan Cook
<[hidden email]> wrote:
>
> Hi Alex,
>
> Great, that did the trick. Why is it that onException works and
> errorHandler doesn't for the file moveFailed specifically? Just to
> understand..
>

Because errorHandler will handle the error (eg handled = true) which
makes the Camel file consumer regard the file as success and therefore
delete the file.

In the onException you have fine grained control, and the code did not
set handled(true) and therefore the error is un-handled, and the file
consumer regard the file as un-sucecesful and do not delete the file.
And if you have moveFailed set then that will move the file to that
failed directory.



> Thanks again
> Jonathan
>
> On 18/05/2019 10:14, Jonathan Cook wrote:
> > Thanks for the replies
> >
> > The moveFailed does do what I want but only when I remove the
> > configured errorHandler from my routes. In my error handler I have
> > other logic such as sending an email notification. So I don't think I
> > can sue the moveFailed unless there is a way to be able to use both?
> >
> > Thanks
> >
> > On 17/05/2019 19:03, Jonathan Cook wrote:
> >> Hello,
> >>
> >> I am using the file endpoint with delete=true. My routes pick up
> >> files from a directory and then pass through a number of steps, so
> >> are reasonably complex. I have a dead letter route where in the event
> >> of an error I want to move the original file to a kind of failed
> >> directory area.
> >>
> >> But the problem is that when an error occurs, by the time the code
> >> runs inside the dead letter route, the original file has already been
> >> deleted by the FileComponent because of delete=true. If I set this to
> >> false it works OK but the file stays at the original folder which I
> >> don't want.
> >>
> >> Do I have to manage this myself, i.e not use delete=true and have
> >> another step in my route (bean or processor) which would delete or
> >> move the file from the original directory at the end of all the
> >> processing rather than rely on the file component?
> >>
> >> Thanks for any suggestions!
> >>
> >> Paul
> >>
> >
>


--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2