File polling issue for filename with multiple dots

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

File polling issue for filename with multiple dots

Guillaume D.
Hi guys,

I'm working on un upgrade from Camel 2.6.0 to 2.22.x (yeah... this is huge step !!) and need some help for an issue with file polling.
I'm not very familiar with this framework, so I'll try to give enough info to expose our issue. If I'm not, sorry for that !

Our application read file from a given directory which follow the format : <mySignificativeName>_yyyyMMdd_HHmmss.SSS.ebt
It was working fine in 2.6.0, but after the upgrade our file are not consumed and stay in entry directory.

After a few hours of debugging, I finally found why.
It's because our file names contains two dots, and the GenericFileEndpoint class uses FileUtil.stripExt() function to generate the done file name.
This function has been updated for CAMEL-9064 issue to work in "single mode" and the GenericFileEndpoint does not appear to have been updated too (don't know this stuff very well to say if he should have been or not).

Code from GenericFileEndpoint.createDoneFileName
--------------------------------------------------------------------------
[...]
        pattern = pattern.replaceFirst("\\$\\{file:name.noext\\}", FileUtil.stripExt(onlyName));
        pattern = pattern.replaceFirst("\\$simple\\{file:name.noext\\}", FileUtil.stripExt(onlyName));
[...]

Code from GenericFileEndpoint.createDoneFileName
--------------------------------------------------------------------------
    public static String stripExt(String name) {
        return stripExt(name, false);
    }

    public static String stripExt(String name, boolean singleMode) {
         [...]
    }

Our road is defined this way :
--------------------------------------------------------------------------
   from("file:<entryDirectory>?sortBy=reverse:file:modified&include=B_[A-Z]{3,30}_01.*\.ebt$&charset=UTF-8&delete=true&doneFileName=${file:name.noext}.done", "direct:flux04")
       .process(new OurProcessor())
       .to("cxf://ourEndpoint...")


We updated our source application to generate file names with a single dot for extension, but for compatibility reason we need to read both file names (with a single dot & with multiple dots).
I don't know how to proceed :
    1) It's really an issue in GenericFileEndpoint => so we just have to wait for a patch or even submit it if we are good enough to :)
    2) Maybe we can set a parameter to accept files with multiple dots in their file names => does not seem to be possible in all documentations I read
    3) Or we should extends GenericFileEndpoint to get the behavior that we need => I don't know how to proceed for that

Thanks a lot for your help,

Guillaume
Reply | Threaded
Open this post in threaded view
|

Re: File polling issue for filename with multiple dots

Alex Dettinger
Hi Guillaume,

  Welcome on camel 2.22.x :) I've reported issue
https://issues.apache.org/jira/browse/CAMEL-12940 and provided a fix.
You should be able to check the fix against 2.23.0-SNAPSHOT tomorrow, or
wait for the release of CAMEL-2.22.3.

Enjoy,
Alex

On Thu, Nov 15, 2018 at 3:17 PM DEIMAT Guillaume <
[hidden email]> wrote:

> Hi guys,
>
> I'm working on un upgrade from Camel 2.6.0 to 2.22.x (yeah... this is huge
> step !!) and need some help for an issue with file polling.
> I'm not very familiar with this framework, so I'll try to give enough info
> to expose our issue. If I'm not, sorry for that !
>
> Our application read file from a given directory which follow the format :
> <mySignificativeName>_yyyyMMdd_HHmmss.SSS.ebt
> It was working fine in 2.6.0, but after the upgrade our file are not
> consumed and stay in entry directory.
>
> After a few hours of debugging, I finally found why.
> It's because our file names contains two dots, and the GenericFileEndpoint
> class uses FileUtil.stripExt() function to generate the done file name.
> This function has been updated for CAMEL-9064 issue to work in "single
> mode" and the GenericFileEndpoint does not appear to have been updated too
> (don't know this stuff very well to say if he should have been or not).
>
> Code from GenericFileEndpoint.createDoneFileName
> --------------------------------------------------------------------------
> [...]
>         pattern = pattern.replaceFirst("\\$\\{file:name.noext\\}",
> FileUtil.stripExt(onlyName));
>         pattern = pattern.replaceFirst("\\$simple\\{file:name.noext\\}",
> FileUtil.stripExt(onlyName));
> [...]
>
> Code from GenericFileEndpoint.createDoneFileName
> --------------------------------------------------------------------------
>     public static String stripExt(String name) {
>         return stripExt(name, false);
>     }
>
>     public static String stripExt(String name, boolean singleMode) {
>          [...]
>     }
>
> Our road is defined this way :
> --------------------------------------------------------------------------
>
>  from("file:<entryDirectory>?sortBy=reverse:file:modified&include=B_[A-Z]{3,30}_01.*\.ebt$&charset=UTF-8&delete=true&doneFileName=${file:name.noext}.done",
> "direct:flux04")
>        .process(new OurProcessor())
>        .to("cxf://ourEndpoint...")
>
>
> We updated our source application to generate file names with a single dot
> for extension, but for compatibility reason we need to read both file names
> (with a single dot & with multiple dots).
> I don't know how to proceed :
>     1) It's really an issue in GenericFileEndpoint => so we just have to
> wait for a patch or even submit it if we are good enough to :)
>     2) Maybe we can set a parameter to accept files with multiple dots in
> their file names => does not seem to be possible in all documentations I
> read
>     3) Or we should extends GenericFileEndpoint to get the behavior that
> we need => I don't know how to proceed for that
>
> Thanks a lot for your help,
>
> Guillaume
>
Reply | Threaded
Open this post in threaded view
|

RE: File polling issue for filename with multiple dots

Guillaume D.
Hi Alex,

Thanks a lot for the quick fix !
We'll wait for the next release so :)

Guillaume

-----Message d'origine-----
De : Alex Dettinger <[hidden email]>
Envoyé : jeudi 15 novembre 2018 20:56
À : [hidden email]
Objet : Re: File polling issue for filename with multiple dots

Hi Guillaume,

  Welcome on camel 2.22.x :) I've reported issue
https://issues.apache.org/jira/browse/CAMEL-12940 and provided a fix.
You should be able to check the fix against 2.23.0-SNAPSHOT tomorrow, or wait for the release of CAMEL-2.22.3.

Enjoy,
Alex

On Thu, Nov 15, 2018 at 3:17 PM DEIMAT Guillaume < [hidden email]> wrote:

> Hi guys,
>
> I'm working on un upgrade from Camel 2.6.0 to 2.22.x (yeah... this is
> huge step !!) and need some help for an issue with file polling.
> I'm not very familiar with this framework, so I'll try to give enough
> info to expose our issue. If I'm not, sorry for that !
>
> Our application read file from a given directory which follow the format :
> <mySignificativeName>_yyyyMMdd_HHmmss.SSS.ebt
> It was working fine in 2.6.0, but after the upgrade our file are not
> consumed and stay in entry directory.
>
> After a few hours of debugging, I finally found why.
> It's because our file names contains two dots, and the
> GenericFileEndpoint class uses FileUtil.stripExt() function to generate the done file name.
> This function has been updated for CAMEL-9064 issue to work in "single
> mode" and the GenericFileEndpoint does not appear to have been updated
> too (don't know this stuff very well to say if he should have been or not).
>
> Code from GenericFileEndpoint.createDoneFileName
> ----------------------------------------------------------------------
> ----
> [...]
>         pattern = pattern.replaceFirst("\\$\\{file:name.noext\\}",
> FileUtil.stripExt(onlyName));
>         pattern =
> pattern.replaceFirst("\\$simple\\{file:name.noext\\}",
> FileUtil.stripExt(onlyName));
> [...]
>
> Code from GenericFileEndpoint.createDoneFileName
> --------------------------------------------------------------------------
>     public static String stripExt(String name) {
>         return stripExt(name, false);
>     }
>
>     public static String stripExt(String name, boolean singleMode) {
>          [...]
>     }
>
> Our road is defined this way :
> ----------------------------------------------------------------------
> ----
>
>  
> from("file:<entryDirectory>?sortBy=reverse:file:modified&include=B_[A-
> Z]{3,30}_01.*\.ebt$&charset=UTF-8&delete=true&doneFileName=${file:name
> .noext}.done",
> "direct:flux04")
>        .process(new OurProcessor())
>        .to("cxf://ourEndpoint...")
>
>
> We updated our source application to generate file names with a single
> dot for extension, but for compatibility reason we need to read both
> file names (with a single dot & with multiple dots).
> I don't know how to proceed :
>     1) It's really an issue in GenericFileEndpoint => so we just have
> to wait for a patch or even submit it if we are good enough to :)
>     2) Maybe we can set a parameter to accept files with multiple dots
> in their file names => does not seem to be possible in all
> documentations I read
>     3) Or we should extends GenericFileEndpoint to get the behavior
> that we need => I don't know how to proceed for that
>
> Thanks a lot for your help,
>
> Guillaume
>