Problems with dynamic router and split

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems with dynamic router and split

Bengt Rodehav
I'm using Camel 2.10.3.

I have a route that parses a CSV file (using Bindy) and then splits the
exchange so that each row is treated by itself. Something like this:

*  DataFormat bindy = new
BindyCsvDataFormat(ConfirmedOrder.class.getPackage().getName());*
*
*
*  from("file:in")*
*    .log("Processing file: ${header.CamelFileName}")*
*    .unmarshal(bindy)*
*    .split(body())*
*      .process(new MessageHandlingProcessor())*
*      .to("file:out)*
*    .end()*
*    .log("Done processing file: ${header.CamelFileName}");*

This works fine. However, I now need to use dynamic routing so that each
message (each row in the CSV) is routed to a destination that depends on
one of the fields in the CSV. I therefore tried to use the dynamic router,
like this:

*  from("file:in")*
*    .log("Processing file: ${header.CamelFileName}")*
*    .unmarshal(bindy)*
*    .split(body())*
*      .process(new MessageHandlingProcessor())*
*      .dynamicRouter(method(new ExternalSystemRouter(), "route"))*
*    .end()*
*    .log("Done processing file: ${header.CamelFileName}");*
*    *
*  public class ExternalSystemRouter {*
*    public String route(Exchange theExchange) throws
UnsupportedEncodingException {*
*      return "file:dynout";*
*    }*
*  }*

The "route" method is not very dynamic but it's for illustrative purposes.

What happens is that the route starts looping forever and processes the
same file over and over again. The debug log looks like this:

*...*
*2012-12-28 10:04:10,552 | DEBUG | d #6 - file://in | BeanProcessor
           | mel.component.bean.BeanProcessor  190 | Setting bean
invocation result on the OUT message: file:dynout*
*2012-12-28 10:04:10,555 | DEBUG | d #6 - file://in | FileOperations
            | el.component.file.FileOperations  367 | Using InputStream to
write file: dynout\orderconf-order-id-20121228100410238.xml*
*2012-12-28 10:04:10,559 | DEBUG | d #6 - file://in | GenericFileProducer
           | mponent.file.GenericFileProducer  253 | Wrote
[dynout\orderconf-order-id-20121228100410238.xml] to
[Endpoint[file://dynout]]*
*2012-12-28 10:04:10,562 | DEBUG | d #6 - file://in | BeanProcessor
           | mel.component.bean.BeanProcessor  190 | Setting bean
invocation result on the OUT message: file:dynout*
*2012-12-28 10:04:10,563 | DEBUG | d #6 - file://in | FileOperations
            | el.component.file.FileOperations  367 | Using InputStream to
write file: dynout\orderconf-order-id-20121228100410238.xml*
*2012-12-28 10:04:10,567 | DEBUG | d #6 - file://in | GenericFileProducer
           | mponent.file.GenericFileProducer  253 | Wrote
[dynout\orderconf-order-id-20121228100410238.xml] to
[Endpoint[file://dynout]]*
*2012-12-28 10:04:10,579 | DEBUG | d #6 - file://in | BeanProcessor
           | mel.component.bean.BeanProcessor  190 | Setting bean
invocation result on the OUT message: file:dynout*
*2012-12-28 10:04:10,580 | DEBUG | d #6 - file://in | FileOperations
            | el.component.file.FileOperations  367 | Using InputStream to
write file: dynout\orderconf-order-id-20121228100410238.xml*
*2012-12-28 10:04:10,585 | DEBUG | d #6 - file://in | GenericFileProducer
           | mponent.file.GenericFileProducer  253 | Wrote
[dynout\orderconf-order-id-20121228100410238.xml] to
[Endpoint[file://dynout]]*
*2012-12-28 10:04:10,595 | DEBUG | d #6 - file://in | BeanProcessor
           | mel.component.bean.BeanProcessor  190 | Setting bean
invocation result on the OUT message: file:dynout*
*2012-12-28 10:04:10,596 | DEBUG | d #6 - file://in | FileOperations
            | el.component.file.FileOperations  367 | Using InputStream to
write file: dynout\orderconf-order-id-20121228100410238.xml*
*2012-12-28 10:04:10,601 | DEBUG | d #6 - file://in | GenericFileProducer
           | mponent.file.GenericFileProducer  253 | Wrote
[dynout\orderconf-order-id-20121228100410238.xml] to
[Endpoint[file://dynout]]*
*2012-12-28 10:04:10,610 | DEBUG | d #6 - file://in | BeanProcessor
           | mel.component.bean.BeanProcessor  190 | Setting bean
invocation result on the OUT message: file:dynout*
*2012-12-28 10:04:10,613 | DEBUG | d #6 - file://in | FileOperations
            | el.component.file.FileOperations  367 | Using InputStream to
write file: dynout\orderconf-order-id-20121228100410238.xml*
*...*

This particular CSV file only has one row so only one output file should be
created. It seems that the file that is being processed is never marked as
"handled".

I have a feeling that this is because the "end()" statement is not added to
the route when I use the dynamic router. But how can I fix this?

/Bengt
Reply | Threaded
Open this post in threaded view
|

Re: Problems with dynamic router and split

Bengt Rodehav
After even more googling and reading the documentation again it seems like
the dynamic router is being called repeatably until it returns null. I will
try to rethink my strategy. It seems like this is a bit "overkill" when all
I want is one dynamic destination and not a complete dynamic route.

/Bengt


2012/12/28 Bengt Rodehav <[hidden email]>

> I'm using Camel 2.10.3.
>
> I have a route that parses a CSV file (using Bindy) and then splits the
> exchange so that each row is treated by itself. Something like this:
>
> *  DataFormat bindy = new
> BindyCsvDataFormat(ConfirmedOrder.class.getPackage().getName());*
> *
> *
> *  from("file:in")*
> *    .log("Processing file: ${header.CamelFileName}")*
> *    .unmarshal(bindy)*
> *    .split(body())*
> *      .process(new MessageHandlingProcessor())*
> *      .to("file:out)*
> *    .end()*
> *    .log("Done processing file: ${header.CamelFileName}");*
>
> This works fine. However, I now need to use dynamic routing so that each
> message (each row in the CSV) is routed to a destination that depends on
> one of the fields in the CSV. I therefore tried to use the dynamic router,
> like this:
>
> *  from("file:in")*
> *    .log("Processing file: ${header.CamelFileName}")*
> *    .unmarshal(bindy)*
> *    .split(body())*
> *      .process(new MessageHandlingProcessor())*
> *      .dynamicRouter(method(new ExternalSystemRouter(), "route"))*
> *    .end()*
> *    .log("Done processing file: ${header.CamelFileName}");*
> *    *
> *  public class ExternalSystemRouter {*
> *    public String route(Exchange theExchange) throws
> UnsupportedEncodingException {*
> *      return "file:dynout";*
> *    }*
> *  }*
>
> The "route" method is not very dynamic but it's for illustrative purposes.
>
> What happens is that the route starts looping forever and processes the
> same file over and over again. The debug log looks like this:
>
> *...*
> *2012-12-28 10:04:10,552 | DEBUG | d #6 - file://in | BeanProcessor
>              | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,555 | DEBUG | d #6 - file://in | FileOperations
>               | el.component.file.FileOperations  367 | Using InputStream
> to write file: dynout\orderconf-order-id-20121228100410238.xml*
> *2012-12-28 10:04:10,559 | DEBUG | d #6 - file://in | GenericFileProducer
>              | mponent.file.GenericFileProducer  253 | Wrote
> [dynout\orderconf-order-id-20121228100410238.xml] to
> [Endpoint[file://dynout]]*
> *2012-12-28 10:04:10,562 | DEBUG | d #6 - file://in | BeanProcessor
>              | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,563 | DEBUG | d #6 - file://in | FileOperations
>               | el.component.file.FileOperations  367 | Using InputStream
> to write file: dynout\orderconf-order-id-20121228100410238.xml*
> *2012-12-28 10:04:10,567 | DEBUG | d #6 - file://in | GenericFileProducer
>              | mponent.file.GenericFileProducer  253 | Wrote
> [dynout\orderconf-order-id-20121228100410238.xml] to
> [Endpoint[file://dynout]]*
> *2012-12-28 10:04:10,579 | DEBUG | d #6 - file://in | BeanProcessor
>              | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,580 | DEBUG | d #6 - file://in | FileOperations
>               | el.component.file.FileOperations  367 | Using InputStream
> to write file: dynout\orderconf-order-id-20121228100410238.xml*
> *2012-12-28 10:04:10,585 | DEBUG | d #6 - file://in | GenericFileProducer
>              | mponent.file.GenericFileProducer  253 | Wrote
> [dynout\orderconf-order-id-20121228100410238.xml] to
> [Endpoint[file://dynout]]*
> *2012-12-28 10:04:10,595 | DEBUG | d #6 - file://in | BeanProcessor
>              | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,596 | DEBUG | d #6 - file://in | FileOperations
>               | el.component.file.FileOperations  367 | Using InputStream
> to write file: dynout\orderconf-order-id-20121228100410238.xml*
> *2012-12-28 10:04:10,601 | DEBUG | d #6 - file://in | GenericFileProducer
>              | mponent.file.GenericFileProducer  253 | Wrote
> [dynout\orderconf-order-id-20121228100410238.xml] to
> [Endpoint[file://dynout]]*
> *2012-12-28 10:04:10,610 | DEBUG | d #6 - file://in | BeanProcessor
>              | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,613 | DEBUG | d #6 - file://in | FileOperations
>               | el.component.file.FileOperations  367 | Using InputStream
> to write file: dynout\orderconf-order-id-20121228100410238.xml*
> *...*
>
> This particular CSV file only has one row so only one output file should
> be created. It seems that the file that is being processed is never marked
> as "handled".
>
> I have a feeling that this is because the "end()" statement is not added
> to the route when I use the dynamic router. But how can I fix this?
>
> /Bengt
>
Reply | Threaded
Open this post in threaded view
|

Re: Problems with dynamic router and split

Claus Ibsen-2
In reply to this post by Bengt Rodehav
Hi

See the content based router, or the dynamic recipient list EIPs.

On Fri, Dec 28, 2012 at 10:12 AM, Bengt Rodehav <[hidden email]> wrote:

> I'm using Camel 2.10.3.
>
> I have a route that parses a CSV file (using Bindy) and then splits the
> exchange so that each row is treated by itself. Something like this:
>
> *  DataFormat bindy = new
> BindyCsvDataFormat(ConfirmedOrder.class.getPackage().getName());*
> *
> *
> *  from("file:in")*
> *    .log("Processing file: ${header.CamelFileName}")*
> *    .unmarshal(bindy)*
> *    .split(body())*
> *      .process(new MessageHandlingProcessor())*
> *      .to("file:out)*
> *    .end()*
> *    .log("Done processing file: ${header.CamelFileName}");*
>
> This works fine. However, I now need to use dynamic routing so that each
> message (each row in the CSV) is routed to a destination that depends on
> one of the fields in the CSV. I therefore tried to use the dynamic router,
> like this:
>
> *  from("file:in")*
> *    .log("Processing file: ${header.CamelFileName}")*
> *    .unmarshal(bindy)*
> *    .split(body())*
> *      .process(new MessageHandlingProcessor())*
> *      .dynamicRouter(method(new ExternalSystemRouter(), "route"))*
> *    .end()*
> *    .log("Done processing file: ${header.CamelFileName}");*
> *    *
> *  public class ExternalSystemRouter {*
> *    public String route(Exchange theExchange) throws
> UnsupportedEncodingException {*
> *      return "file:dynout";*
> *    }*
> *  }*
>
> The "route" method is not very dynamic but it's for illustrative purposes.
>
> What happens is that the route starts looping forever and processes the
> same file over and over again. The debug log looks like this:
>
> *...*
> *2012-12-28 10:04:10,552 | DEBUG | d #6 - file://in | BeanProcessor
>            | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,555 | DEBUG | d #6 - file://in | FileOperations
>             | el.component.file.FileOperations  367 | Using InputStream to
> write file: dynout\orderconf-order-id-20121228100410238.xml*
> *2012-12-28 10:04:10,559 | DEBUG | d #6 - file://in | GenericFileProducer
>            | mponent.file.GenericFileProducer  253 | Wrote
> [dynout\orderconf-order-id-20121228100410238.xml] to
> [Endpoint[file://dynout]]*
> *2012-12-28 10:04:10,562 | DEBUG | d #6 - file://in | BeanProcessor
>            | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,563 | DEBUG | d #6 - file://in | FileOperations
>             | el.component.file.FileOperations  367 | Using InputStream to
> write file: dynout\orderconf-order-id-20121228100410238.xml*
> *2012-12-28 10:04:10,567 | DEBUG | d #6 - file://in | GenericFileProducer
>            | mponent.file.GenericFileProducer  253 | Wrote
> [dynout\orderconf-order-id-20121228100410238.xml] to
> [Endpoint[file://dynout]]*
> *2012-12-28 10:04:10,579 | DEBUG | d #6 - file://in | BeanProcessor
>            | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,580 | DEBUG | d #6 - file://in | FileOperations
>             | el.component.file.FileOperations  367 | Using InputStream to
> write file: dynout\orderconf-order-id-20121228100410238.xml*
> *2012-12-28 10:04:10,585 | DEBUG | d #6 - file://in | GenericFileProducer
>            | mponent.file.GenericFileProducer  253 | Wrote
> [dynout\orderconf-order-id-20121228100410238.xml] to
> [Endpoint[file://dynout]]*
> *2012-12-28 10:04:10,595 | DEBUG | d #6 - file://in | BeanProcessor
>            | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,596 | DEBUG | d #6 - file://in | FileOperations
>             | el.component.file.FileOperations  367 | Using InputStream to
> write file: dynout\orderconf-order-id-20121228100410238.xml*
> *2012-12-28 10:04:10,601 | DEBUG | d #6 - file://in | GenericFileProducer
>            | mponent.file.GenericFileProducer  253 | Wrote
> [dynout\orderconf-order-id-20121228100410238.xml] to
> [Endpoint[file://dynout]]*
> *2012-12-28 10:04:10,610 | DEBUG | d #6 - file://in | BeanProcessor
>            | mel.component.bean.BeanProcessor  190 | Setting bean
> invocation result on the OUT message: file:dynout*
> *2012-12-28 10:04:10,613 | DEBUG | d #6 - file://in | FileOperations
>             | el.component.file.FileOperations  367 | Using InputStream to
> write file: dynout\orderconf-order-id-20121228100410238.xml*
> *...*
>
> This particular CSV file only has one row so only one output file should be
> created. It seems that the file that is being processed is never marked as
> "handled".
>
> I have a feeling that this is because the "end()" statement is not added to
> the route when I use the dynamic router. But how can I fix this?
>
> /Bengt



--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Reply | Threaded
Open this post in threaded view
|

Re: Problems with dynamic router and split

Bengt Rodehav
Thanks, will do,

/Bengt


2012/12/28 Claus Ibsen <[hidden email]>

> Hi
>
> See the content based router, or the dynamic recipient list EIPs.
>
> On Fri, Dec 28, 2012 at 10:12 AM, Bengt Rodehav <[hidden email]> wrote:
> > I'm using Camel 2.10.3.
> >
> > I have a route that parses a CSV file (using Bindy) and then splits the
> > exchange so that each row is treated by itself. Something like this:
> >
> > *  DataFormat bindy = new
> > BindyCsvDataFormat(ConfirmedOrder.class.getPackage().getName());*
> > *
> > *
> > *  from("file:in")*
> > *    .log("Processing file: ${header.CamelFileName}")*
> > *    .unmarshal(bindy)*
> > *    .split(body())*
> > *      .process(new MessageHandlingProcessor())*
> > *      .to("file:out)*
> > *    .end()*
> > *    .log("Done processing file: ${header.CamelFileName}");*
> >
> > This works fine. However, I now need to use dynamic routing so that each
> > message (each row in the CSV) is routed to a destination that depends on
> > one of the fields in the CSV. I therefore tried to use the dynamic
> router,
> > like this:
> >
> > *  from("file:in")*
> > *    .log("Processing file: ${header.CamelFileName}")*
> > *    .unmarshal(bindy)*
> > *    .split(body())*
> > *      .process(new MessageHandlingProcessor())*
> > *      .dynamicRouter(method(new ExternalSystemRouter(), "route"))*
> > *    .end()*
> > *    .log("Done processing file: ${header.CamelFileName}");*
> > *    *
> > *  public class ExternalSystemRouter {*
> > *    public String route(Exchange theExchange) throws
> > UnsupportedEncodingException {*
> > *      return "file:dynout";*
> > *    }*
> > *  }*
> >
> > The "route" method is not very dynamic but it's for illustrative
> purposes.
> >
> > What happens is that the route starts looping forever and processes the
> > same file over and over again. The debug log looks like this:
> >
> > *...*
> > *2012-12-28 10:04:10,552 | DEBUG | d #6 - file://in | BeanProcessor
> >            | mel.component.bean.BeanProcessor  190 | Setting bean
> > invocation result on the OUT message: file:dynout*
> > *2012-12-28 10:04:10,555 | DEBUG | d #6 - file://in | FileOperations
> >             | el.component.file.FileOperations  367 | Using InputStream
> to
> > write file: dynout\orderconf-order-id-20121228100410238.xml*
> > *2012-12-28 10:04:10,559 | DEBUG | d #6 - file://in | GenericFileProducer
> >            | mponent.file.GenericFileProducer  253 | Wrote
> > [dynout\orderconf-order-id-20121228100410238.xml] to
> > [Endpoint[file://dynout]]*
> > *2012-12-28 10:04:10,562 | DEBUG | d #6 - file://in | BeanProcessor
> >            | mel.component.bean.BeanProcessor  190 | Setting bean
> > invocation result on the OUT message: file:dynout*
> > *2012-12-28 10:04:10,563 | DEBUG | d #6 - file://in | FileOperations
> >             | el.component.file.FileOperations  367 | Using InputStream
> to
> > write file: dynout\orderconf-order-id-20121228100410238.xml*
> > *2012-12-28 10:04:10,567 | DEBUG | d #6 - file://in | GenericFileProducer
> >            | mponent.file.GenericFileProducer  253 | Wrote
> > [dynout\orderconf-order-id-20121228100410238.xml] to
> > [Endpoint[file://dynout]]*
> > *2012-12-28 10:04:10,579 | DEBUG | d #6 - file://in | BeanProcessor
> >            | mel.component.bean.BeanProcessor  190 | Setting bean
> > invocation result on the OUT message: file:dynout*
> > *2012-12-28 10:04:10,580 | DEBUG | d #6 - file://in | FileOperations
> >             | el.component.file.FileOperations  367 | Using InputStream
> to
> > write file: dynout\orderconf-order-id-20121228100410238.xml*
> > *2012-12-28 10:04:10,585 | DEBUG | d #6 - file://in | GenericFileProducer
> >            | mponent.file.GenericFileProducer  253 | Wrote
> > [dynout\orderconf-order-id-20121228100410238.xml] to
> > [Endpoint[file://dynout]]*
> > *2012-12-28 10:04:10,595 | DEBUG | d #6 - file://in | BeanProcessor
> >            | mel.component.bean.BeanProcessor  190 | Setting bean
> > invocation result on the OUT message: file:dynout*
> > *2012-12-28 10:04:10,596 | DEBUG | d #6 - file://in | FileOperations
> >             | el.component.file.FileOperations  367 | Using InputStream
> to
> > write file: dynout\orderconf-order-id-20121228100410238.xml*
> > *2012-12-28 10:04:10,601 | DEBUG | d #6 - file://in | GenericFileProducer
> >            | mponent.file.GenericFileProducer  253 | Wrote
> > [dynout\orderconf-order-id-20121228100410238.xml] to
> > [Endpoint[file://dynout]]*
> > *2012-12-28 10:04:10,610 | DEBUG | d #6 - file://in | BeanProcessor
> >            | mel.component.bean.BeanProcessor  190 | Setting bean
> > invocation result on the OUT message: file:dynout*
> > *2012-12-28 10:04:10,613 | DEBUG | d #6 - file://in | FileOperations
> >             | el.component.file.FileOperations  367 | Using InputStream
> to
> > write file: dynout\orderconf-order-id-20121228100410238.xml*
> > *...*
> >
> > This particular CSV file only has one row so only one output file should
> be
> > created. It seems that the file that is being processed is never marked
> as
> > "handled".
> >
> > I have a feeling that this is because the "end()" statement is not added
> to
> > the route when I use the dynamic router. But how can I fix this?
> >
> > /Bengt
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: [hidden email]
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
>