Dynamic routing based on collection values

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Dynamic routing based on collection values

raghavender.anthwar@gmail.com
Hello All,

Can someone help me with a solution for this problem:

I want to route to different destinations based on the values that are present in the collection. For example, I've a map/any other collection with list of different values. I want to iterate over the collection and want to route the message to different destination based on the value of the collection.

Assume that my map contains below keys and values:

Map<String, List<String>> myMap = new HashMap<String, List<String>>();

I would like to iterate over the map and based on key I wanted to pass on the corresponding value list to a different route. Like:

If (Key == "Hyderabad") then route to Hyderabad uri etc.

Your help is much appreciated, Thank you.

Best,
Raghavender Anthwar
Reply | Threaded
Open this post in threaded view
|

Re: Dynamic routing based on collection values

ravi narayanan
Please have a look at recipient list EIP.
http://camel.apache.org/recipient-list.html

On Monday, 24 October 2016, [hidden email] <
[hidden email]> wrote:

> Hello All,
>
> Can someone help me with a solution for this problem:
>
> I want to route to different destinations based on the values that are
> present in the collection. For example, I've a map/any other collection
> with
> list of different values. I want to iterate over the collection and want to
> route the message to different destination based on the value of the
> collection.
>
> Assume that my map contains below keys and values:
>
> Map<String, List&lt;String>> myMap = new HashMap<String,
> List&lt;String>>();
>
> I would like to iterate over the map and based on key I wanted to pass on
> the corresponding value list to a different route. Like:
>
> If (Key == "Hyderabad") then route to Hyderabad uri etc.
>
> Your help is much appreciated, Thank you.
>
> Best,
> Raghavender Anthwar
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.
> com/Dynamic-routing-based-on-collection-values-tp5789157.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

AW: Dynamic routing based on collection values

Jan Matèrne (jhm)
In reply to this post by raghavender.anthwar@gmail.com
So you want to split [1] the list into Key-Value-Pairs and do a content
based routing [2]?

Jan


[1] http://camel.apache.org/splitter.html
[2] http://camel.apache.org/content-based-router.html 

> -----Ursprüngliche Nachricht-----
> Von: [hidden email]
> [mailto:[hidden email]]
> Gesendet: Montag, 24. Oktober 2016 09:48
> An: [hidden email]
> Betreff: Dynamic routing based on collection values
>
> Hello All,
>
> Can someone help me with a solution for this problem:
>
> I want to route to different destinations based on the values that are
> present in the collection. For example, I've a map/any other collection
> with list of different values. I want to iterate over the collection
> and want to route the message to different destination based on the
> value of the collection.
>
> Assume that my map contains below keys and values:
>
> Map<String, List&lt;String>> myMap = new HashMap<String,
> List&lt;String>>();
>
> I would like to iterate over the map and based on key I wanted to pass
> on the corresponding value list to a different route. Like:
>
> If (Key == "Hyderabad") then route to Hyderabad uri etc.
>
> Your help is much appreciated, Thank you.
>
> Best,
> Raghavender Anthwar
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Dynamic-routing-based-on-collection-
> values-tp5789157.html
> Sent from the Camel - Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Dynamic routing based on collection values

raghavender.anthwar@gmail.com
In reply to this post by ravi narayanan
Thanks for the quick response but I don't see whether recipient list can iterate over the map and send each of the value to different endpoint.

Can you point me to a specific example where the routing is done based on the collection values in the incoming message?

Sorry, am a beginner to Camel so looking for specific example. Thanks again.

Best,
Raghu
Reply | Threaded
Open this post in threaded view
|

Re: AW: Dynamic routing based on collection values

raghavender.anthwar@gmail.com
In reply to this post by Jan Matèrne (jhm)
Basically I've the map that was mentioned in the post as input in the incoming message. And I wanted to iterate over each key of the map and send it as input to different endpoint based on the key value.

Yes, it's kind of content based routing but the content is a collection here.
Reply | Threaded
Open this post in threaded view
|

Re: Dynamic routing based on collection values

raghavender.anthwar@gmail.com
In reply to this post by ravi narayanan
I've gone through the document of recipient list and seems it send the same copy of the message to all routes.

I'm looking for sending each value of the map to a different router based on the key.

Thank you.
Reply | Threaded
Open this post in threaded view
|

Re: Dynamic routing based on collection values

Steve973
Have you tried first using splitter, and then using a choice to determine
the route?  From the example in "Composed Message Processor", they do
something similar:

// split up the order so individual OrderItems can be validated by the
appropriate bean
from("direct:start")
    .split().body()
    .choice()
        .when().method("orderItemHelper", "isWidget")
            .to("bean:widgetInventory")
        .otherwise()
            .to("bean:gadgetInventory")
    .end()
    .to("seda:aggregate");

// collect and re-assemble the validated OrderItems into an order again
from("seda:aggregate")
    .aggregate(new
MyOrderAggregationStrategy()).header("orderId").completionTimeout(1000L)
        .to("mock:result");

Of course, your "when" calls will be different to look at the key values.


On Mon, Oct 24, 2016 at 5:03 AM, [hidden email] <
[hidden email]> wrote:

> I've gone through the document of recipient list and seems it send the same
> copy of the message to all routes.
>
> I'm looking for sending each value of the map to a different router based
> on
> the key.
>
> Thank you.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.
> com/Dynamic-routing-based-on-collection-values-tp5789157p5789165.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Dynamic routing based on collection values

Steve973
Note that you will have to split on the map's entryset, since a map,
itself, is not iterable.  This is probably obvious, but I wanted to mention
it anyway, since splitting requires an iterable to split.

On Mon, Oct 24, 2016 at 5:16 AM, Steve973 <[hidden email]> wrote:

> Have you tried first using splitter, and then using a choice to determine
> the route?  From the example in "Composed Message Processor", they do
> something similar:
>
> // split up the order so individual OrderItems can be validated by the
> appropriate bean
> from("direct:start")
>     .split().body()
>     .choice()
>         .when().method("orderItemHelper", "isWidget")
>             .to("bean:widgetInventory")
>         .otherwise()
>             .to("bean:gadgetInventory")
>     .end()
>     .to("seda:aggregate");
>
> // collect and re-assemble the validated OrderItems into an order again
> from("seda:aggregate")
>     .aggregate(new MyOrderAggregationStrategy()).header("orderId").
> completionTimeout(1000L)
>         .to("mock:result");
>
> Of course, your "when" calls will be different to look at the key values.
>
>
> On Mon, Oct 24, 2016 at 5:03 AM, [hidden email] <
> [hidden email]> wrote:
>
>> I've gone through the document of recipient list and seems it send the
>> same
>> copy of the message to all routes.
>>
>> I'm looking for sending each value of the map to a different router based
>> on
>> the key.
>>
>> Thank you.
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.
>> com/Dynamic-routing-based-on-collection-values-tp5789157p5789165.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Dynamic routing based on collection values

raghavender.anthwar@gmail.com
Thanks will tr the split method. Is there any example available on the web to split the collection? I searched but couldn't find much help.
Reply | Threaded
Open this post in threaded view
|

Re: Dynamic routing based on collection values

Steve973
The easiest way that I can think of is to send map.entryset in the message
body instead of the map itself.  Then you can simply do .split().body().
That should work for you.

On Mon, Oct 24, 2016 at 5:52 AM, [hidden email] <
[hidden email]> wrote:

> Thanks will tr the split method. Is there any example available on the web
> to
> split the collection? I searched but couldn't find much help.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.
> com/Dynamic-routing-based-on-collection-values-tp5789157p5789169.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

AW: Dynamic routing based on collection values

Jan Matèrne (jhm)
In reply to this post by raghavender.anthwar@gmail.com
Here an example what I unterstood.

Jan



package de.materne.camel;

import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;

import org.apache.camel.EndpointInject;
import org.apache.camel.RoutesBuilder;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.junit.Test;

public class IterateOverMapTest extends CamelTestSupport {

        @EndpointInject(uri="mock:foo")
        private MockEndpoint foo;
       
        @EndpointInject(uri="mock:bar")
        private MockEndpoint bar;
       
        @Test
        public void test() throws InterruptedException {
                Map<String, String> data = new HashMap<>();
                data.put("foo.suffix", "One");
                data.put("foo.anotherSuffix", "Two");
                data.put("bar.suffix", "Three");
               
                foo.expectedMessageCount(2);
                bar.expectedMessageCount(1);
                template().sendBody("direct:in", data);
               
                assertMockEndpointsSatisfied(1, TimeUnit.SECONDS);
        }
       
        @Override
        protected RoutesBuilder createRouteBuilder() throws Exception {
                return new RouteBuilder() {
                        @Override
                        public void configure() throws Exception {
                                from("direct:in")
                                        // for splitting, we need an
Iterable
       
.transform(simple("${in.body.entrySet()}"))
                                        // now simple split over the
Iterable
                                        .split(body())
                                        // content based routing on key
prefix
                                        .choice()
       
.when(simple("${in.body.key()} starts with 'foo'"))
                                                        .to("mock:foo")
       
.when(simple("${in.body.key()} starts with 'bar'"))
                                                        .to("mock:bar")
                                                .otherwise()
                                                        .to("mock:end")
                                ;
                        }
                };
        }
       
}



> -----Ursprüngliche Nachricht-----
> Von: [hidden email]
> [mailto:[hidden email]]
> Gesendet: Montag, 24. Oktober 2016 11:53
> An: [hidden email]
> Betreff: Re: Dynamic routing based on collection values
>
> Thanks will tr the split method. Is there any example available on the
> web to split the collection? I searched but couldn't find much help.
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Dynamic-routing-based-on-collection-
> values-tp5789157p5789169.html
> Sent from the Camel - Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: AW: Dynamic routing based on collection values

raghavender.anthwar@gmail.com
Thanks a lot for the response. This seems like useful, I'll give out a try and will update you accordingly.

Best,
Raghavender Anthwar
Reply | Threaded
Open this post in threaded view
|

Re: AW: Dynamic routing based on collection values

Brad Johnson
This is also one of those fuzzy areas where sometimes it is easier to
simply pass the message body into a bean and then use ProducerTemplate
instances to send values us different routes.  Under normal circumstances I
wouldn't do it that way as Camel does a great job of heavy lifting.  But in
some cases just using a Java bean is simpler and cleaner. Imagine having a
CityRouteHandler with one method on it.


public void routeToCities(Map<String, List<String>> myMap){
{
         for (Map.Entry<String, List<String>> entry : map.entrySet()) {
if("Hyderabad".equals(entry.getKey()){
for(String message: entry.getValue())
hyderabad.sendBody(message);
...etc.
}
}
}

You could then create a helper method as well that instead of looping
inside for each new entry and sending to the producer template, have the
helper method take the List<String> and producer template and loop inside
it.  That would simplify the map code.

The ProducerTemplate entries in this class would be declared something like
this:

@InjectEndpoint(uri="direct:hyderabad")
ProducerTemplate hyderabad;

I've written this free hand so while the ideas are correct I can't vouch
for the correctness of the code itself.





On Mon, Oct 24, 2016 at 8:04 AM, [hidden email] <
[hidden email]> wrote:

> Thanks a lot for the response. This seems like useful, I'll give out a try
> and will update you accordingly.
>
> Best,
> Raghavender Anthwar
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.
> com/Dynamic-routing-based-on-collection-values-tp5789157p5789179.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: AW: Dynamic routing based on collection values

Steve973
I'd worry about the overhead of creating so many producer templates. I
would probably set a header that contains a map of keys to destinations,
then you could set a recipient list of one endpoint that contains the value
at that key. Maybe brad's method isn't as overhead intensive because the
template is injected, but this is more manual implementation than i would
prefer.

On Oct 24, 2016 9:21 AM, "Brad Johnson" <[hidden email]>
wrote:

> This is also one of those fuzzy areas where sometimes it is easier to
> simply pass the message body into a bean and then use ProducerTemplate
> instances to send values us different routes.  Under normal circumstances I
> wouldn't do it that way as Camel does a great job of heavy lifting.  But in
> some cases just using a Java bean is simpler and cleaner. Imagine having a
> CityRouteHandler with one method on it.
>
>
> public void routeToCities(Map<String, List<String>> myMap){
> {
>          for (Map.Entry<String, List<String>> entry : map.entrySet()) {
> if("Hyderabad".equals(entry.getKey()){
> for(String message: entry.getValue())
> hyderabad.sendBody(message);
> ...etc.
> }
> }
> }
>
> You could then create a helper method as well that instead of looping
> inside for each new entry and sending to the producer template, have the
> helper method take the List<String> and producer template and loop inside
> it.  That would simplify the map code.
>
> The ProducerTemplate entries in this class would be declared something like
> this:
>
> @InjectEndpoint(uri="direct:hyderabad")
> ProducerTemplate hyderabad;
>
> I've written this free hand so while the ideas are correct I can't vouch
> for the correctness of the code itself.
>
>
>
>
>
> On Mon, Oct 24, 2016 at 8:04 AM, [hidden email] <
> [hidden email]> wrote:
>
> > Thanks a lot for the response. This seems like useful, I'll give out a
> try
> > and will update you accordingly.
> >
> > Best,
> > Raghavender Anthwar
> >
> >
> >
> > --
> > View this message in context: http://camel.465427.n5.nabble.
> > com/Dynamic-routing-based-on-collection-values-tp5789157p5789179.html
> > Sent from the Camel - Users mailing list archive at Nabble.com.
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: AW: Dynamic routing based on collection values

Brad Johnson
How many producer templates would be produced?  There's only a single
handler instantiated so if there are 5 routes that's 5 producer templates.

On Mon, Oct 24, 2016 at 1:32 PM, Steve973 <[hidden email]> wrote:

> I'd worry about the overhead of creating so many producer templates. I
> would probably set a header that contains a map of keys to destinations,
> then you could set a recipient list of one endpoint that contains the value
> at that key. Maybe brad's method isn't as overhead intensive because the
> template is injected, but this is more manual implementation than i would
> prefer.
>
> On Oct 24, 2016 9:21 AM, "Brad Johnson" <[hidden email]>
> wrote:
>
> > This is also one of those fuzzy areas where sometimes it is easier to
> > simply pass the message body into a bean and then use ProducerTemplate
> > instances to send values us different routes.  Under normal
> circumstances I
> > wouldn't do it that way as Camel does a great job of heavy lifting.  But
> in
> > some cases just using a Java bean is simpler and cleaner. Imagine having
> a
> > CityRouteHandler with one method on it.
> >
> >
> > public void routeToCities(Map<String, List<String>> myMap){
> > {
> >          for (Map.Entry<String, List<String>> entry : map.entrySet()) {
> > if("Hyderabad".equals(entry.getKey()){
> > for(String message: entry.getValue())
> > hyderabad.sendBody(message);
> > ...etc.
> > }
> > }
> > }
> >
> > You could then create a helper method as well that instead of looping
> > inside for each new entry and sending to the producer template, have the
> > helper method take the List<String> and producer template and loop inside
> > it.  That would simplify the map code.
> >
> > The ProducerTemplate entries in this class would be declared something
> like
> > this:
> >
> > @InjectEndpoint(uri="direct:hyderabad")
> > ProducerTemplate hyderabad;
> >
> > I've written this free hand so while the ideas are correct I can't vouch
> > for the correctness of the code itself.
> >
> >
> >
> >
> >
> > On Mon, Oct 24, 2016 at 8:04 AM, [hidden email] <
> > [hidden email]> wrote:
> >
> > > Thanks a lot for the response. This seems like useful, I'll give out a
> > try
> > > and will update you accordingly.
> > >
> > > Best,
> > > Raghavender Anthwar
> > >
> > >
> > >
> > > --
> > > View this message in context: http://camel.465427.n5.nabble.
> > > com/Dynamic-routing-based-on-collection-values-tp5789157p5789179.html
> > > Sent from the Camel - Users mailing list archive at Nabble.com.
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: AW: Dynamic routing based on collection values

Steve973
And, by default, that is a lot of threads that are not needed for his use
case.  Though I am not sure how many threads are created vs used when you
are doing a recipient list and a splitter.

On Mon, Oct 24, 2016 at 2:51 PM, Brad Johnson <[hidden email]>
wrote:

> How many producer templates would be produced?  There's only a single
> handler instantiated so if there are 5 routes that's 5 producer templates.
>
> On Mon, Oct 24, 2016 at 1:32 PM, Steve973 <[hidden email]> wrote:
>
> > I'd worry about the overhead of creating so many producer templates. I
> > would probably set a header that contains a map of keys to destinations,
> > then you could set a recipient list of one endpoint that contains the
> value
> > at that key. Maybe brad's method isn't as overhead intensive because the
> > template is injected, but this is more manual implementation than i would
> > prefer.
> >
> > On Oct 24, 2016 9:21 AM, "Brad Johnson" <[hidden email]>
> > wrote:
> >
> > > This is also one of those fuzzy areas where sometimes it is easier to
> > > simply pass the message body into a bean and then use ProducerTemplate
> > > instances to send values us different routes.  Under normal
> > circumstances I
> > > wouldn't do it that way as Camel does a great job of heavy lifting.
> But
> > in
> > > some cases just using a Java bean is simpler and cleaner. Imagine
> having
> > a
> > > CityRouteHandler with one method on it.
> > >
> > >
> > > public void routeToCities(Map<String, List<String>> myMap){
> > > {
> > >          for (Map.Entry<String, List<String>> entry : map.entrySet()) {
> > > if("Hyderabad".equals(entry.getKey()){
> > > for(String message: entry.getValue())
> > > hyderabad.sendBody(message);
> > > ...etc.
> > > }
> > > }
> > > }
> > >
> > > You could then create a helper method as well that instead of looping
> > > inside for each new entry and sending to the producer template, have
> the
> > > helper method take the List<String> and producer template and loop
> inside
> > > it.  That would simplify the map code.
> > >
> > > The ProducerTemplate entries in this class would be declared something
> > like
> > > this:
> > >
> > > @InjectEndpoint(uri="direct:hyderabad")
> > > ProducerTemplate hyderabad;
> > >
> > > I've written this free hand so while the ideas are correct I can't
> vouch
> > > for the correctness of the code itself.
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Oct 24, 2016 at 8:04 AM, [hidden email] <
> > > [hidden email]> wrote:
> > >
> > > > Thanks a lot for the response. This seems like useful, I'll give out
> a
> > > try
> > > > and will update you accordingly.
> > > >
> > > > Best,
> > > > Raghavender Anthwar
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context: http://camel.465427.n5.nabble.
> > > > com/Dynamic-routing-based-on-collection-values-
> tp5789157p5789179.html
> > > > Sent from the Camel - Users mailing list archive at Nabble.com.
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: AW: Dynamic routing based on collection values

raghavender.anthwar@gmail.com
Thanks a ton everyone, I was able to achieve it by using split and by writing a custom bean/method.
Reply | Threaded
Open this post in threaded view
|

Re: Dynamic routing based on collection values

Ryan T
In reply to this post by raghavender.anthwar@gmail.com
Here is another option to look at that takes a little bit different approach.


import org.apache.camel.Exchange;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.junit.Test;

import java.util.ArrayList;
import java.util.Collection;

public class CamelTestExamples2 extends CamelTestSupport {

    @Test
    public void test() throws Exception {

        Collection<Message> messages = new ArrayList<>();
        Message message = new Message();
        message.setRoute("direct:north");
        message.setHeader("northHeader");
        message.setBody("northBody");
        messages.add(message);

        message = new Message();
        message.setRoute("direct:east");
        message.setHeader("eastHeader");
        message.setBody("eastBody");
        messages.add(message);

        message = new Message();
        message.setRoute("direct:south");
        message.setHeader("southHeader");
        message.setBody("southBody");
        messages.add(message);

        message = new Message();
        message.setRoute("direct:west");
        message.setHeader("westHeader");
        message.setBody("westBody");
        messages.add(message);

        template.request("direct:collection", (Exchange exchange) -> {
            exchange.getIn().setBody(messages);
        });
    }

    @Override
    protected RouteBuilder createRouteBuilder() throws Exception {
        return new RouteBuilder() {

            @Override
            public void configure() throws Exception {

                from("direct:collection")
                        .split(body()).recipientList(bodyAs(Message.class).method("route"));

                from("direct:north").to("log:input");

                from("direct:east").to("log:input");

                from("direct:south").to("log:input");

                from("direct:west").to("log:input");
            }
        };
    }


}
Reply | Threaded
Open this post in threaded view
|

Re: AW: Dynamic routing based on collection values

Brad Johnson
In reply to this post by raghavender.anthwar@gmail.com
@Steve

I'm fairly certain the ProducerTemplate is simply an object and doesn't
have its own separate thread.  That would create a lot of different
problems given the nature of the methods on the ProducerTemplate and the
@EndpointInject.

Even when you make async calls using it the PT uses an ExecutorService
which has a thread pool.  But how else are you going to do an async call
with or with Callback/Future? Having the ExectorService at least means you
aren't spinning up a lot of new threads and then throwing them away.



On Wed, Oct 26, 2016 at 6:44 AM, [hidden email] <
[hidden email]> wrote:

> Thanks a ton everyone, I was able to achieve it by using split and by
> writing
> a custom bean/method.
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.
> com/Dynamic-routing-based-on-collection-values-tp5789157p5789282.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>