Apache Camel EIP to process batch of messages identifying those which can be filtered

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Apache Camel EIP to process batch of messages identifying those which can be filtered

gilboy
Hi

My route calls a DB procedure which returns a large number of order objects.

Before I send out the orders to a counterparty I want to examine the list of orders and see if the same order ID appears as both a new order and a cancel order, e.g. order 1001 returned as a new order and order 1001 returned as a cancel order. If I find any instances of the above I want to remove the order, i.e. not send 1001 out to counterparty.

I looked at using the aggregator EIP but don't think its a great fit. Just wondering if there is another more suitable EIP that fits this use case

Thanks
Joe
Reply | Threaded
Open this post in threaded view
|

Re: Apache Camel EIP to process batch of messages identifying those which can be filtered

Quinn Stevenson
Sounds like a good use case for the Idempotent Consumer EIP
http://camel.apache.org/idempotent-consumer.html <http://camel.apache.org/idempotent-consumer.html>



> On Feb 18, 2016, at 7:06 AM, gilboy <[hidden email]> wrote:
>
> Hi
>
> My route calls a DB procedure which returns a large number of order objects.
>
> Before I send out the orders to a counterparty I want to examine the list of
> orders and see if the same order ID appears as both a new order and a cancel
> order, e.g. order 1001 returned as a new order and order 1001 returned as a
> cancel order. If I find any instances of the above I want to remove the
> order, i.e. not send 1001 out to counterparty.
>
> I looked at using the aggregator EIP but don't think its a great fit. Just
> wondering if there is another more suitable EIP that fits this use case
>
> Thanks
> Joe
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Apache-Camel-EIP-to-process-batch-of-messages-identifying-those-which-can-be-filtered-tp5777844.html
> Sent from the Camel - Users mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Apache Camel EIP to process batch of messages identifying those which can be filtered

gilboy
Thanks for the quick feedback.

If I use the Idempotent Consumer EIP this would mean that the 2nd order (Order cancel) will be filtered out. However, I would want the new order to be filtered also. However, that would be sent to the counterparty with the Idempotent Consumer EIP

Regards
Joe
Reply | Threaded
Open this post in threaded view
|

Re: Apache Camel EIP to process batch of messages identifying those which can be filtered

Quinn Stevenson
How about a simple splitter that removes the duplicates?  For example, a route like this
<camelContext xmlns="http://camel.apache.org/schema/spring">
    <route>
        <from uri="file://src/data?noop=true" />
        <split strategyRef="removeDupsStrategy">
            <tokenize token="\n" />
            <setHeader headerName="MY_KEY">
                <simple>${body}</simple>
            </setHeader>
        </split>
        <log message="complete: ${body}" />
    </route>
</camelContext>

Could use a Splitter like this
public class RemoveDuplicateSplitterStrategry implements AggregationStrategy{
    @Override
    public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
        Logger log = LoggerFactory.getLogger(this.getClass());

        if ( null == oldExchange ) {
            Map<String, String> messageList = new HashMap<>();
            messageList.put( newExchange.getIn().getHeader("MY_KEY", String.class), newExchange.getIn().getBody(String.class) );
            newExchange.getIn().setBody( messageList);
            return newExchange;
        }

        Map<String, String> messageList = oldExchange.getIn().getBody( Map.class) ;
        String key = newExchange.getIn().getHeader("MY_KEY", String.class);
        if (messageList.containsKey(key)) {
            String value = messageList.remove(key);
            log.warn( "Removing duplicate for {}: {}", key, value);
        } else {
            messageList.put(key, newExchange.getIn().getBody(String.class) );
        }

        return oldExchange;
    }
}

> On Feb 18, 2016, at 10:17 AM, gilboy <[hidden email]> wrote:
>
> Thanks for the quick feedback.
>
> If I use the Idempotent Consumer EIP this would mean that the 2nd order
> (Order cancel) will be filtered out. However, I would want the new order to
> be filtered also. However, that would be sent to the counterparty with the
> Idempotent Consumer EIP
>
> Regards
> Joe
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Apache-Camel-EIP-to-process-batch-of-messages-identifying-those-which-can-be-filtered-tp5777844p5777868.html
> Sent from the Camel - Users mailing list archive at Nabble.com.