aggregator: dynamic completion size

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

aggregator: dynamic completion size

javamonkey79
It seems that the completionSize attribute of the aggregator will "make up the difference" if there are no more messages.

I'd like to be able to do something like:
                        <aggregate strategyRef="myAggregationStrategy" completionSize="500">
                                <correlationExpression>
                                        <constant>true</constant>
                                </correlationExpression>
                                <to id="out" uri="sftp://someone@something.com/test/" />
                        </aggregate>

Where, the from on this route is a queue that gets in 500+ messages at a time and it puts them together in batches of 500. This works fine, if you actually have 500 messages. However, if you have less than 500, it duplicates messages to make it 500.

I tried to use a completionPredicate instead, but I suspect that I have to do something that is a hybrid of a completionSize and a completionTimeout. Or perhaps I am over thinking this?

Is there a way to do what I am trying to do in Camel (v2.11 FWIW)?
Reply | Threaded
Open this post in threaded view
|

Re: aggregator: dynamic completion size

Claus Ibsen-2
Hi

Yeah read the docs
http://camel.apache.org/aggregator2

You can use dynamic completion size / timeout etc. to trigger when you
have < 500 messages.

On Fri, May 24, 2013 at 3:39 AM, javamonkey79 <[hidden email]> wrote:

> It seems that the completionSize attribute of the aggregator will "make up
> the difference" if there are no more messages.
>
> I'd like to be able to do something like:
>                         <aggregate strategyRef="myAggregationStrategy" completionSize="500">
>                                 <correlationExpression>
>                                         <constant>true</constant>
>                                 </correlationExpression>
>                                 <to id="out" uri="sftp://[hidden email]/test/" />
>                         </aggregate>
>
> Where, the from on this route is a queue that gets in 500+ messages at a
> time and it puts them together in batches of 500. This works fine, if you
> actually have 500 messages. However, if you have less than 500, it
> duplicates messages to make it 500.
>
> I tried to use a completionPredicate instead, but I suspect that I have to
> do something that is a hybrid of a completionSize and a completionTimeout.
> Or perhaps I am over thinking this?
>
> Is there a way to do what I am trying to do in Camel (v2.11 FWIW)?
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/aggregator-dynamic-completion-size-tp5733048.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



--
Claus Ibsen
-----------------
www.camelone.org: The open source integration conference.

Red Hat, Inc.
FuseSource is now part of Red Hat
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Reply | Threaded
Open this post in threaded view
|

Re: aggregator: dynamic completion size

javamonkey79
I did read the documentation, many times :)

It was not camel that was wrong - it was my unit test.

From what I can tell, if you set BOTH a completionSize and a completionTimeout  it will use both: e.g. if you set it to completionSize="500" completionTimeout="500" for example, it will stop when it gets too 500 messages or 500ms, whichever comes first. I don't know that this is clear in the documentation. Can you confirm what I am talking about?
Reply | Threaded
Open this post in threaded view
|

Re: aggregator: dynamic completion size

Claus Ibsen-2
Hi

Yes its very clear in the docs at
http://camel.apache.org/aggregator2

> Notice that all the completion ways are per correlation key. And you can combine them in any way you like. It's basically the first which triggers that wins. So you can use a completion size together with a completion timeout. Only completionTimeout and completionInterval cannot be used at the same time.


On Fri, May 24, 2013 at 7:20 PM, javamonkey79 <[hidden email]> wrote:

> I did read the documentation, many times :)
>
> It was not camel that was wrong - it was my unit test.
>
> From what I can tell, if you set BOTH a completionSize and a
> completionTimeout  it will use both: e.g. if you set it to
> completionSize="500" completionTimeout="500" for example, it will stop when
> it gets too 500 messages or 500ms, whichever comes first. I don't know that
> this is clear in the documentation. Can you confirm what I am talking about?
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/aggregator-dynamic-completion-size-tp5733048p5733132.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



--
Claus Ibsen
-----------------
www.camelone.org: The open source integration conference.

Red Hat, Inc.
FuseSource is now part of Red Hat
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen