Multiprocessing a single message into multiple messages

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiprocessing a single message into multiple messages

wjmcdonald
How do I process a single message with multiple dynamic list of processors sending a new message from each processor to a filter/aggregator?

My task is to process a single message with many different processors.  Think of the message as describing a bunch of colored shapes.  One processor counts the number of squares, another counts the number of red objects, another collects all those shapes with 3 or more virticies (ie. not circles), each into a new message.  All of these processors (and I want them to be dynamic so I can add or subtract them to the set) need to process a single message from an endpoint.  After processing them, they route a new message.  The content of the message may be different:
Color Counters might be...
msg1: from the redShapesCounter processor impl.
 <count counter="redShapes">5</count>
msg2: from the blueShapesCounter processor impl.
 <count counter="blueShapes">2</count>
msg3: from the purpleShapesCounter processor impl.
 <count counter="purpleShapes">3</count>

A Vertices Lists msg from the verticiesLister impl might be...
 <shape type="withVerticies">
    <square id="s1">...
    <triangle id="t1">...
 </shape>)

After processing, all of these messages are sent to a queue for that type (eg. ShapColorCounters).

Next, a dynamic filter is used to select those messages that we want to report on.  A PrimaryColorOfShapesCountReport processor might aggregate all the primary color counter messages and produce a report with a total at the bottom.  It will filter only on primary color count messages.

Design:
=====
I was going to have a bean that setup routes for each processor:  sedaQ[i]->processor[i]->destQ[j]
The sedaQs would be fed using the RecipientList EIP from a <method> call to a bean to create the list of sedaQs.
Then destQ[j] -> filter(listOfThingsWeAreInterestedIn) -> aggregator(eg.PrimaryColorShapesReportGenerator) -> report.in.endpoint.

Is there a better way of doing this?  I'm thinking I'd like the processors (eg. counters) to be in parallel/asynch too.  I'm not quite sure if there is a better design of EIP usage...
Reply | Threaded
Open this post in threaded view
|

Re: Multiprocessing a single message into multiple messages

Christian Mueller
Administrator
Study the following EIP's:
- http://camel.apache.org/dynamic-router.html
- http://camel.apache.org/recipient-list.html

Best,
Christian

On Thu, Jul 5, 2012 at 11:04 PM, wjmcdonald <
[hidden email]> wrote:

> How do I process a single message with multiple dynamic list of processors
> sending a new message from each processor to a filter/aggregator?
>
> My task is to process a single message with many different processors.
> Think of the message as describing a bunch of colored shapes.  One
> processor
> counts the number of squares, another counts the number of red objects,
> another collects all those shapes with 3 or more virticies (ie. not
> circles), each into a new message.  All of these processors (and I want
> them
> to be dynamic so I can add or subtract them to the set) need to process a
> single message from an endpoint.  After processing them, they route a new
> message.  The content of the message may be different:
> Color Counters might be...
> msg1: from the redShapesCounter processor impl.
>  <count counter="redShapes">5</count>
> msg2: from the blueShapesCounter processor impl.
>  <count counter="blueShapes">2</count>
> msg3: from the purpleShapesCounter processor impl.
>  <count counter="purpleShapes">3</count>
>
> A Vertices Lists msg from the verticiesLister impl might be...
>  <shape type="withVerticies">
>     <square id="s1">...
>     <triangle id="t1">...
>  </shape>)
>
> After processing, all of these messages are sent to a queue for that type
> (eg. ShapColorCounters).
>
> Next, a dynamic filter is used to select those messages that we want to
> report on.  A PrimaryColorOfShapesCountReport processor might aggregate all
> the primary color counter messages and produce a report with a total at the
> bottom.  It will filter only on primary color count messages.
>
> Design:
> =====
> I was going to have a bean that setup routes for each processor:
> sedaQ[i]->processor[i]->destQ[j]
> The sedaQs would be fed using the RecipientList EIP from a <method> call to
> a bean to create the list of sedaQs.
> Then destQ[j] -> filter(listOfThingsWeAreInterestedIn) ->
> aggregator(eg.PrimaryColorShapesReportGenerator) -> report.in.endpoint.
>
> Is there a better way of doing this?  I'm thinking I'd like the processors
> (eg. counters) to be in parallel/asynch too.  I'm not quite sure if there
> is
> a better design of EIP usage...
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Multiprocessing-a-single-message-into-multiple-messages-tp5715576.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>