Quantcast

[DISCUSS] Message Store

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[DISCUSS] Message Store

Christian Ohr
Hi everyone,

in the past I came several times across situations that required the
one or other kind of Message Store. I noticed that in Camel this
doesn't seem to be a "first class citizen" in the sense of a primary
architectural concept, which can be applied consistently whereever
needed.

In short, it might make sense to have a unified, generic, pluggable
Message Store (probably more of an "Exchange store") in Camel that
consolidates the different approaches and allows to similarly
parameterize persistence to various EIP patterns, and can be used
independently of EIP patterns as well. Implementations would handle
the mapping to the underlying database or file system or NoSQL or
whatever.
It seems that Spring Integration provides something corresponding
(http://static.springsource.org/spring-integration/reference/htmlsingle/#message-store).

Message Store implementations are already used by Camel in various
places, although using different approaches :
* Stream Caching (only file system)
* AggregationRepository (used for splitters, multicasts etc., but the
interface is not specific at all to aggregation use cases)
* IdempotentRepository

Message Store is requested for in other places:
* Reliable stream resequencing (CAMEL-949)
* Persistent Dead Letter Queue (CAMEL-4575)

And there might be other areas (seda, bam) that might benefit as well.
Maybe generalizing the AggregationRepository is a way to go forward.

Side note: The OeHF IPF platform (which is built on top of Camel and
partly extends it to the health care domain) has something called a
"Flow Manager" (http://www.openehealth.org/display/ipf2/Flow+management)
that is used for tracking exchanges while they are processed or after
processing is done, thereby being able to re-insert them into the
route. Not that I consider this being a shining example, but it shows
that message stores make sense outside their implicit use in EIP
processors as well.

Looking forward to your opinions!

regards
Christian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

hadrian
Hi Christian,

You are correct about the fact that a Message Store is missing (kinda).
There are a few components that support a message store implementation
of sorts, as you mentioned. You are also correct that it should be a
first class citizen.

Practically, an endpoint that supports a polling consumer is a message
store. What is actually missing is random access via a polling consumer.
A message store would then work atop any endpoint that supports random
retrieval (polling). That would create the opportunity to cleanup a bit
the pollEnrich implementation as well.

My $0.02,
Hadrian


On 10/23/2012 07:18 AM, Christian Ohr wrote:

> Hi everyone,
>
> in the past I came several times across situations that required the
> one or other kind of Message Store. I noticed that in Camel this
> doesn't seem to be a "first class citizen" in the sense of a primary
> architectural concept, which can be applied consistently whereever
> needed.
>
> In short, it might make sense to have a unified, generic, pluggable
> Message Store (probably more of an "Exchange store") in Camel that
> consolidates the different approaches and allows to similarly
> parameterize persistence to various EIP patterns, and can be used
> independently of EIP patterns as well. Implementations would handle
> the mapping to the underlying database or file system or NoSQL or
> whatever.
> It seems that Spring Integration provides something corresponding
> (http://static.springsource.org/spring-integration/reference/htmlsingle/#message-store).
>
> Message Store implementations are already used by Camel in various
> places, although using different approaches :
> * Stream Caching (only file system)
> * AggregationRepository (used for splitters, multicasts etc., but the
> interface is not specific at all to aggregation use cases)
> * IdempotentRepository
>
> Message Store is requested for in other places:
> * Reliable stream resequencing (CAMEL-949)
> * Persistent Dead Letter Queue (CAMEL-4575)
>
> And there might be other areas (seda, bam) that might benefit as well.
> Maybe generalizing the AggregationRepository is a way to go forward.
>
> Side note: The OeHF IPF platform (which is built on top of Camel and
> partly extends it to the health care domain) has something called a
> "Flow Manager" (http://www.openehealth.org/display/ipf2/Flow+management)
> that is used for tracking exchanges while they are processed or after
> processing is done, thereby being able to re-insert them into the
> route. Not that I consider this being a shining example, but it shows
> that message stores make sense outside their implicit use in EIP
> processors as well.
>
> Looking forward to your opinions!
>
> regards
> Christian
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

Willem.Jiang
There is a lots of way to implement to Message store.
Option1. If we treat is a first class of Camel, we could build up a set of API to let aggregator or tracer to use. As it is common usage that we need to store the exchange persistently, It make sense to provides such kind of API in Camel as a first class. +1 to provide this kind of API in Camel 3.x.

Option2. We can also leverage the already exits camel components such as polling consumer or camel-JPA components to implement a such Message store. I think IPF shows us a good way to go. In this way we don't need to wait for the New API from Camel, it could be good solution for Camel 2.x.

Maybe it is a good time for us to think about Camel 3.0 again.

Any thoughts?

Willem

--
Willem Jiang

Red Hat, Inc.
FuseSource is now part of Red Hat
Web: http://www.fusesource.com | http://www.redhat.com
Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/) (English)
          http://jnn.javaeye.com (http://jnn.javaeye.com/) (Chinese)
Twitter: willemjiang
Weibo: willemjiang





On Wednesday, October 24, 2012 at 9:43 AM, Hadrian Zbarcea wrote:

> Hi Christian,
>
> You are correct about the fact that a Message Store is missing (kinda).
> There are a few components that support a message store implementation
> of sorts, as you mentioned. You are also correct that it should be a
> first class citizen.
>
> Practically, an endpoint that supports a polling consumer is a message
> store. What is actually missing is random access via a polling consumer.
> A message store would then work atop any endpoint that supports random
> retrieval (polling). That would create the opportunity to cleanup a bit
> the pollEnrich implementation as well.
>
> My $0.02,
> Hadrian
>
>
> On 10/23/2012 07:18 AM, Christian Ohr wrote:
> > Hi everyone,
> >
> > in the past I came several times across situations that required the
> > one or other kind of Message Store. I noticed that in Camel this
> > doesn't seem to be a "first class citizen" in the sense of a primary
> > architectural concept, which can be applied consistently whereever
> > needed.
> >
> > In short, it might make sense to have a unified, generic, pluggable
> > Message Store (probably more of an "Exchange store") in Camel that
> > consolidates the different approaches and allows to similarly
> > parameterize persistence to various EIP patterns, and can be used
> > independently of EIP patterns as well. Implementations would handle
> > the mapping to the underlying database or file system or NoSQL or
> > whatever.
> > It seems that Spring Integration provides something corresponding
> > (http://static.springsource.org/spring-integration/reference/htmlsingle/#message-store).
> >
> > Message Store implementations are already used by Camel in various
> > places, although using different approaches :
> > * Stream Caching (only file system)
> > * AggregationRepository (used for splitters, multicasts etc., but the
> > interface is not specific at all to aggregation use cases)
> > * IdempotentRepository
> >
> > Message Store is requested for in other places:
> > * Reliable stream resequencing (CAMEL-949)
> > * Persistent Dead Letter Queue (CAMEL-4575)
> >
> > And there might be other areas (seda, bam) that might benefit as well.
> > Maybe generalizing the AggregationRepository is a way to go forward.
> >
> > Side note: The OeHF IPF platform (which is built on top of Camel and
> > partly extends it to the health care domain) has something called a
> > "Flow Manager" (http://www.openehealth.org/display/ipf2/Flow+management)
> > that is used for tracking exchanges while they are processed or after
> > processing is done, thereby being able to re-insert them into the
> > route. Not that I consider this being a shining example, but it shows
> > that message stores make sense outside their implicit use in EIP
> > processors as well.
> >
> > Looking forward to your opinions!
> >
> > regards
> > Christian
>



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

Bengt Rodehav
I think this is an excellent idea. We're using Camel as the basis for our
integration platform and a message store is a mandatory component. We need:

- To store all message traffic in order to have an audit trail. Both
successful and failed exchanges.
- To provide a certain level of manual retry. That is to get the original
message from the store and feed it back in the originating route.
- Flexibility to specify what should be stored (e g what exchange- and
message properties) and also filter what exchanges should be stored.

Currently we have implemented our own message store but it requires us to
adapt the routes to make this happen. A more generic way of handling this
would be terrific.

/Bengt

2012/10/24 Willem jiang <[hidden email]>

> There is a lots of way to implement to Message store.
> Option1. If we treat is a first class of Camel, we could build up a set of
> API to let aggregator or tracer to use. As it is common usage that we need
> to store the exchange persistently, It make sense to provides such kind of
> API in Camel as a first class. +1 to provide this kind of API in Camel 3.x.
>
> Option2. We can also leverage the already exits camel components such as
> polling consumer or camel-JPA components to implement a such Message store.
> I think IPF shows us a good way to go. In this way we don't need to wait
> for the New API from Camel, it could be good solution for Camel 2.x.
>
> Maybe it is a good time for us to think about Camel 3.0 again.
>
> Any thoughts?
>
> Willem
>
> --
> Willem Jiang
>
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Web: http://www.fusesource.com | http://www.redhat.com
> Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/)
> (English)
>           http://jnn.javaeye.com (http://jnn.javaeye.com/) (Chinese)
> Twitter: willemjiang
> Weibo: willemjiang
>
>
>
>
>
> On Wednesday, October 24, 2012 at 9:43 AM, Hadrian Zbarcea wrote:
>
> > Hi Christian,
> >
> > You are correct about the fact that a Message Store is missing (kinda).
> > There are a few components that support a message store implementation
> > of sorts, as you mentioned. You are also correct that it should be a
> > first class citizen.
> >
> > Practically, an endpoint that supports a polling consumer is a message
> > store. What is actually missing is random access via a polling consumer.
> > A message store would then work atop any endpoint that supports random
> > retrieval (polling). That would create the opportunity to cleanup a bit
> > the pollEnrich implementation as well.
> >
> > My $0.02,
> > Hadrian
> >
> >
> > On 10/23/2012 07:18 AM, Christian Ohr wrote:
> > > Hi everyone,
> > >
> > > in the past I came several times across situations that required the
> > > one or other kind of Message Store. I noticed that in Camel this
> > > doesn't seem to be a "first class citizen" in the sense of a primary
> > > architectural concept, which can be applied consistently whereever
> > > needed.
> > >
> > > In short, it might make sense to have a unified, generic, pluggable
> > > Message Store (probably more of an "Exchange store") in Camel that
> > > consolidates the different approaches and allows to similarly
> > > parameterize persistence to various EIP patterns, and can be used
> > > independently of EIP patterns as well. Implementations would handle
> > > the mapping to the underlying database or file system or NoSQL or
> > > whatever.
> > > It seems that Spring Integration provides something corresponding
> > > (
> http://static.springsource.org/spring-integration/reference/htmlsingle/#message-store
> ).
> > >
> > > Message Store implementations are already used by Camel in various
> > > places, although using different approaches :
> > > * Stream Caching (only file system)
> > > * AggregationRepository (used for splitters, multicasts etc., but the
> > > interface is not specific at all to aggregation use cases)
> > > * IdempotentRepository
> > >
> > > Message Store is requested for in other places:
> > > * Reliable stream resequencing (CAMEL-949)
> > > * Persistent Dead Letter Queue (CAMEL-4575)
> > >
> > > And there might be other areas (seda, bam) that might benefit as well.
> > > Maybe generalizing the AggregationRepository is a way to go forward.
> > >
> > > Side note: The OeHF IPF platform (which is built on top of Camel and
> > > partly extends it to the health care domain) has something called a
> > > "Flow Manager" (
> http://www.openehealth.org/display/ipf2/Flow+management)
> > > that is used for tracking exchanges while they are processed or after
> > > processing is done, thereby being able to re-insert them into the
> > > route. Not that I consider this being a shining example, but it shows
> > > that message stores make sense outside their implicit use in EIP
> > > processors as well.
> > >
> > > Looking forward to your opinions!
> > >
> > > regards
> > > Christian
> >
>
>
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

aedwards
+1 for this feature.  It is very much needed.  Agree with Bengt's comments.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

Claus Ibsen-2
In reply to this post by Bengt Rodehav
On Wed, Oct 24, 2012 at 8:46 AM, Bengt Rodehav <[hidden email]> wrote:

> I think this is an excellent idea. We're using Camel as the basis for our
> integration platform and a message store is a mandatory component. We need:
>
> - To store all message traffic in order to have an audit trail. Both
> successful and failed exchanges.
> - To provide a certain level of manual retry. That is to get the original
> message from the store and feed it back in the originating route.
> - Flexibility to specify what should be stored (e g what exchange- and
> message properties) and also filter what exchanges should be stored.
>
> Currently we have implemented our own message store but it requires us to
> adapt the routes to make this happen. A more generic way of handling this
> would be terrific.
>

The Message History EIP is scheduled for overhaul/improvement in Camel 3.0.
And in this light we will look at a message store API as well.



> /Bengt
>
> 2012/10/24 Willem jiang <[hidden email]>
>
>> There is a lots of way to implement to Message store.
>> Option1. If we treat is a first class of Camel, we could build up a set of
>> API to let aggregator or tracer to use. As it is common usage that we need
>> to store the exchange persistently, It make sense to provides such kind of
>> API in Camel as a first class. +1 to provide this kind of API in Camel 3.x.
>>
>> Option2. We can also leverage the already exits camel components such as
>> polling consumer or camel-JPA components to implement a such Message store.
>> I think IPF shows us a good way to go. In this way we don't need to wait
>> for the New API from Camel, it could be good solution for Camel 2.x.
>>
>> Maybe it is a good time for us to think about Camel 3.0 again.
>>
>> Any thoughts?
>>
>> Willem
>>
>> --
>> Willem Jiang
>>
>> Red Hat, Inc.
>> FuseSource is now part of Red Hat
>> Web: http://www.fusesource.com | http://www.redhat.com
>> Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/)
>> (English)
>>           http://jnn.javaeye.com (http://jnn.javaeye.com/) (Chinese)
>> Twitter: willemjiang
>> Weibo: willemjiang
>>
>>
>>
>>
>>
>> On Wednesday, October 24, 2012 at 9:43 AM, Hadrian Zbarcea wrote:
>>
>> > Hi Christian,
>> >
>> > You are correct about the fact that a Message Store is missing (kinda).
>> > There are a few components that support a message store implementation
>> > of sorts, as you mentioned. You are also correct that it should be a
>> > first class citizen.
>> >
>> > Practically, an endpoint that supports a polling consumer is a message
>> > store. What is actually missing is random access via a polling consumer.
>> > A message store would then work atop any endpoint that supports random
>> > retrieval (polling). That would create the opportunity to cleanup a bit
>> > the pollEnrich implementation as well.
>> >
>> > My $0.02,
>> > Hadrian
>> >
>> >
>> > On 10/23/2012 07:18 AM, Christian Ohr wrote:
>> > > Hi everyone,
>> > >
>> > > in the past I came several times across situations that required the
>> > > one or other kind of Message Store. I noticed that in Camel this
>> > > doesn't seem to be a "first class citizen" in the sense of a primary
>> > > architectural concept, which can be applied consistently whereever
>> > > needed.
>> > >
>> > > In short, it might make sense to have a unified, generic, pluggable
>> > > Message Store (probably more of an "Exchange store") in Camel that
>> > > consolidates the different approaches and allows to similarly
>> > > parameterize persistence to various EIP patterns, and can be used
>> > > independently of EIP patterns as well. Implementations would handle
>> > > the mapping to the underlying database or file system or NoSQL or
>> > > whatever.
>> > > It seems that Spring Integration provides something corresponding
>> > > (
>> http://static.springsource.org/spring-integration/reference/htmlsingle/#message-store
>> ).
>> > >
>> > > Message Store implementations are already used by Camel in various
>> > > places, although using different approaches :
>> > > * Stream Caching (only file system)
>> > > * AggregationRepository (used for splitters, multicasts etc., but the
>> > > interface is not specific at all to aggregation use cases)
>> > > * IdempotentRepository
>> > >
>> > > Message Store is requested for in other places:
>> > > * Reliable stream resequencing (CAMEL-949)
>> > > * Persistent Dead Letter Queue (CAMEL-4575)
>> > >
>> > > And there might be other areas (seda, bam) that might benefit as well.
>> > > Maybe generalizing the AggregationRepository is a way to go forward.
>> > >
>> > > Side note: The OeHF IPF platform (which is built on top of Camel and
>> > > partly extends it to the health care domain) has something called a
>> > > "Flow Manager" (
>> http://www.openehealth.org/display/ipf2/Flow+management)
>> > > that is used for tracking exchanges while they are processed or after
>> > > processing is done, thereby being able to re-insert them into the
>> > > route. Not that I consider this being a shining example, but it shows
>> > > that message stores make sense outside their implicit use in EIP
>> > > processors as well.
>> > >
>> > > Looking forward to your opinions!
>> > >
>> > > regards
>> > > Christian
>> >
>>
>>
>>
>>



--
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

James Carman
I'm +1 for this feature and wouldn't mind helping out.  We've done
something very similar to this at work and would very much like for it
to be "baked in" already for us.



On Wed, Oct 24, 2012 at 9:50 AM, Claus Ibsen <[hidden email]> wrote:

> On Wed, Oct 24, 2012 at 8:46 AM, Bengt Rodehav <[hidden email]> wrote:
>> I think this is an excellent idea. We're using Camel as the basis for our
>> integration platform and a message store is a mandatory component. We need:
>>
>> - To store all message traffic in order to have an audit trail. Both
>> successful and failed exchanges.
>> - To provide a certain level of manual retry. That is to get the original
>> message from the store and feed it back in the originating route.
>> - Flexibility to specify what should be stored (e g what exchange- and
>> message properties) and also filter what exchanges should be stored.
>>
>> Currently we have implemented our own message store but it requires us to
>> adapt the routes to make this happen. A more generic way of handling this
>> would be terrific.
>>
>
> The Message History EIP is scheduled for overhaul/improvement in Camel 3.0.
> And in this light we will look at a message store API as well.
>
>
>
>> /Bengt
>>
>> 2012/10/24 Willem jiang <[hidden email]>
>>
>>> There is a lots of way to implement to Message store.
>>> Option1. If we treat is a first class of Camel, we could build up a set of
>>> API to let aggregator or tracer to use. As it is common usage that we need
>>> to store the exchange persistently, It make sense to provides such kind of
>>> API in Camel as a first class. +1 to provide this kind of API in Camel 3.x.
>>>
>>> Option2. We can also leverage the already exits camel components such as
>>> polling consumer or camel-JPA components to implement a such Message store.
>>> I think IPF shows us a good way to go. In this way we don't need to wait
>>> for the New API from Camel, it could be good solution for Camel 2.x.
>>>
>>> Maybe it is a good time for us to think about Camel 3.0 again.
>>>
>>> Any thoughts?
>>>
>>> Willem
>>>
>>> --
>>> Willem Jiang
>>>
>>> Red Hat, Inc.
>>> FuseSource is now part of Red Hat
>>> Web: http://www.fusesource.com | http://www.redhat.com
>>> Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/)
>>> (English)
>>>           http://jnn.javaeye.com (http://jnn.javaeye.com/) (Chinese)
>>> Twitter: willemjiang
>>> Weibo: willemjiang
>>>
>>>
>>>
>>>
>>>
>>> On Wednesday, October 24, 2012 at 9:43 AM, Hadrian Zbarcea wrote:
>>>
>>> > Hi Christian,
>>> >
>>> > You are correct about the fact that a Message Store is missing (kinda).
>>> > There are a few components that support a message store implementation
>>> > of sorts, as you mentioned. You are also correct that it should be a
>>> > first class citizen.
>>> >
>>> > Practically, an endpoint that supports a polling consumer is a message
>>> > store. What is actually missing is random access via a polling consumer.
>>> > A message store would then work atop any endpoint that supports random
>>> > retrieval (polling). That would create the opportunity to cleanup a bit
>>> > the pollEnrich implementation as well.
>>> >
>>> > My $0.02,
>>> > Hadrian
>>> >
>>> >
>>> > On 10/23/2012 07:18 AM, Christian Ohr wrote:
>>> > > Hi everyone,
>>> > >
>>> > > in the past I came several times across situations that required the
>>> > > one or other kind of Message Store. I noticed that in Camel this
>>> > > doesn't seem to be a "first class citizen" in the sense of a primary
>>> > > architectural concept, which can be applied consistently whereever
>>> > > needed.
>>> > >
>>> > > In short, it might make sense to have a unified, generic, pluggable
>>> > > Message Store (probably more of an "Exchange store") in Camel that
>>> > > consolidates the different approaches and allows to similarly
>>> > > parameterize persistence to various EIP patterns, and can be used
>>> > > independently of EIP patterns as well. Implementations would handle
>>> > > the mapping to the underlying database or file system or NoSQL or
>>> > > whatever.
>>> > > It seems that Spring Integration provides something corresponding
>>> > > (
>>> http://static.springsource.org/spring-integration/reference/htmlsingle/#message-store
>>> ).
>>> > >
>>> > > Message Store implementations are already used by Camel in various
>>> > > places, although using different approaches :
>>> > > * Stream Caching (only file system)
>>> > > * AggregationRepository (used for splitters, multicasts etc., but the
>>> > > interface is not specific at all to aggregation use cases)
>>> > > * IdempotentRepository
>>> > >
>>> > > Message Store is requested for in other places:
>>> > > * Reliable stream resequencing (CAMEL-949)
>>> > > * Persistent Dead Letter Queue (CAMEL-4575)
>>> > >
>>> > > And there might be other areas (seda, bam) that might benefit as well.
>>> > > Maybe generalizing the AggregationRepository is a way to go forward.
>>> > >
>>> > > Side note: The OeHF IPF platform (which is built on top of Camel and
>>> > > partly extends it to the health care domain) has something called a
>>> > > "Flow Manager" (
>>> http://www.openehealth.org/display/ipf2/Flow+management)
>>> > > that is used for tracking exchanges while they are processed or after
>>> > > processing is done, thereby being able to re-insert them into the
>>> > > route. Not that I consider this being a shining example, but it shows
>>> > > that message stores make sense outside their implicit use in EIP
>>> > > processors as well.
>>> > >
>>> > > Looking forward to your opinions!
>>> > >
>>> > > regards
>>> > > Christian
>>> >
>>>
>>>
>>>
>>>
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: [hidden email]
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

Björn Bength
In reply to this post by Claus Ibsen-2
Yeah
We also need this kind of message store right now.
Our need is claim-check, history, polling consumer and with random access.

I'd thought we'd try with MongoDB, because it fits our current use
case, if no camel component fits the bill.
But it would be nice to be able to leverage an "official" solution .

/Björn



On Wed, Oct 24, 2012 at 3:50 PM, Claus Ibsen <[hidden email]> wrote:

> On Wed, Oct 24, 2012 at 8:46 AM, Bengt Rodehav <[hidden email]> wrote:
>> I think this is an excellent idea. We're using Camel as the basis for our
>> integration platform and a message store is a mandatory component. We need:
>>
>> - To store all message traffic in order to have an audit trail. Both
>> successful and failed exchanges.
>> - To provide a certain level of manual retry. That is to get the original
>> message from the store and feed it back in the originating route.
>> - Flexibility to specify what should be stored (e g what exchange- and
>> message properties) and also filter what exchanges should be stored.
>>
>> Currently we have implemented our own message store but it requires us to
>> adapt the routes to make this happen. A more generic way of handling this
>> would be terrific.
>>
>
> The Message History EIP is scheduled for overhaul/improvement in Camel 3.0.
> And in this light we will look at a message store API as well.
>
>
>
>> /Bengt
>>
>> 2012/10/24 Willem jiang <[hidden email]>
>>
>>> There is a lots of way to implement to Message store.
>>> Option1. If we treat is a first class of Camel, we could build up a set of
>>> API to let aggregator or tracer to use. As it is common usage that we need
>>> to store the exchange persistently, It make sense to provides such kind of
>>> API in Camel as a first class. +1 to provide this kind of API in Camel 3.x.
>>>
>>> Option2. We can also leverage the already exits camel components such as
>>> polling consumer or camel-JPA components to implement a such Message store.
>>> I think IPF shows us a good way to go. In this way we don't need to wait
>>> for the New API from Camel, it could be good solution for Camel 2.x.
>>>
>>> Maybe it is a good time for us to think about Camel 3.0 again.
>>>
>>> Any thoughts?
>>>
>>> Willem
>>>
>>> --
>>> Willem Jiang
>>>
>>> Red Hat, Inc.
>>> FuseSource is now part of Red Hat
>>> Web: http://www.fusesource.com | http://www.redhat.com
>>> Blog: http://willemjiang.blogspot.com (http://willemjiang.blogspot.com/)
>>> (English)
>>>           http://jnn.javaeye.com (http://jnn.javaeye.com/) (Chinese)
>>> Twitter: willemjiang
>>> Weibo: willemjiang
>>>
>>>
>>>
>>>
>>>
>>> On Wednesday, October 24, 2012 at 9:43 AM, Hadrian Zbarcea wrote:
>>>
>>> > Hi Christian,
>>> >
>>> > You are correct about the fact that a Message Store is missing (kinda).
>>> > There are a few components that support a message store implementation
>>> > of sorts, as you mentioned. You are also correct that it should be a
>>> > first class citizen.
>>> >
>>> > Practically, an endpoint that supports a polling consumer is a message
>>> > store. What is actually missing is random access via a polling consumer.
>>> > A message store would then work atop any endpoint that supports random
>>> > retrieval (polling). That would create the opportunity to cleanup a bit
>>> > the pollEnrich implementation as well.
>>> >
>>> > My $0.02,
>>> > Hadrian
>>> >
>>> >
>>> > On 10/23/2012 07:18 AM, Christian Ohr wrote:
>>> > > Hi everyone,
>>> > >
>>> > > in the past I came several times across situations that required the
>>> > > one or other kind of Message Store. I noticed that in Camel this
>>> > > doesn't seem to be a "first class citizen" in the sense of a primary
>>> > > architectural concept, which can be applied consistently whereever
>>> > > needed.
>>> > >
>>> > > In short, it might make sense to have a unified, generic, pluggable
>>> > > Message Store (probably more of an "Exchange store") in Camel that
>>> > > consolidates the different approaches and allows to similarly
>>> > > parameterize persistence to various EIP patterns, and can be used
>>> > > independently of EIP patterns as well. Implementations would handle
>>> > > the mapping to the underlying database or file system or NoSQL or
>>> > > whatever.
>>> > > It seems that Spring Integration provides something corresponding
>>> > > (
>>> http://static.springsource.org/spring-integration/reference/htmlsingle/#message-store
>>> ).
>>> > >
>>> > > Message Store implementations are already used by Camel in various
>>> > > places, although using different approaches :
>>> > > * Stream Caching (only file system)
>>> > > * AggregationRepository (used for splitters, multicasts etc., but the
>>> > > interface is not specific at all to aggregation use cases)
>>> > > * IdempotentRepository
>>> > >
>>> > > Message Store is requested for in other places:
>>> > > * Reliable stream resequencing (CAMEL-949)
>>> > > * Persistent Dead Letter Queue (CAMEL-4575)
>>> > >
>>> > > And there might be other areas (seda, bam) that might benefit as well.
>>> > > Maybe generalizing the AggregationRepository is a way to go forward.
>>> > >
>>> > > Side note: The OeHF IPF platform (which is built on top of Camel and
>>> > > partly extends it to the health care domain) has something called a
>>> > > "Flow Manager" (
>>> http://www.openehealth.org/display/ipf2/Flow+management)
>>> > > that is used for tracking exchanges while they are processed or after
>>> > > processing is done, thereby being able to re-insert them into the
>>> > > route. Not that I consider this being a shining example, but it shows
>>> > > that message stores make sense outside their implicit use in EIP
>>> > > processors as well.
>>> > >
>>> > > Looking forward to your opinions!
>>> > >
>>> > > regards
>>> > > Christian
>>> >
>>>
>>>
>>>
>>>
>
>
>
> --
> Claus Ibsen
> -----------------
> Red Hat, Inc.
> FuseSource is now part of Red Hat
> Email: [hidden email]
> Web: http://fusesource.com
> Twitter: davsclaus
> Blog: http://davsclaus.com
> Author of Camel in Action: http://www.manning.com/ibsen
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

Christian Ohr
In reply to this post by James Carman
Let me quickly summarize -

there appears to be quite some interest from the community (and
apparently a number of homegrown solutions to fill the gap) and some
plans around a Message History EIP (at the bottom of
http://camel.apache.org/camel-30-roadmap.html) for a yet unscheduled
future Camel release.

Personally, on this issue I would prefer quality (consistency,
sustainability) over time, but just moving this into the future may be
not sufficient either. If someone of the PMC plans to attend the
ApacheCon EU in November, maybe we can brainstorm a bit or exchange
further ideas...

regards
Christian
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

hadrian
There will be quite a few Camel committers at ACEU. Technical
discussions must be inclusive of the community and take place on this
dev@ list though.

Hadrian


On 10/25/2012 03:47 AM, Christian Ohr wrote:

> Let me quickly summarize -
>
> there appears to be quite some interest from the community (and
> apparently a number of homegrown solutions to fill the gap) and some
> plans around a Message History EIP (at the bottom of
> http://camel.apache.org/camel-30-roadmap.html) for a yet unscheduled
> future Camel release.
>
> Personally, on this issue I would prefer quality (consistency,
> sustainability) over time, but just moving this into the future may be
> not sufficient either. If someone of the PMC plans to attend the
> ApacheCon EU in November, maybe we can brainstorm a bit or exchange
> further ideas...
>
> regards
> Christian
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

Rob Davies
+1



On 25 Oct 2012, at 12:44, Hadrian Zbarcea <[hidden email]> wrote:

> There will be quite a few Camel committers at ACEU. Technical discussions must be inclusive of the community and take place on this dev@ list though.
>
> Hadrian
>
>
> On 10/25/2012 03:47 AM, Christian Ohr wrote:
>> Let me quickly summarize -
>>
>> there appears to be quite some interest from the community (and
>> apparently a number of homegrown solutions to fill the gap) and some
>> plans around a Message History EIP (at the bottom of
>> http://camel.apache.org/camel-30-roadmap.html) for a yet unscheduled
>> future Camel release.
>>
>> Personally, on this issue I would prefer quality (consistency,
>> sustainability) over time, but just moving this into the future may be
>> not sufficient either. If someone of the PMC plans to attend the
>> ApacheCon EU in November, maybe we can brainstorm a bit or exchange
>> further ideas...
>>
>> regards
>> Christian
>>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

Christian Ohr
I would never dare to subvert the Camel community - in fact that's why
I started this thread here.

Christian


2012/10/25 Rob Davies <[hidden email]>:

> +1
>
>
>
> On 25 Oct 2012, at 12:44, Hadrian Zbarcea <[hidden email]> wrote:
>
>> There will be quite a few Camel committers at ACEU. Technical discussions must be inclusive of the community and take place on this dev@ list though.
>>
>> Hadrian
>>
>>
>> On 10/25/2012 03:47 AM, Christian Ohr wrote:
>>> Let me quickly summarize -
>>>
>>> there appears to be quite some interest from the community (and
>>> apparently a number of homegrown solutions to fill the gap) and some
>>> plans around a Message History EIP (at the bottom of
>>> http://camel.apache.org/camel-30-roadmap.html) for a yet unscheduled
>>> future Camel release.
>>>
>>> Personally, on this issue I would prefer quality (consistency,
>>> sustainability) over time, but just moving this into the future may be
>>> not sufficient either. If someone of the PMC plans to attend the
>>> ApacheCon EU in November, maybe we can brainstorm a bit or exchange
>>> further ideas...
>>>
>>> regards
>>> Christian
>>>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [DISCUSS] Message Store

Christian Mueller
We can discuss some ideas at ApacheCon EU (I will be there). But we have to
make sure we share the content of our discussion afterwards here at this
list.

Looking forward to meeting you there...

Best,
Christian

Sent from a mobile device
Am 25.10.2012 15:55 schrieb "Christian Ohr" <[hidden email]>:

> I would never dare to subvert the Camel community - in fact that's why
> I started this thread here.
>
> Christian
>
>
> 2012/10/25 Rob Davies <[hidden email]>:
> > +1
> >
> >
> >
> > On 25 Oct 2012, at 12:44, Hadrian Zbarcea <[hidden email]> wrote:
> >
> >> There will be quite a few Camel committers at ACEU. Technical
> discussions must be inclusive of the community and take place on this dev@list though.
> >>
> >> Hadrian
> >>
> >>
> >> On 10/25/2012 03:47 AM, Christian Ohr wrote:
> >>> Let me quickly summarize -
> >>>
> >>> there appears to be quite some interest from the community (and
> >>> apparently a number of homegrown solutions to fill the gap) and some
> >>> plans around a Message History EIP (at the bottom of
> >>> http://camel.apache.org/camel-30-roadmap.html) for a yet unscheduled
> >>> future Camel release.
> >>>
> >>> Personally, on this issue I would prefer quality (consistency,
> >>> sustainability) over time, but just moving this into the future may be
> >>> not sufficient either. If someone of the PMC plans to attend the
> >>> ApacheCon EU in November, maybe we can brainstorm a bit or exchange
> >>> further ideas...
> >>>
> >>> regards
> >>> Christian
> >>>
>
Loading...