Newbie: How to implement the Message Channel pattern?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Newbie: How to implement the Message Channel pattern?

borgel
Hi, I am trying to figure out how to use Camel. Let's say I have three applications (A, B and C), running on Tomcat. Application C has employee information that I would like to use in application A and B. Can I use Camel to send messages between these applications? If so, how do I do that? If I configure Camel in all of these applications they will have different CamelContext. Can I set up one instance of Camel and use that in all applications?
I guess I am trying to implement the Message Channel pattern (http://activemq.apache.org/camel/message-channel.html). I just don't know how to do that with Camel.
Reply | Threaded
Open this post in threaded view
|

Re: Newbie: How to implement the Message Channel pattern?

jstrachan
On 6/27/07, borgel <[hidden email]> wrote:
> Hi, I am trying to figure out how to use Camel. Let's say I have three
> applications (A, B and C), running on Tomcat. Application C has employee
> information that I would like to use in application A and B. Can I use Camel
> to send messages between these applications?

Sure! :)

Are the 3 applications in the same JVM; or in separate ones? Also are
they deployed as a single unit all the time; or could they come and go
individually?


> If so, how do I do that? If I
> configure Camel in all of these applications they will have different
> CamelContext. Can I set up one instance of Camel and use that in all
> applications?
> I guess I am trying to implement the Message Channel pattern
> (http://activemq.apache.org/camel/message-channel.html). I just don't know
> how to do that with Camel.

The simplest way to deal with communication between applications or
services (which may individually be started/stopped, be inside
different class loaders or deployed across different JVMs) is often
using messaging. Then the applications are completely decoupled across
time, location and JVM. e.g. using ActiveMQ queues for example.
Another option could be to use disk (file based messaging).

If you deploy the 3 WARs together, you could put Camel in the system
classpath & each web app could share the same CamelContext to exchange
messages via in-memory SEDA queues. Or if you really wanted to use a
local CamelContext in each web app, but still exchange messages using
in-memory SEDA we could introduce some kinda static version of the in
memory queue component; so that each web app could communicate using
in-memory SEDA queues yet use their own CamelContexts. Though you'd
have to be careful to ensure that Camel is on the system class loader
(or a shared classloader) across the web apps.

--
James
-------
http://macstrac.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: Newbie: How to implement the Message Channel pattern?

borgel
James.Strachan wrote
On 6/27/07, borgel <borge.lotre@gmail.com> wrote:
>> (...) Can I use Camel to send messages between these applications?

>Sure! :)

Great! :)

>Are the 3 applications in the same JVM; or in separate ones? Also are
>they deployed as a single unit all the time; or could they come and go
>individually?

Same JVM, but this might change. The applications are deployed separatly and can come and go individually.


>The simplest way to deal with communication between applications or
>services (which may individually be started/stopped, be inside
>different class loaders or deployed across different JVMs) is often
>using messaging. Then the applications are completely decoupled across
>time, location and JVM. e.g. using ActiveMQ queues for example.
>Another option could be to use disk (file based messaging).

This is what I tried, using direct connection. I sat up a Consumer in one app (app C with employee information) and a Producer in another application. From the producer i tried to call the consumer, but here it stopped for me, because the Producer accesses a different context than the Consumer. The same goes for JMS. For instance:
CamelContext container = new DefaultCamelContext();
(...)
JmsEndpoint endpoint = (JmsEndpoint) container.getEndpoint("jms:employee");    

The Consumer which is listening on this endpoint never receives anything becuase it's a different context. What is it I am missing?

Should each application (war) include Camel or should it be deployed to Tomcat in a different way?

>If you deploy the 3 WARs together, (...)

I can, but I would prefer not to. I must be able to start and stop each application independently from the others.

>James
>-------
>http://macstrac.blogspot.com/

--Børge.
Reply | Threaded
Open this post in threaded view
|

Re: Newbie: How to implement the Message Channel pattern?

jstrachan
On 6/27/07, borgel <[hidden email]> wrote:

>
>
> James.Strachan wrote:
> >
> > On 6/27/07, borgel <[hidden email]> wrote:
> >>> (...) Can I use Camel to send messages between these applications?
> >
> >>Sure! :)
> >
> > Great! :)
> >
> >>Are the 3 applications in the same JVM; or in separate ones? Also are
> >>they deployed as a single unit all the time; or could they come and go
> >>individually?
> >
> > Same JVM, but this might change. The applications are deployed separatly
> > and can come and go individually.
> >
> >
> >>The simplest way to deal with communication between applications or
> >>services (which may individually be started/stopped, be inside
> >>different class loaders or deployed across different JVMs) is often
> >>using messaging. Then the applications are completely decoupled across
> >>time, location and JVM. e.g. using ActiveMQ queues for example.
> >>Another option could be to use disk (file based messaging).
> >
> > This is what I tried, using direct connection. I sat up a Consumer in one
> > app (app C with employee information) and a Producer in another
> > application. From the producer i tried to call the consumer, but here it
> > stopped for me, because the Producer accesses a different context than the
> > Consumer. The same goes for JMS. For instance:
> > CamelContext container = new DefaultCamelContext();
> > (...)
> > JmsEndpoint endpoint = (JmsEndpoint)
> > container.getEndpoint("jms:employee");
> >
> > The Consumer which is listening on this endpoint never receives anything
> > becuase it's a different context. What is it I am missing?

Were you using ActiveMQ by any chance? If you were; ActiveMQ supports
embedded brokers. However the same kinda classloading issues occur
with ActiveMQ - to access the same broker via the VM protocol (to
avoid using sockets and marshalling stuff) - you must have ActiveMQ on
the system classpath; otherwise each web app ends up having its own
broker & the web apps can't communicate with each other.

The easiest way to ensure this works is to run a separate ActiveMQ
broker on the same machine (or in a WAR which is always deployed in
tomcat and never stopped) - then just use the ActiveMQ component in
Camel (which defaults to using tcp://locahost:61616 as the connection
URI)

using "activemq:employeeQueue" as the Camel URI will then allow
contexts to communicate with each other using the same queue.



> > Should each application (war) include Camel or should it be deployed to
> > Tomcat in a different way?
> >
> >>If you deploy the 3 WARs together, (...)
> >
> > I can, but I would prefer not to. I must be able to start and stop each
> > application independently from the others.

I get it, thanks.

JMS is definitely gonna be the most flexible (as you can stop and
start your entire Tomcat JVM whenever you like without loosing a
message); though I understand it adds a bit of complexity of learning
about MOM if you've never done much JMS before etc.

If you don't mind loosing all messages if you reboot tomcat (and are
happy to keep all your web apps in the same JVM) then you could also
look at a pure in-memory (in the system classpath) SEDA type approach.

Yet another approach; if you don't mind using JPA beans for your
messages, is to use the JPA component
http://activemq.apache.org/camel/jpa.html

Its not as efficient (in fact in the large scale it can be kinda slow)
and the load balancing sucks (basically 1 thread tends to grab most of
the messages on the consume side) - but for simple lowish volume
scenarios its worth considering.

The big-wins of using MOM is that its an awesome high performance load
balancer across threads and processes (queues) and also supports
efficient pulbish and subscribe (via topics); but for simpler
requirements, you can still use a database & if you don't mind loosing
messages on tomcat restart, there's the in memory SEDA alternative
--
James
-------
http://macstrac.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: Newbie: How to implement the Message Channel pattern?

borgel
James.Strachan wrote
On 6/27/07, borgel <borge.lotre@gmail.com> wrote:
>
>
> James.Strachan wrote:
> >
> > On 6/27/07, borgel <borge.lotre@gmail.com> wrote:
> >>> (...) Can I use Camel to send messages between these applications?
> >
> >>Sure! :)
> >
> > Great! :)
> >
> >>Are the 3 applications in the same JVM; or in separate ones? Also are
> >>they deployed as a single unit all the time; or could they come and go
> >>individually?
> >
> > Same JVM, but this might change. The applications are deployed separatly
> > and can come and go individually.
> >
> >
> >>The simplest way to deal with communication between applications or
> >>services (which may individually be started/stopped, be inside
> >>different class loaders or deployed across different JVMs) is often
> >>using messaging. Then the applications are completely decoupled across
> >>time, location and JVM. e.g. using ActiveMQ queues for example.
> >>Another option could be to use disk (file based messaging).
> >
> > This is what I tried, using direct connection. I sat up a Consumer in one
> > app (app C with employee information) and a Producer in another
> > application. From the producer i tried to call the consumer, but here it
> > stopped for me, because the Producer accesses a different context than the
> > Consumer. The same goes for JMS. For instance:
> > CamelContext container = new DefaultCamelContext();
> > (...)
> > JmsEndpoint endpoint = (JmsEndpoint)
> > container.getEndpoint("jms:employee");
> >
> > The Consumer which is listening on this endpoint never receives anything
> > becuase it's a different context. What is it I am missing?

Were you using ActiveMQ by any chance? If you were; ActiveMQ supports
embedded brokers. However the same kinda classloading issues occur
with ActiveMQ - to access the same broker via the VM protocol (to
avoid using sockets and marshalling stuff) - you must have ActiveMQ on
the system classpath; otherwise each web app ends up having its own
broker & the web apps can't communicate with each other.

The easiest way to ensure this works is to run a separate ActiveMQ
broker on the same machine (or in a WAR which is always deployed in
tomcat and never stopped) - then just use the ActiveMQ component in
Camel (which defaults to using tcp://locahost:61616 as the connection
URI)

using "activemq:employeeQueue" as the Camel URI will then allow
contexts to communicate with each other using the same queue.



> > Should each application (war) include Camel or should it be deployed to
> > Tomcat in a different way?
> >
> >>If you deploy the 3 WARs together, (...)
> >
> > I can, but I would prefer not to. I must be able to start and stop each
> > application independently from the others.

I get it, thanks.

JMS is definitely gonna be the most flexible (as you can stop and
start your entire Tomcat JVM whenever you like without loosing a
message); though I understand it adds a bit of complexity of learning
about MOM if you've never done much JMS before etc.

If you don't mind loosing all messages if you reboot tomcat (and are
happy to keep all your web apps in the same JVM) then you could also
look at a pure in-memory (in the system classpath) SEDA type approach.

Yet another approach; if you don't mind using JPA beans for your
messages, is to use the JPA component
http://activemq.apache.org/camel/jpa.html

Its not as efficient (in fact in the large scale it can be kinda slow)
and the load balancing sucks (basically 1 thread tends to grab most of
the messages on the consume side) - but for simple lowish volume
scenarios its worth considering.

The big-wins of using MOM is that its an awesome high performance load
balancer across threads and processes (queues) and also supports
efficient pulbish and subscribe (via topics); but for simpler
requirements, you can still use a database & if you don't mind loosing
messages on tomcat restart, there's the in memory SEDA alternative
--
James
-------
http://macstrac.blogspot.com/
Thank you.
I'll look into the alternatives. Sounds like JMS or SEDA are the most relavant alternatives for us then.
It would be good with some more examples when version 1.0 is released...  
Reply | Threaded
Open this post in threaded view
|

Re: Newbie: How to implement the Message Channel pattern?

jstrachan
On 6/28/07, borgel <[hidden email]> wrote:

>
>
> James.Strachan wrote:
> >
> > On 6/27/07, borgel <[hidden email]> wrote:
> >>
> >>
> >> James.Strachan wrote:
> >> >
> >> > On 6/27/07, borgel <[hidden email]> wrote:
> >> >>> (...) Can I use Camel to send messages between these applications?
> >> >
> >> >>Sure! :)
> >> >
> >> > Great! :)
> >> >
> >> >>Are the 3 applications in the same JVM; or in separate ones? Also are
> >> >>they deployed as a single unit all the time; or could they come and go
> >> >>individually?
> >> >
> >> > Same JVM, but this might change. The applications are deployed
> >> separatly
> >> > and can come and go individually.
> >> >
> >> >
> >> >>The simplest way to deal with communication between applications or
> >> >>services (which may individually be started/stopped, be inside
> >> >>different class loaders or deployed across different JVMs) is often
> >> >>using messaging. Then the applications are completely decoupled across
> >> >>time, location and JVM. e.g. using ActiveMQ queues for example.
> >> >>Another option could be to use disk (file based messaging).
> >> >
> >> > This is what I tried, using direct connection. I sat up a Consumer in
> >> one
> >> > app (app C with employee information) and a Producer in another
> >> > application. From the producer i tried to call the consumer, but here
> >> it
> >> > stopped for me, because the Producer accesses a different context than
> >> the
> >> > Consumer. The same goes for JMS. For instance:
> >> > CamelContext container = new DefaultCamelContext();
> >> > (...)
> >> > JmsEndpoint endpoint = (JmsEndpoint)
> >> > container.getEndpoint("jms:employee");
> >> >
> >> > The Consumer which is listening on this endpoint never receives
> >> anything
> >> > becuase it's a different context. What is it I am missing?
> >
> > Were you using ActiveMQ by any chance? If you were; ActiveMQ supports
> > embedded brokers. However the same kinda classloading issues occur
> > with ActiveMQ - to access the same broker via the VM protocol (to
> > avoid using sockets and marshalling stuff) - you must have ActiveMQ on
> > the system classpath; otherwise each web app ends up having its own
> > broker & the web apps can't communicate with each other.
> >
> > The easiest way to ensure this works is to run a separate ActiveMQ
> > broker on the same machine (or in a WAR which is always deployed in
> > tomcat and never stopped) - then just use the ActiveMQ component in
> > Camel (which defaults to using tcp://locahost:61616 as the connection
> > URI)
> >
> > using "activemq:employeeQueue" as the Camel URI will then allow
> > contexts to communicate with each other using the same queue.
> >
> >
> >
> >> > Should each application (war) include Camel or should it be deployed to
> >> > Tomcat in a different way?
> >> >
> >> >>If you deploy the 3 WARs together, (...)
> >> >
> >> > I can, but I would prefer not to. I must be able to start and stop each
> >> > application independently from the others.
> >
> > I get it, thanks.
> >
> > JMS is definitely gonna be the most flexible (as you can stop and
> > start your entire Tomcat JVM whenever you like without loosing a
> > message); though I understand it adds a bit of complexity of learning
> > about MOM if you've never done much JMS before etc.
> >
> > If you don't mind loosing all messages if you reboot tomcat (and are
> > happy to keep all your web apps in the same JVM) then you could also
> > look at a pure in-memory (in the system classpath) SEDA type approach.
> >
> > Yet another approach; if you don't mind using JPA beans for your
> > messages, is to use the JPA component
> > http://activemq.apache.org/camel/jpa.html
> >
> > Its not as efficient (in fact in the large scale it can be kinda slow)
> > and the load balancing sucks (basically 1 thread tends to grab most of
> > the messages on the consume side) - but for simple lowish volume
> > scenarios its worth considering.
> >
> > The big-wins of using MOM is that its an awesome high performance load
> > balancer across threads and processes (queues) and also supports
> > efficient pulbish and subscribe (via topics); but for simpler
> > requirements, you can still use a database & if you don't mind loosing
> > messages on tomcat restart, there's the in memory SEDA alternative
> > --
> > James
> > -------
> > http://macstrac.blogspot.com/
> >
> >
>
> Thank you.
> I'll look into the alternatives. Sounds like JMS or SEDA are the most
> relavant alternatives for us then.
> It would be good with some more examples when version 1.0 is released...  :)

Definitely :)

I'll try write up a little page on the wiki on communicating between
web apps as it seems a pretty common requirement

--
James
-------
http://macstrac.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: Newbie: How to implement the Message Channel pattern?

jstrachan
BTW there's now a vm:// component for communicating within the same
JVM, assuming that camel-core.jar is on the system/boot classpath...

http://cwiki.apache.org/CAMEL/vm.html

On 6/28/07, James Strachan <[hidden email]> wrote:

> On 6/28/07, borgel <[hidden email]> wrote:
> >
> >
> > James.Strachan wrote:
> > >
> > > On 6/27/07, borgel <[hidden email]> wrote:
> > >>
> > >>
> > >> James.Strachan wrote:
> > >> >
> > >> > On 6/27/07, borgel <[hidden email]> wrote:
> > >> >>> (...) Can I use Camel to send messages between these applications?
> > >> >
> > >> >>Sure! :)
> > >> >
> > >> > Great! :)
> > >> >
> > >> >>Are the 3 applications in the same JVM; or in separate ones? Also are
> > >> >>they deployed as a single unit all the time; or could they come and go
> > >> >>individually?
> > >> >
> > >> > Same JVM, but this might change. The applications are deployed
> > >> separatly
> > >> > and can come and go individually.
> > >> >
> > >> >
> > >> >>The simplest way to deal with communication between applications or
> > >> >>services (which may individually be started/stopped, be inside
> > >> >>different class loaders or deployed across different JVMs) is often
> > >> >>using messaging. Then the applications are completely decoupled across
> > >> >>time, location and JVM. e.g. using ActiveMQ queues for example.
> > >> >>Another option could be to use disk (file based messaging).
> > >> >
> > >> > This is what I tried, using direct connection. I sat up a Consumer in
> > >> one
> > >> > app (app C with employee information) and a Producer in another
> > >> > application. From the producer i tried to call the consumer, but here
> > >> it
> > >> > stopped for me, because the Producer accesses a different context than
> > >> the
> > >> > Consumer. The same goes for JMS. For instance:
> > >> > CamelContext container = new DefaultCamelContext();
> > >> > (...)
> > >> > JmsEndpoint endpoint = (JmsEndpoint)
> > >> > container.getEndpoint("jms:employee");
> > >> >
> > >> > The Consumer which is listening on this endpoint never receives
> > >> anything
> > >> > becuase it's a different context. What is it I am missing?
> > >
> > > Were you using ActiveMQ by any chance? If you were; ActiveMQ supports
> > > embedded brokers. However the same kinda classloading issues occur
> > > with ActiveMQ - to access the same broker via the VM protocol (to
> > > avoid using sockets and marshalling stuff) - you must have ActiveMQ on
> > > the system classpath; otherwise each web app ends up having its own
> > > broker & the web apps can't communicate with each other.
> > >
> > > The easiest way to ensure this works is to run a separate ActiveMQ
> > > broker on the same machine (or in a WAR which is always deployed in
> > > tomcat and never stopped) - then just use the ActiveMQ component in
> > > Camel (which defaults to using tcp://locahost:61616 as the connection
> > > URI)
> > >
> > > using "activemq:employeeQueue" as the Camel URI will then allow
> > > contexts to communicate with each other using the same queue.
> > >
> > >
> > >
> > >> > Should each application (war) include Camel or should it be deployed to
> > >> > Tomcat in a different way?
> > >> >
> > >> >>If you deploy the 3 WARs together, (...)
> > >> >
> > >> > I can, but I would prefer not to. I must be able to start and stop each
> > >> > application independently from the others.
> > >
> > > I get it, thanks.
> > >
> > > JMS is definitely gonna be the most flexible (as you can stop and
> > > start your entire Tomcat JVM whenever you like without loosing a
> > > message); though I understand it adds a bit of complexity of learning
> > > about MOM if you've never done much JMS before etc.
> > >
> > > If you don't mind loosing all messages if you reboot tomcat (and are
> > > happy to keep all your web apps in the same JVM) then you could also
> > > look at a pure in-memory (in the system classpath) SEDA type approach.
> > >
> > > Yet another approach; if you don't mind using JPA beans for your
> > > messages, is to use the JPA component
> > > http://activemq.apache.org/camel/jpa.html
> > >
> > > Its not as efficient (in fact in the large scale it can be kinda slow)
> > > and the load balancing sucks (basically 1 thread tends to grab most of
> > > the messages on the consume side) - but for simple lowish volume
> > > scenarios its worth considering.
> > >
> > > The big-wins of using MOM is that its an awesome high performance load
> > > balancer across threads and processes (queues) and also supports
> > > efficient pulbish and subscribe (via topics); but for simpler
> > > requirements, you can still use a database & if you don't mind loosing
> > > messages on tomcat restart, there's the in memory SEDA alternative
> > > --
> > > James
> > > -------
> > > http://macstrac.blogspot.com/
> > >
> > >
> >
> > Thank you.
> > I'll look into the alternatives. Sounds like JMS or SEDA are the most
> > relavant alternatives for us then.
> > It would be good with some more examples when version 1.0 is released...  :)
>
> Definitely :)
>
> I'll try write up a little page on the wiki on communicating between
> web apps as it seems a pretty common requirement
>
> --
> James
> -------
> http://macstrac.blogspot.com/
>


--
James
-------
http://macstrac.blogspot.com/