Integration Patterns inside the ActiveMQ Message Broker

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Integration Patterns inside the ActiveMQ Message Broker

jpcook01
Hello,

We are in the process of evaluating Camel, so far what I have read is really good. I just want to put something together now to demonstrate the concept to people.

I want to run my EIP Rules using java code from within ActiveMQ and have read the following page
http://activemq.apache.org/enterprise-integration-patterns.html

The bit I'm just having a little difficulty with is - I just want to create a simple example that picks up messages from our JMS and applies some xslt before writing the file somewhere.

I've create a class called MyRoute. I'm not sure what to put in it. Do I just put a main method and then the dsl within that like the getting started example? Then I have read the information on updating the Camel context so that bit should be fine.

Sorry if this is a pretty basic question but new to some of this stuff.

Appreciate your help
Thanks
Jonathan
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jstrachan
On 04/10/2007, jpcook <[hidden email]> wrote:
>
> Hello,
>
> We are in the process of evaluating Camel, so far what I have read is really
> good. I just want to put something together now to demonstrate the concept
> to people.

Great! :)

> I want to run my EIP Rules using java code from within ActiveMQ and have
> read the following page
> http://activemq.apache.org/enterprise-integration-patterns.html
>
> The bit I'm just having a little difficulty with is - I just want to create
> a simple example that picks up messages from our JMS and applies some xslt
> before writing the file somewhere.
>
> I've create a class called MyRoute. I'm not sure what to put in it. Do I
> just put a main method and then the dsl within that like the getting started
> example? Then I have read the information on updating the Camel context so
> that bit should be fine.
>
> Sorry if this is a pretty basic question but new to some of this stuff.

No need to be sorry; its a clear sign our documentation is lacking -
so its us who should be saying sorry :)

Basically if you put your MyRoute class (and any of its dependent
classes) in a jar and drop it into the activemq/lib directory it
should just work.

(Am assuming you're running the broker via the bin/activemq script right?)

We should make this a bit more obvious and easy to setup. e.g. maybe
we should make it easier to write Java rules by having a rules/src and
rules/classes directories (rather like web apps) so folks can just
write Java code for routing rules and have them compiled and added to
the classloader on startup?

--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

Nicky Sandhu
What would be really cool is to write only the routing code as a DSL in Java have the container wrap it up in a camel context (either existing or a new one)
E.g. the user would write something like

from("jms:myqueue").process(xslt("myxslt.xsl")).to("file:mydir");

and the container would wrap that in a RouteBuilder and add it to a new or existing CamelContext.

Ditto for xml config as well.

This would help in separation of roles of a Camel Extender (extends Camel) to a Camel User (writes routing rules)

James.Strachan wrote
We should make this a bit more obvious and easy to setup. e.g. maybe
we should make it easier to write Java rules by having a rules/src and
rules/classes directories (rather like web apps) so folks can just
write Java code for routing rules and have them compiled and added to
the classloader on startup?
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jstrachan
On 04/10/2007, Nicky Sandhu <[hidden email]> wrote:
> What would be really cool is to write only the routing code as a DSL in Java
> have the container wrap it up in a camel context (either existing or a new
> one)
> E.g. the user would write something like
>
> from("jms:myqueue").process(xslt("myxslt.xsl")).to("file:mydir");
>
> and the container would wrap that in a RouteBuilder and add it to a new or
> existing CamelContext.

The thing is, thats not actually Java without the class declaration
:).  To get all the nice IDE completion; we'd need that line inside a
method in a class for a Java IDE.

We could support just the routing as a DSL in groovy or ruby (I've
kinda started some experiments in that direction in the camel-ruby and
camel-groovy modules).

Or we could actually write a real DSL - a very early attempt here:
https://svn.apache.org/repos/asf/activemq/camel/ide/camel-eclipse/

which has little of the actual Antlr grammar there; but at least it
generates a nice text based editor in eclipse :)



> Ditto for xml config as well.

Yeah, the XML is much easier; we could just allow routes to be
specified in a separate file to the main spring.xml and included.


<!-- routes.xml -->
<routes xmlns="...">
  <route>
    <from uri="activemq:someQueue"/> <to uri="xmpp:someRoom"/>
  </route>
</routes>

<!-- spring.xml -->
...
<camelContext xmlns="...">
  <import uri="routes.xml"/>
</camelContext>


> This would help in separation of roles of a Camel Extender (extends Camel)
> to a Camel User (writes routing rules)

Yeah. In the pure XML world, I can see the separation between system
configuration (data sources, JMS providers etc) with the routing
rules; which may be semi-dynamic at runtime etc.

--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jpcook01
In reply to this post by jstrachan
Thanks for the help. I kind of figured it out reading through the documentation and there was another post in the forums which was useful. Just started off by using the simple example in the activemq.xml file under camel context to output message from queue Example.A to a file. I can't believe it is just 2 lines :)


>We should make this a bit more obvious and easy to setup. e.g. maybe
>we should make it easier to write Java rules by having a rules/src and
>rules/classes directories (rather like web apps) so folks can just
>write Java code for routing rules and have them compiled and added to
>the classloader on startup?

That would be quite cool at the moment I just have an ant script which builds the jar.

Quite like the other example you referred to of being able to import other xml files into the spring one. But think I want to use dsl for the time being.

One other quick question, do all your classes in the jar have to extend from RouteBuilder?

Thanks again
Jonathan
ps. I may have some more questions once I get past the basic examples.

James.Strachan wrote
On 04/10/2007, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Hello,
>
> We are in the process of evaluating Camel, so far what I have read is really
> good. I just want to put something together now to demonstrate the concept
> to people.

Great! :)

> I want to run my EIP Rules using java code from within ActiveMQ and have
> read the following page
> http://activemq.apache.org/enterprise-integration-patterns.html
>
> The bit I'm just having a little difficulty with is - I just want to create
> a simple example that picks up messages from our JMS and applies some xslt
> before writing the file somewhere.
>
> I've create a class called MyRoute. I'm not sure what to put in it. Do I
> just put a main method and then the dsl within that like the getting started
> example? Then I have read the information on updating the Camel context so
> that bit should be fine.
>
> Sorry if this is a pretty basic question but new to some of this stuff.

No need to be sorry; its a clear sign our documentation is lacking -
so its us who should be saying sorry :)

Basically if you put your MyRoute class (and any of its dependent
classes) in a jar and drop it into the activemq/lib directory it
should just work.

(Am assuming you're running the broker via the bin/activemq script right?)

We should make this a bit more obvious and easy to setup. e.g. maybe
we should make it easier to write Java rules by having a rules/src and
rules/classes directories (rather like web apps) so folks can just
write Java code for routing rules and have them compiled and added to
the classloader on startup?

--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

Nicky Sandhu
In reply to this post by jstrachan
James.Strachan wrote
The thing is, thats not actually Java without the class declaration
:).  To get all the nice IDE completion; we'd need that line inside a
method in a class for a Java IDE.
Agreed! That is one thing that is unique to Camel and a good thing.

James.Strachan wrote
We could support just the routing as a DSL in groovy or ruby (I've
kinda started some experiments in that direction in the camel-ruby and
camel-groovy modules).
I am a ruby user but I don't see why i'd want to give up on my autocompletion dsl in java or xml

James.Strachan wrote
Or we could actually write a real DSL - a very early attempt here:
https://svn.apache.org/repos/asf/activemq/camel/ide/camel-eclipse/

which has little of the actual Antlr grammar there; but at least it
generates a nice text based editor in eclipse :)
Fantastic!! Just the other day I was doodling javacc grammar to represent an integration language that would translate to Camel.

Another idea I was twirling around with was using GEF (eclipse project) to graphically edit a route...seems like you have some basics of eclipse plugins going on...
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jstrachan
In reply to this post by jpcook01
On 04/10/2007, jpcook <[hidden email]> wrote:
>
> Thanks for the help. I kind of figured it out reading through the
> documentation and there was another post in the forums which was useful.
> Just started off by using the simple example in the activemq.xml file under
> camel context to output message from queue Example.A to a file. I can't
> believe it is just 2 lines :)

Pretty neat eh :)


> >We should make this a bit more obvious and easy to setup. e.g. maybe
> >we should make it easier to write Java rules by having a rules/src and
> >rules/classes directories (rather like web apps) so folks can just
> >write Java code for routing rules and have them compiled and added to
> >the classloader on startup?
>
> That would be quite cool at the moment I just have an ant script which
> builds the jar.
>
> Quite like the other example you referred to of being able to import other
> xml files into the spring one. But think I want to use dsl for the time
> being.
>
> One other quick question, do all your classes in the jar have to extend from
> RouteBuilder?

The ones that define the Java DSL do yes; though you can have others
classes in the jar if you want too.


> ps. I may have some more questions once I get past the basic examples.

Keep 'em coming! :)

--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jstrachan
In reply to this post by Nicky Sandhu
On 04/10/2007, Nicky Sandhu <[hidden email]> wrote:

>
>
> James.Strachan wrote:
> >
> > The thing is, thats not actually Java without the class declaration
> > :).  To get all the nice IDE completion; we'd need that line inside a
> > method in a class for a Java IDE.
> >
> Agreed! That is one thing that is unique to Camel and a good thing.
>
>
> James.Strachan wrote:
> >
> > We could support just the routing as a DSL in groovy or ruby (I've
> > kinda started some experiments in that direction in the camel-ruby and
> > camel-groovy modules).
> >
> I am a ruby user but I don't see why i'd want to give up on my
> autocompletion dsl in java or xml

Agreed. Though IDE tooling is getting better for groovy / ruby; I've
seen method completion working in ruby and the groovy plugin for the
latest IDEA EAP is amazing!


> James.Strachan wrote:
> >
> > Or we could actually write a real DSL - a very early attempt here:
> > https://svn.apache.org/repos/asf/activemq/camel/ide/camel-eclipse/
> >
> > which has little of the actual Antlr grammar there; but at least it
> > generates a nice text based editor in eclipse :)
> >
>
> Fantastic!! Just the other day I was doodling javacc grammar to represent an
> integration language that would translate to Camel.

Oooh! Do you fancy hacking the Antlr script instead? Then we'd have a
great editor too! The eclipse xText plugin is amazing; it generates
the tree view, colour coded text and the little warning/errror dots in
the text, scrollbar and problem log - with text completion too.


> Another idea I was twirling around with was using GEF (eclipse project) to
> graphically edit a route...seems like you have some basics of eclipse
> plugins going on...

Definitely! :)

Ultimately I'd love to have a single editor that you could use the
textual DSL (with completion) as one tab or flip ot the graphical
layout while editing the same model (being EMF underneath etc).

There's already an EIP editor for eclipse....
http://code.google.com/p/cimero2editor/

though this uses its own EMF model; which is a bit too servicemix-3
specific right now but at least it does the EIP drawing stuff; we
could maybe bridge that to Camel.
--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jpcook01
In reply to this post by jstrachan
So this line doesn't actually compile:
from("jms:myqueue").process(xslt("myxslt.xsl")).to("file:mydir");
The method xslt is undefined for type MyRouter.


James.Strachan wrote
On 04/10/2007, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Thanks for the help. I kind of figured it out reading through the
> documentation and there was another post in the forums which was useful.
> Just started off by using the simple example in the activemq.xml file under
> camel context to output message from queue Example.A to a file. I can't
> believe it is just 2 lines :)

Pretty neat eh :)


> >We should make this a bit more obvious and easy to setup. e.g. maybe
> >we should make it easier to write Java rules by having a rules/src and
> >rules/classes directories (rather like web apps) so folks can just
> >write Java code for routing rules and have them compiled and added to
> >the classloader on startup?
>
> That would be quite cool at the moment I just have an ant script which
> builds the jar.
>
> Quite like the other example you referred to of being able to import other
> xml files into the spring one. But think I want to use dsl for the time
> being.
>
> One other quick question, do all your classes in the jar have to extend from
> RouteBuilder?

The ones that define the Java DSL do yes; though you can have others
classes in the jar if you want too.


> ps. I may have some more questions once I get past the basic examples.

Keep 'em coming! :)

--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jpcook01
Doh. ignore last post.
URL styleSheet = getClass().getResource("/xslt/rugbyFixtures.xslt"); from("jms:ss.service.solrenderservice").process(XsltBuilder.xslt(styleSheet)).to("file:rugbyFixtures");

What I would like to be able to do is. read in a message from my jms.  Then get one of the xml attributes from the message to lookup the location of where I want to send the output.

I think it is that recipientList pattern. Would that be possible?

Finally I'm not quite sure what I should be doing with exceptions. Just logging them, and which log will they appear in?

jpcook wrote
So this line doesn't actually compile:
from("jms:myqueue").process(xslt("myxslt.xsl")).to("file:mydir");
The method xslt is undefined for type MyRouter.


James.Strachan wrote
On 04/10/2007, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Thanks for the help. I kind of figured it out reading through the
> documentation and there was another post in the forums which was useful.
> Just started off by using the simple example in the activemq.xml file under
> camel context to output message from queue Example.A to a file. I can't
> believe it is just 2 lines :)

Pretty neat eh :)


> >We should make this a bit more obvious and easy to setup. e.g. maybe
> >we should make it easier to write Java rules by having a rules/src and
> >rules/classes directories (rather like web apps) so folks can just
> >write Java code for routing rules and have them compiled and added to
> >the classloader on startup?
>
> That would be quite cool at the moment I just have an ant script which
> builds the jar.
>
> Quite like the other example you referred to of being able to import other
> xml files into the spring one. But think I want to use dsl for the time
> being.
>
> One other quick question, do all your classes in the jar have to extend from
> RouteBuilder?

The ones that define the Java DSL do yes; though you can have others
classes in the jar if you want too.


> ps. I may have some more questions once I get past the basic examples.

Keep 'em coming! :)

--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

Hiram Chirino
Sounds like we need to(Expression ) method in the DSL.

On 10/5/07, jpcook <[hidden email]> wrote:

>
> Doh. ignore last post.
> URL styleSheet = getClass().getResource("/xslt/rugbyFixtures.xslt");
> from("jms:ss.service.solrenderservice").process(XsltBuilder.xslt(styleSheet)).to("file:rugbyFixtures");
>
> What I would like to be able to do is. read in a message from my jms.  Then
> get one of the xml attributes from the message to lookup the location of
> where I want to send the output.
>
> I think it is that recipientList pattern. Would that be possible?
>
> Finally I'm not quite sure what I should be doing with exceptions. Just
> logging them, and which log will they appear in?
>
>
> jpcook wrote:
> >
> > So this line doesn't actually compile:
> > from("jms:myqueue").process(xslt("myxslt.xsl")).to("file:mydir");
> > The method xslt is undefined for type MyRouter.
> >
> >
> >
> > James.Strachan wrote:
> >>
> >> On 04/10/2007, jpcook <[hidden email]> wrote:
> >>>
> >>> Thanks for the help. I kind of figured it out reading through the
> >>> documentation and there was another post in the forums which was useful.
> >>> Just started off by using the simple example in the activemq.xml file
> >>> under
> >>> camel context to output message from queue Example.A to a file. I can't
> >>> believe it is just 2 lines :)
> >>
> >> Pretty neat eh :)
> >>
> >>
> >>> >We should make this a bit more obvious and easy to setup. e.g. maybe
> >>> >we should make it easier to write Java rules by having a rules/src and
> >>> >rules/classes directories (rather like web apps) so folks can just
> >>> >write Java code for routing rules and have them compiled and added to
> >>> >the classloader on startup?
> >>>
> >>> That would be quite cool at the moment I just have an ant script which
> >>> builds the jar.
> >>>
> >>> Quite like the other example you referred to of being able to import
> >>> other
> >>> xml files into the spring one. But think I want to use dsl for the time
> >>> being.
> >>>
> >>> One other quick question, do all your classes in the jar have to extend
> >>> from
> >>> RouteBuilder?
> >>
> >> The ones that define the Java DSL do yes; though you can have others
> >> classes in the jar if you want too.
> >>
> >>
> >>> ps. I may have some more questions once I get past the basic examples.
> >>
> >> Keep 'em coming! :)
> >>
> >> --
> >> James
> >> -------
> >> http://macstrac.blogspot.com/
> >>
> >> Open Source SOA
> >> http://open.iona.com
> >>
> >>
> >
> >
>
> --
> View this message in context: http://www.nabble.com/Integration-Patterns-inside-the-ActiveMQ-Message-Broker-tf4569100s22882.html#a13060889
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>


--
Regards,
Hiram

Blog: http://hiramchirino.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jstrachan
In reply to this post by jpcook01
On 05/10/2007, jpcook <[hidden email]> wrote:
>
> Doh. ignore last post.
> URL styleSheet = getClass().getResource("/xslt/rugbyFixtures.xslt");
> from("jms:ss.service.solrenderservice").process(XsltBuilder.xslt(styleSheet)).to("file:rugbyFixtures");

Yeah - one of the downsides with the current DSL is often we use
static imports to make things extensible; so the use of xslt in that
one was probably a static import you saw. We should try and remove
those and add things like xpath, xslt and the like to the DSL so that
IDEs properly smart complete etc.


> What I would like to be able to do is. read in a message from my jms.  Then
> get one of the xml attributes from the message to lookup the location of
> where I want to send the output.
>
> I think it is that recipientList pattern. Would that be possible?

Yeah - you could either do a content based router using XPath
predicates, or use an XPath expression to determine the URL to send
to.

e.g.

from("activemq:myQueue").recipientList(xpath("'activemq:my.' + /foo/bar"));

i.e. using an XPath expression to determine the URI to send the message to.


> Finally I'm not quite sure what I should be doing with exceptions. Just
> logging them, and which log will they appear in?

There's an error handler plugin you can use to customize what to do
with exceptions and errors...

http://activemq.apache.org/camel/error-handler.html

so for example you can use a dead letter channel to retry on
exceptions until some number of times and then log it to some queue or
log.

--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jstrachan
In reply to this post by Hiram Chirino
On 05/10/2007, Hiram Chirino <[hidden email]> wrote:
> Sounds like we need to(Expression ) method in the DSL.

Thats kinda what recipientList(Expression) is for :)
http://activemq.apache.org/camel/recipient-list.html
--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com
Reply | Threaded
Open this post in threaded view
|

Re: Integration Patterns inside the ActiveMQ Message Broker

jpcook01
In reply to this post by jstrachan
Hi,

I'm trying to do a simple xslt transformation. Here is my simple main method. Sorry if it is difficult to read.

public static void main(String[] args) throws Exception {
                // TODO Auto-generated method stub
                CamelContext context = new DefaultCamelContext();
                ConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost?broker.persistent=false");
                context.addComponent("test-jms", JmsComponent.jmsComponentAutoAcknowledge(connectionFactory));
                                               
                 context.addRoutes(new RouteBuilder() {

                    public void configure() throws Exception {
                   
                        from("test-jms:queue:test.queue").process(new Processor() {

                            public void process(Exchange e) throws Exception {
                           
                            System.out.println("Received exchange: " + e.getIn());
                           
                    File styleSheet = new File("rugby_fixtures.xslt");
                            XsltBuilder xsltBuilder = XsltBuilder.xslt(styleSheet);
                            XsltBuilder xsltBuilder1 = xsltBuilder.outputString();
                           
                           
                            ProcessorType processorType = from("jms:ss.input.rugby_union").process(XsltBuilder.xslt(styleSheet));
                           
                           
                            from("jms:ss.input.rugby_union").process(XsltBuilder.xslt(styleSheet)).to("file://rugbyFixutes/fixtures.inc");
                           
                                    System.out.println("Received exchange: " + e.getIn());
                            }
                        });
                    }
                });
                 
                 CamelTemplate template = new CamelTemplate(context);
                 context.start();
                 
                 File file = new File("rugbyFixturesCollated.xml");
                 
                 XPP3Reader reader = new XPP3Reader();
                 Document rugbyFixtures = reader.read(file);
                 String xmlMessage = rugbyFixtures.asXML();
                 
                 template.sendBody("test-jms:queue:test.queue", xmlMessage );
             
                 Thread.sleep(1000);
                 context.stop();
        }

So I know the xslt works as I've tested it in a browser. The println line outputs the message as well and I can see the complete xml.

However what I really wanted to do was to be able to see the result of the transformation which is me playing around with the outputString method but that just sets the output. Is there a way I can look the output for debuggins purposes.

Also this doesn't appear to output a file. I'd expect a directory and rugbyFixtures/fixtures.inc to be created but one isn't.

Apologies if this is a bit basic.

Jonathan

James.Strachan wrote
On 05/10/2007, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Doh. ignore last post.
> URL styleSheet = getClass().getResource("/xslt/rugbyFixtures.xslt");
> from("jms:ss.service.solrenderservice").process(XsltBuilder.xslt(styleSheet)).to("file:rugbyFixtures");

Yeah - one of the downsides with the current DSL is often we use
static imports to make things extensible; so the use of xslt in that
one was probably a static import you saw. We should try and remove
those and add things like xpath, xslt and the like to the DSL so that
IDEs properly smart complete etc.


> What I would like to be able to do is. read in a message from my jms.  Then
> get one of the xml attributes from the message to lookup the location of
> where I want to send the output.
>
> I think it is that recipientList pattern. Would that be possible?

Yeah - you could either do a content based router using XPath
predicates, or use an XPath expression to determine the URL to send
to.

e.g.

from("activemq:myQueue").recipientList(xpath("'activemq:my.' + /foo/bar"));

i.e. using an XPath expression to determine the URI to send the message to.


> Finally I'm not quite sure what I should be doing with exceptions. Just
> logging them, and which log will they appear in?

There's an error handler plugin you can use to customize what to do
with exceptions and errors...

http://activemq.apache.org/camel/error-handler.html

so for example you can use a dead letter channel to retry on
exceptions until some number of times and then log it to some queue or
log.

--
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com