Atom Component

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Atom Component

jpcook01
Hello,

I have a requirement to pull 24 atom feeds, process them in the same way via xslt and then write the results to a file location which is slightly different for each feed. This is fairly straight forward.

I was looking at the atom component as it looks almost perfect. But I wondered if there was a clever way I could maybe specify a list of urls to the component and then it could process them concurrently as I don't want to have to do this synchronously? A bit like when you use the splitter you can specify parallelProcessing()

I guess another alternative would be to have a route for each feed I need to pull but this seemed a bit overkill as they would essentially be all the same. Also I wanted to make the atom urls and the location that the result gets written to configurable but we are not using the spring dsl xml configuration. As an alternative I could make these parameters configurable via my own configuration but I also wondered if I could perhaps control these parameters via JMX or even better via the web console?

Any thoughts much appreciated. Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

Claus Ibsen-2
Hi

You can use a scheduler / timer to trigger a route at a certain
interval (quartz or timer)

And then use a processor / bean with a consumerTemplate to consume
from the atom feeds.
Then you can use dynamic URIs.

And if you want that to route in parallel you can use the JDK
concurrency API for that as well.

Sometimes the easiest stuff is to do that using regular java in a POJO.
Submit tasks to the JDK executor services and then afterwards route
the result to a file endpoint to store the file.
Or a "direct" endpoint so you can do additional routing.




On Fri, Sep 25, 2009 at 12:02 PM, jpcook <[hidden email]> wrote:

>
> Hello,
>
> I have a requirement to pull 24 atom feeds, process them in the same way via
> xslt and then write the results to a file location which is slightly
> different for each feed. This is fairly straight forward.
>
> I was looking at the atom component as it looks almost perfect. But I
> wondered if there was a clever way I could maybe specify a list of urls to
> the component and then it could process them concurrently as I don't want to
> have to do this synchronously? A bit like when you use the splitter you can
> specify parallelProcessing()
>
> I guess another alternative would be to have a route for each feed I need to
> pull but this seemed a bit overkill as they would essentially be all the
> same. Also I wanted to make the atom urls and the location that the result
> gets written to configurable but we are not using the spring dsl xml
> configuration. As an alternative I could make these parameters configurable
> via my own configuration but I also wondered if I could perhaps control
> these parameters via JMX or even better via the web console?
>
> Any thoughts much appreciated. Thanks.
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25609495.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
thanks

The atom endpoint already has consumer.delay which I think I would use.

Is there an example of the consumerTemplate?

Could I use the web-console to configure the dynamic uris or just a normal xml configuration file?

About the last point, that is how I have done it in the past but just thought I'd check if there was something built into camel now. :)


Claus Ibsen-2 wrote
Hi

You can use a scheduler / timer to trigger a route at a certain
interval (quartz or timer)

And then use a processor / bean with a consumerTemplate to consume
from the atom feeds.
Then you can use dynamic URIs.

And if you want that to route in parallel you can use the JDK
concurrency API for that as well.

Sometimes the easiest stuff is to do that using regular java in a POJO.
Submit tasks to the JDK executor services and then afterwards route
the result to a file endpoint to store the file.
Or a "direct" endpoint so you can do additional routing.




On Fri, Sep 25, 2009 at 12:02 PM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Hello,
>
> I have a requirement to pull 24 atom feeds, process them in the same way via
> xslt and then write the results to a file location which is slightly
> different for each feed. This is fairly straight forward.
>
> I was looking at the atom component as it looks almost perfect. But I
> wondered if there was a clever way I could maybe specify a list of urls to
> the component and then it could process them concurrently as I don't want to
> have to do this synchronously? A bit like when you use the splitter you can
> specify parallelProcessing()
>
> I guess another alternative would be to have a route for each feed I need to
> pull but this seemed a bit overkill as they would essentially be all the
> same. Also I wanted to make the atom urls and the location that the result
> gets written to configurable but we are not using the spring dsl xml
> configuration. As an alternative I could make these parameters configurable
> via my own configuration but I also wondered if I could perhaps control
> these parameters via JMX or even better via the web console?
>
> Any thoughts much appreciated. Thanks.
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25609495.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

Charles Moulliard
JP,

For the polling consumer : have a look on this page :
http://camel.apache.org/polling-consumer.html


Charles Moulliard
Senior Enterprise Architect
Apache Camel Committer

*****************************
blog : http://cmoulliard.blogspot.com



On Fri, Sep 25, 2009 at 1:02 PM, jpcook <[hidden email]> wrote:

>
> thanks
>
> The atom endpoint already has consumer.delay which I think I would use.
>
> Is there an example of the consumerTemplate?
>
> Could I use the web-console to configure the dynamic uris or just a normal
> xml configuration file?
>
> About the last point, that is how I have done it in the past but just
> thought I'd check if there was something built into camel now. :)
>
>
>
> Claus Ibsen-2 wrote:
>>
>> Hi
>>
>> You can use a scheduler / timer to trigger a route at a certain
>> interval (quartz or timer)
>>
>> And then use a processor / bean with a consumerTemplate to consume
>> from the atom feeds.
>> Then you can use dynamic URIs.
>>
>> And if you want that to route in parallel you can use the JDK
>> concurrency API for that as well.
>>
>> Sometimes the easiest stuff is to do that using regular java in a POJO.
>> Submit tasks to the JDK executor services and then afterwards route
>> the result to a file endpoint to store the file.
>> Or a "direct" endpoint so you can do additional routing.
>>
>>
>>
>>
>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook <[hidden email]>
>> wrote:
>>>
>>> Hello,
>>>
>>> I have a requirement to pull 24 atom feeds, process them in the same way
>>> via
>>> xslt and then write the results to a file location which is slightly
>>> different for each feed. This is fairly straight forward.
>>>
>>> I was looking at the atom component as it looks almost perfect. But I
>>> wondered if there was a clever way I could maybe specify a list of urls
>>> to
>>> the component and then it could process them concurrently as I don't want
>>> to
>>> have to do this synchronously? A bit like when you use the splitter you
>>> can
>>> specify parallelProcessing()
>>>
>>> I guess another alternative would be to have a route for each feed I need
>>> to
>>> pull but this seemed a bit overkill as they would essentially be all the
>>> same. Also I wanted to make the atom urls and the location that the
>>> result
>>> gets written to configurable but we are not using the spring dsl xml
>>> configuration. As an alternative I could make these parameters
>>> configurable
>>> via my own configuration but I also wondered if I could perhaps control
>>> these parameters via JMX or even better via the web console?
>>>
>>> Any thoughts much appreciated. Thanks.
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25610124.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
Apache Committer / Sr. Pr. Consultant at FuseSource.com
Email: [hidden email]
Twitter : @cmoulliard, @fusenews
Blog : http://cmoulliard.blogspot.com
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

Claus Ibsen-2
In reply to this post by jpcook01
On Fri, Sep 25, 2009 at 1:02 PM, jpcook <[hidden email]> wrote:
>
> thanks
>
> The atom endpoint already has consumer.delay which I think I would use.

Yeah but the atom endpoint will then be configured with a static endpoint URI
from("atom:staticUriHere?consumer.delay=5000")...

Where as if you use a processor/bean/ with a consumer template you can do

from("timer://foo?delay=5000").beanRef("myBean", "doSomething)

And then in your POJO you can poll the atom endpoint using a dynamic URI

private List<String> uris;

public void doSomething() {
   // loop the list of dynamic uris and get the content from it
  // and then consume from the endpoint using consumer template
  Exchange out = consumerTemplate.receive(uri, 1000);
}

And see that link Charles mentioned.


>
> Is there an example of the consumerTemplate?
>
> Could I use the web-console to configure the dynamic uris or just a normal
> xml configuration file?
>
> About the last point, that is how I have done it in the past but just
> thought I'd check if there was something built into camel now. :)
>
>
>
> Claus Ibsen-2 wrote:
>>
>> Hi
>>
>> You can use a scheduler / timer to trigger a route at a certain
>> interval (quartz or timer)
>>
>> And then use a processor / bean with a consumerTemplate to consume
>> from the atom feeds.
>> Then you can use dynamic URIs.
>>
>> And if you want that to route in parallel you can use the JDK
>> concurrency API for that as well.
>>
>> Sometimes the easiest stuff is to do that using regular java in a POJO.
>> Submit tasks to the JDK executor services and then afterwards route
>> the result to a file endpoint to store the file.
>> Or a "direct" endpoint so you can do additional routing.
>>
>>
>>
>>
>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook <[hidden email]>
>> wrote:
>>>
>>> Hello,
>>>
>>> I have a requirement to pull 24 atom feeds, process them in the same way
>>> via
>>> xslt and then write the results to a file location which is slightly
>>> different for each feed. This is fairly straight forward.
>>>
>>> I was looking at the atom component as it looks almost perfect. But I
>>> wondered if there was a clever way I could maybe specify a list of urls
>>> to
>>> the component and then it could process them concurrently as I don't want
>>> to
>>> have to do this synchronously? A bit like when you use the splitter you
>>> can
>>> specify parallelProcessing()
>>>
>>> I guess another alternative would be to have a route for each feed I need
>>> to
>>> pull but this seemed a bit overkill as they would essentially be all the
>>> same. Also I wanted to make the atom urls and the location that the
>>> result
>>> gets written to configurable but we are not using the spring dsl xml
>>> configuration. As an alternative I could make these parameters
>>> configurable
>>> via my own configuration but I also wondered if I could perhaps control
>>> these parameters via JMX or even better via the web console?
>>>
>>> Any thoughts much appreciated. Thanks.
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25610124.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
Thanks!

One last probably silly question. What you initialise consumerTemplate as or do you extend from a particular class for this to work?

We're not using Spring so can't do any injection as per another thread I saw.

And then does the url actually contain the atom part? Looking at the wiki page it would indicate it should do eg) Exchange exchange = consumerTemplate.receive("activemq:my.queue");


Claus Ibsen-2 wrote
On Fri, Sep 25, 2009 at 1:02 PM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> thanks
>
> The atom endpoint already has consumer.delay which I think I would use.

Yeah but the atom endpoint will then be configured with a static endpoint URI
from("atom:staticUriHere?consumer.delay=5000")...

Where as if you use a processor/bean/ with a consumer template you can do

from("timer://foo?delay=5000").beanRef("myBean", "doSomething)

And then in your POJO you can poll the atom endpoint using a dynamic URI

private List<String> uris;

public void doSomething() {
   // loop the list of dynamic uris and get the content from it
  // and then consume from the endpoint using consumer template
  Exchange out = consumerTemplate.receive(uri, 1000);
}

And see that link Charles mentioned.


>
> Is there an example of the consumerTemplate?
>
> Could I use the web-console to configure the dynamic uris or just a normal
> xml configuration file?
>
> About the last point, that is how I have done it in the past but just
> thought I'd check if there was something built into camel now. :)
>
>
>
> Claus Ibsen-2 wrote:
>>
>> Hi
>>
>> You can use a scheduler / timer to trigger a route at a certain
>> interval (quartz or timer)
>>
>> And then use a processor / bean with a consumerTemplate to consume
>> from the atom feeds.
>> Then you can use dynamic URIs.
>>
>> And if you want that to route in parallel you can use the JDK
>> concurrency API for that as well.
>>
>> Sometimes the easiest stuff is to do that using regular java in a POJO.
>> Submit tasks to the JDK executor services and then afterwards route
>> the result to a file endpoint to store the file.
>> Or a "direct" endpoint so you can do additional routing.
>>
>>
>>
>>
>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook <jonathan.cook@erars.plus.com>
>> wrote:
>>>
>>> Hello,
>>>
>>> I have a requirement to pull 24 atom feeds, process them in the same way
>>> via
>>> xslt and then write the results to a file location which is slightly
>>> different for each feed. This is fairly straight forward.
>>>
>>> I was looking at the atom component as it looks almost perfect. But I
>>> wondered if there was a clever way I could maybe specify a list of urls
>>> to
>>> the component and then it could process them concurrently as I don't want
>>> to
>>> have to do this synchronously? A bit like when you use the splitter you
>>> can
>>> specify parallelProcessing()
>>>
>>> I guess another alternative would be to have a route for each feed I need
>>> to
>>> pull but this seemed a bit overkill as they would essentially be all the
>>> same. Also I wanted to make the atom urls and the location that the
>>> result
>>> gets written to configurable but we are not using the spring dsl xml
>>> configuration. As an alternative I could make these parameters
>>> configurable
>>> via my own configuration but I also wondered if I could perhaps control
>>> these parameters via JMX or even better via the web console?
>>>
>>> Any thoughts much appreciated. Thanks.
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25610124.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

Claus Ibsen-2
On Mon, Sep 28, 2009 at 10:05 AM, jpcook <[hidden email]> wrote:
>
> Thanks!
>
> One last probably silly question. What you initialise consumerTemplate as or
> do you extend from a particular class for this to work?
>
> We're not using Spring so can't do any injection as per another thread I
> saw.
>

You can create a consumer template from the CamelContext
And then you can reuse it on subsequent invocations.

If you use a bean/pojo you can just have a CamelContext parameter in
the method signature and Camel will inject the context for you.

If you use a processor then you can get hold of the CamelContext from
the Exchange using getContext()

public void doSomething(CamelContext context) {
if (consumerTemplate == null) {
  consumerTemplate = context.createConsumerTemplate();
}


> And then does the url actually contain the atom part? Looking at the wiki
> page it would indicate it should do eg) Exchange exchange =
> consumerTemplate.receive("activemq:my.queue");
>

Yeah you need to use the complete URL
 consumerTemplate.receive("atom://and some more here");



>
>
> Claus Ibsen-2 wrote:
>>
>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook <[hidden email]>
>> wrote:
>>>
>>> thanks
>>>
>>> The atom endpoint already has consumer.delay which I think I would use.
>>
>> Yeah but the atom endpoint will then be configured with a static endpoint
>> URI
>> from("atom:staticUriHere?consumer.delay=5000")...
>>
>> Where as if you use a processor/bean/ with a consumer template you can do
>>
>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>
>> And then in your POJO you can poll the atom endpoint using a dynamic URI
>>
>> private List<String> uris;
>>
>> public void doSomething() {
>>    // loop the list of dynamic uris and get the content from it
>>   // and then consume from the endpoint using consumer template
>>   Exchange out = consumerTemplate.receive(uri, 1000);
>> }
>>
>> And see that link Charles mentioned.
>>
>>
>>>
>>> Is there an example of the consumerTemplate?
>>>
>>> Could I use the web-console to configure the dynamic uris or just a
>>> normal
>>> xml configuration file?
>>>
>>> About the last point, that is how I have done it in the past but just
>>> thought I'd check if there was something built into camel now. :)
>>>
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> Hi
>>>>
>>>> You can use a scheduler / timer to trigger a route at a certain
>>>> interval (quartz or timer)
>>>>
>>>> And then use a processor / bean with a consumerTemplate to consume
>>>> from the atom feeds.
>>>> Then you can use dynamic URIs.
>>>>
>>>> And if you want that to route in parallel you can use the JDK
>>>> concurrency API for that as well.
>>>>
>>>> Sometimes the easiest stuff is to do that using regular java in a POJO.
>>>> Submit tasks to the JDK executor services and then afterwards route
>>>> the result to a file endpoint to store the file.
>>>> Or a "direct" endpoint so you can do additional routing.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook <[hidden email]>
>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I have a requirement to pull 24 atom feeds, process them in the same
>>>>> way
>>>>> via
>>>>> xslt and then write the results to a file location which is slightly
>>>>> different for each feed. This is fairly straight forward.
>>>>>
>>>>> I was looking at the atom component as it looks almost perfect. But I
>>>>> wondered if there was a clever way I could maybe specify a list of urls
>>>>> to
>>>>> the component and then it could process them concurrently as I don't
>>>>> want
>>>>> to
>>>>> have to do this synchronously? A bit like when you use the splitter you
>>>>> can
>>>>> specify parallelProcessing()
>>>>>
>>>>> I guess another alternative would be to have a route for each feed I
>>>>> need
>>>>> to
>>>>> pull but this seemed a bit overkill as they would essentially be all
>>>>> the
>>>>> same. Also I wanted to make the atom urls and the location that the
>>>>> result
>>>>> gets written to configurable but we are not using the spring dsl xml
>>>>> configuration. As an alternative I could make these parameters
>>>>> configurable
>>>>> via my own configuration but I also wondered if I could perhaps control
>>>>> these parameters via JMX or even better via the web console?
>>>>>
>>>>> Any thoughts much appreciated. Thanks.
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25641681.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
Hi,

So just trying a very simple example like:
from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");

But it returns:
2009-09-30 12:05:04,650 INFO [main] afterAtom - Exchange[BodyType:null, Body:null]

No errors in the console so not sure what I'm doing wrong?

Claus Ibsen-2 wrote
On Mon, Sep 28, 2009 at 10:05 AM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Thanks!
>
> One last probably silly question. What you initialise consumerTemplate as or
> do you extend from a particular class for this to work?
>
> We're not using Spring so can't do any injection as per another thread I
> saw.
>

You can create a consumer template from the CamelContext
And then you can reuse it on subsequent invocations.

If you use a bean/pojo you can just have a CamelContext parameter in
the method signature and Camel will inject the context for you.

If you use a processor then you can get hold of the CamelContext from
the Exchange using getContext()

public void doSomething(CamelContext context) {
if (consumerTemplate == null) {
  consumerTemplate = context.createConsumerTemplate();
}


> And then does the url actually contain the atom part? Looking at the wiki
> page it would indicate it should do eg) Exchange exchange =
> consumerTemplate.receive("activemq:my.queue");
>

Yeah you need to use the complete URL
 consumerTemplate.receive("atom://and some more here");



>
>
> Claus Ibsen-2 wrote:
>>
>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook <jonathan.cook@erars.plus.com>
>> wrote:
>>>
>>> thanks
>>>
>>> The atom endpoint already has consumer.delay which I think I would use.
>>
>> Yeah but the atom endpoint will then be configured with a static endpoint
>> URI
>> from("atom:staticUriHere?consumer.delay=5000")...
>>
>> Where as if you use a processor/bean/ with a consumer template you can do
>>
>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>
>> And then in your POJO you can poll the atom endpoint using a dynamic URI
>>
>> private List<String> uris;
>>
>> public void doSomething() {
>>    // loop the list of dynamic uris and get the content from it
>>   // and then consume from the endpoint using consumer template
>>   Exchange out = consumerTemplate.receive(uri, 1000);
>> }
>>
>> And see that link Charles mentioned.
>>
>>
>>>
>>> Is there an example of the consumerTemplate?
>>>
>>> Could I use the web-console to configure the dynamic uris or just a
>>> normal
>>> xml configuration file?
>>>
>>> About the last point, that is how I have done it in the past but just
>>> thought I'd check if there was something built into camel now. :)
>>>
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> Hi
>>>>
>>>> You can use a scheduler / timer to trigger a route at a certain
>>>> interval (quartz or timer)
>>>>
>>>> And then use a processor / bean with a consumerTemplate to consume
>>>> from the atom feeds.
>>>> Then you can use dynamic URIs.
>>>>
>>>> And if you want that to route in parallel you can use the JDK
>>>> concurrency API for that as well.
>>>>
>>>> Sometimes the easiest stuff is to do that using regular java in a POJO.
>>>> Submit tasks to the JDK executor services and then afterwards route
>>>> the result to a file endpoint to store the file.
>>>> Or a "direct" endpoint so you can do additional routing.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook <jonathan.cook@erars.plus.com>
>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I have a requirement to pull 24 atom feeds, process them in the same
>>>>> way
>>>>> via
>>>>> xslt and then write the results to a file location which is slightly
>>>>> different for each feed. This is fairly straight forward.
>>>>>
>>>>> I was looking at the atom component as it looks almost perfect. But I
>>>>> wondered if there was a clever way I could maybe specify a list of urls
>>>>> to
>>>>> the component and then it could process them concurrently as I don't
>>>>> want
>>>>> to
>>>>> have to do this synchronously? A bit like when you use the splitter you
>>>>> can
>>>>> specify parallelProcessing()
>>>>>
>>>>> I guess another alternative would be to have a route for each feed I
>>>>> need
>>>>> to
>>>>> pull but this seemed a bit overkill as they would essentially be all
>>>>> the
>>>>> same. Also I wanted to make the atom urls and the location that the
>>>>> result
>>>>> gets written to configurable but we are not using the spring dsl xml
>>>>> configuration. As an alternative I could make these parameters
>>>>> configurable
>>>>> via my own configuration but I also wondered if I could perhaps control
>>>>> these parameters via JMX or even better via the web console?
>>>>>
>>>>> Any thoughts much appreciated. Thanks.
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25641681.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
Strange just using the http endpoint works fine.

jpcook wrote
Hi,

So just trying a very simple example like:
from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");

But it returns:
2009-09-30 12:05:04,650 INFO [main] afterAtom - Exchange[BodyType:null, Body:null]

No errors in the console so not sure what I'm doing wrong?

Claus Ibsen-2 wrote
On Mon, Sep 28, 2009 at 10:05 AM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Thanks!
>
> One last probably silly question. What you initialise consumerTemplate as or
> do you extend from a particular class for this to work?
>
> We're not using Spring so can't do any injection as per another thread I
> saw.
>

You can create a consumer template from the CamelContext
And then you can reuse it on subsequent invocations.

If you use a bean/pojo you can just have a CamelContext parameter in
the method signature and Camel will inject the context for you.

If you use a processor then you can get hold of the CamelContext from
the Exchange using getContext()

public void doSomething(CamelContext context) {
if (consumerTemplate == null) {
  consumerTemplate = context.createConsumerTemplate();
}


> And then does the url actually contain the atom part? Looking at the wiki
> page it would indicate it should do eg) Exchange exchange =
> consumerTemplate.receive("activemq:my.queue");
>

Yeah you need to use the complete URL
 consumerTemplate.receive("atom://and some more here");



>
>
> Claus Ibsen-2 wrote:
>>
>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook <jonathan.cook@erars.plus.com>
>> wrote:
>>>
>>> thanks
>>>
>>> The atom endpoint already has consumer.delay which I think I would use.
>>
>> Yeah but the atom endpoint will then be configured with a static endpoint
>> URI
>> from("atom:staticUriHere?consumer.delay=5000")...
>>
>> Where as if you use a processor/bean/ with a consumer template you can do
>>
>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>
>> And then in your POJO you can poll the atom endpoint using a dynamic URI
>>
>> private List<String> uris;
>>
>> public void doSomething() {
>>    // loop the list of dynamic uris and get the content from it
>>   // and then consume from the endpoint using consumer template
>>   Exchange out = consumerTemplate.receive(uri, 1000);
>> }
>>
>> And see that link Charles mentioned.
>>
>>
>>>
>>> Is there an example of the consumerTemplate?
>>>
>>> Could I use the web-console to configure the dynamic uris or just a
>>> normal
>>> xml configuration file?
>>>
>>> About the last point, that is how I have done it in the past but just
>>> thought I'd check if there was something built into camel now. :)
>>>
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> Hi
>>>>
>>>> You can use a scheduler / timer to trigger a route at a certain
>>>> interval (quartz or timer)
>>>>
>>>> And then use a processor / bean with a consumerTemplate to consume
>>>> from the atom feeds.
>>>> Then you can use dynamic URIs.
>>>>
>>>> And if you want that to route in parallel you can use the JDK
>>>> concurrency API for that as well.
>>>>
>>>> Sometimes the easiest stuff is to do that using regular java in a POJO.
>>>> Submit tasks to the JDK executor services and then afterwards route
>>>> the result to a file endpoint to store the file.
>>>> Or a "direct" endpoint so you can do additional routing.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook <jonathan.cook@erars.plus.com>
>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I have a requirement to pull 24 atom feeds, process them in the same
>>>>> way
>>>>> via
>>>>> xslt and then write the results to a file location which is slightly
>>>>> different for each feed. This is fairly straight forward.
>>>>>
>>>>> I was looking at the atom component as it looks almost perfect. But I
>>>>> wondered if there was a clever way I could maybe specify a list of urls
>>>>> to
>>>>> the component and then it could process them concurrently as I don't
>>>>> want
>>>>> to
>>>>> have to do this synchronously? A bit like when you use the splitter you
>>>>> can
>>>>> specify parallelProcessing()
>>>>>
>>>>> I guess another alternative would be to have a route for each feed I
>>>>> need
>>>>> to
>>>>> pull but this seemed a bit overkill as they would essentially be all
>>>>> the
>>>>> same. Also I wanted to make the atom urls and the location that the
>>>>> result
>>>>> gets written to configurable but we are not using the spring dsl xml
>>>>> configuration. As an alternative I could make these parameters
>>>>> configurable
>>>>> via my own configuration but I also wondered if I could perhaps control
>>>>> these parameters via JMX or even better via the web console?
>>>>>
>>>>> Any thoughts much appreciated. Thanks.
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25641681.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
Got further.

What version of abdera does this work with. I have downloaded the latest which is 0.3.0 and it doesn't seem to contain the getInstance method used here.

Abdera.getInstance().getParser();

if I use Abdera.getNewParser() it works ok.

Any ideas please?

jpcook wrote
Strange just using the http endpoint works fine.

jpcook wrote
Hi,

So just trying a very simple example like:
from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");

But it returns:
2009-09-30 12:05:04,650 INFO [main] afterAtom - Exchange[BodyType:null, Body:null]

No errors in the console so not sure what I'm doing wrong?

Claus Ibsen-2 wrote
On Mon, Sep 28, 2009 at 10:05 AM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Thanks!
>
> One last probably silly question. What you initialise consumerTemplate as or
> do you extend from a particular class for this to work?
>
> We're not using Spring so can't do any injection as per another thread I
> saw.
>

You can create a consumer template from the CamelContext
And then you can reuse it on subsequent invocations.

If you use a bean/pojo you can just have a CamelContext parameter in
the method signature and Camel will inject the context for you.

If you use a processor then you can get hold of the CamelContext from
the Exchange using getContext()

public void doSomething(CamelContext context) {
if (consumerTemplate == null) {
  consumerTemplate = context.createConsumerTemplate();
}


> And then does the url actually contain the atom part? Looking at the wiki
> page it would indicate it should do eg) Exchange exchange =
> consumerTemplate.receive("activemq:my.queue");
>

Yeah you need to use the complete URL
 consumerTemplate.receive("atom://and some more here");



>
>
> Claus Ibsen-2 wrote:
>>
>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook <jonathan.cook@erars.plus.com>
>> wrote:
>>>
>>> thanks
>>>
>>> The atom endpoint already has consumer.delay which I think I would use.
>>
>> Yeah but the atom endpoint will then be configured with a static endpoint
>> URI
>> from("atom:staticUriHere?consumer.delay=5000")...
>>
>> Where as if you use a processor/bean/ with a consumer template you can do
>>
>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>
>> And then in your POJO you can poll the atom endpoint using a dynamic URI
>>
>> private List<String> uris;
>>
>> public void doSomething() {
>>    // loop the list of dynamic uris and get the content from it
>>   // and then consume from the endpoint using consumer template
>>   Exchange out = consumerTemplate.receive(uri, 1000);
>> }
>>
>> And see that link Charles mentioned.
>>
>>
>>>
>>> Is there an example of the consumerTemplate?
>>>
>>> Could I use the web-console to configure the dynamic uris or just a
>>> normal
>>> xml configuration file?
>>>
>>> About the last point, that is how I have done it in the past but just
>>> thought I'd check if there was something built into camel now. :)
>>>
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> Hi
>>>>
>>>> You can use a scheduler / timer to trigger a route at a certain
>>>> interval (quartz or timer)
>>>>
>>>> And then use a processor / bean with a consumerTemplate to consume
>>>> from the atom feeds.
>>>> Then you can use dynamic URIs.
>>>>
>>>> And if you want that to route in parallel you can use the JDK
>>>> concurrency API for that as well.
>>>>
>>>> Sometimes the easiest stuff is to do that using regular java in a POJO.
>>>> Submit tasks to the JDK executor services and then afterwards route
>>>> the result to a file endpoint to store the file.
>>>> Or a "direct" endpoint so you can do additional routing.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook <jonathan.cook@erars.plus.com>
>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I have a requirement to pull 24 atom feeds, process them in the same
>>>>> way
>>>>> via
>>>>> xslt and then write the results to a file location which is slightly
>>>>> different for each feed. This is fairly straight forward.
>>>>>
>>>>> I was looking at the atom component as it looks almost perfect. But I
>>>>> wondered if there was a clever way I could maybe specify a list of urls
>>>>> to
>>>>> the component and then it could process them concurrently as I don't
>>>>> want
>>>>> to
>>>>> have to do this synchronously? A bit like when you use the splitter you
>>>>> can
>>>>> specify parallelProcessing()
>>>>>
>>>>> I guess another alternative would be to have a route for each feed I
>>>>> need
>>>>> to
>>>>> pull but this seemed a bit overkill as they would essentially be all
>>>>> the
>>>>> same. Also I wanted to make the atom urls and the location that the
>>>>> result
>>>>> gets written to configurable but we are not using the spring dsl xml
>>>>> configuration. As an alternative I could make these parameters
>>>>> configurable
>>>>> via my own configuration but I also wondered if I could perhaps control
>>>>> these parameters via JMX or even better via the web console?
>>>>>
>>>>> Any thoughts much appreciated. Thanks.
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25641681.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
Upgraded to the latest latest version 0.4.0 and it works. Might be useful to list the dependancies on the wiki? Camel just eats the exception as well.

jpcook wrote
Got further.

What version of abdera does this work with. I have downloaded the latest which is 0.3.0 and it doesn't seem to contain the getInstance method used here.

Abdera.getInstance().getParser();

if I use Abdera.getNewParser() it works ok.

Any ideas please?

jpcook wrote
Strange just using the http endpoint works fine.

jpcook wrote
Hi,

So just trying a very simple example like:
from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");

But it returns:
2009-09-30 12:05:04,650 INFO [main] afterAtom - Exchange[BodyType:null, Body:null]

No errors in the console so not sure what I'm doing wrong?

Claus Ibsen-2 wrote
On Mon, Sep 28, 2009 at 10:05 AM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Thanks!
>
> One last probably silly question. What you initialise consumerTemplate as or
> do you extend from a particular class for this to work?
>
> We're not using Spring so can't do any injection as per another thread I
> saw.
>

You can create a consumer template from the CamelContext
And then you can reuse it on subsequent invocations.

If you use a bean/pojo you can just have a CamelContext parameter in
the method signature and Camel will inject the context for you.

If you use a processor then you can get hold of the CamelContext from
the Exchange using getContext()

public void doSomething(CamelContext context) {
if (consumerTemplate == null) {
  consumerTemplate = context.createConsumerTemplate();
}


> And then does the url actually contain the atom part? Looking at the wiki
> page it would indicate it should do eg) Exchange exchange =
> consumerTemplate.receive("activemq:my.queue");
>

Yeah you need to use the complete URL
 consumerTemplate.receive("atom://and some more here");



>
>
> Claus Ibsen-2 wrote:
>>
>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook <jonathan.cook@erars.plus.com>
>> wrote:
>>>
>>> thanks
>>>
>>> The atom endpoint already has consumer.delay which I think I would use.
>>
>> Yeah but the atom endpoint will then be configured with a static endpoint
>> URI
>> from("atom:staticUriHere?consumer.delay=5000")...
>>
>> Where as if you use a processor/bean/ with a consumer template you can do
>>
>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>
>> And then in your POJO you can poll the atom endpoint using a dynamic URI
>>
>> private List<String> uris;
>>
>> public void doSomething() {
>>    // loop the list of dynamic uris and get the content from it
>>   // and then consume from the endpoint using consumer template
>>   Exchange out = consumerTemplate.receive(uri, 1000);
>> }
>>
>> And see that link Charles mentioned.
>>
>>
>>>
>>> Is there an example of the consumerTemplate?
>>>
>>> Could I use the web-console to configure the dynamic uris or just a
>>> normal
>>> xml configuration file?
>>>
>>> About the last point, that is how I have done it in the past but just
>>> thought I'd check if there was something built into camel now. :)
>>>
>>>
>>>
>>> Claus Ibsen-2 wrote:
>>>>
>>>> Hi
>>>>
>>>> You can use a scheduler / timer to trigger a route at a certain
>>>> interval (quartz or timer)
>>>>
>>>> And then use a processor / bean with a consumerTemplate to consume
>>>> from the atom feeds.
>>>> Then you can use dynamic URIs.
>>>>
>>>> And if you want that to route in parallel you can use the JDK
>>>> concurrency API for that as well.
>>>>
>>>> Sometimes the easiest stuff is to do that using regular java in a POJO.
>>>> Submit tasks to the JDK executor services and then afterwards route
>>>> the result to a file endpoint to store the file.
>>>> Or a "direct" endpoint so you can do additional routing.
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook <jonathan.cook@erars.plus.com>
>>>> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I have a requirement to pull 24 atom feeds, process them in the same
>>>>> way
>>>>> via
>>>>> xslt and then write the results to a file location which is slightly
>>>>> different for each feed. This is fairly straight forward.
>>>>>
>>>>> I was looking at the atom component as it looks almost perfect. But I
>>>>> wondered if there was a clever way I could maybe specify a list of urls
>>>>> to
>>>>> the component and then it could process them concurrently as I don't
>>>>> want
>>>>> to
>>>>> have to do this synchronously? A bit like when you use the splitter you
>>>>> can
>>>>> specify parallelProcessing()
>>>>>
>>>>> I guess another alternative would be to have a route for each feed I
>>>>> need
>>>>> to
>>>>> pull but this seemed a bit overkill as they would essentially be all
>>>>> the
>>>>> same. Also I wanted to make the atom urls and the location that the
>>>>> result
>>>>> gets written to configurable but we are not using the spring dsl xml
>>>>> configuration. As an alternative I could make these parameters
>>>>> configurable
>>>>> via my own configuration but I also wondered if I could perhaps control
>>>>> these parameters via JMX or even better via the web console?
>>>>>
>>>>> Any thoughts much appreciated. Thanks.
>>>>> --
>>>>> View this message in context:
>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25641681.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

Claus Ibsen-2
On Wed, Sep 30, 2009 at 3:57 PM, jpcook <[hidden email]> wrote:
>
> Upgraded to the latest latest version 0.4.0 and it works. Might be useful to
> list the dependancies on the wiki? Camel just eats the exception as well.

Most people use maven and have the dependencies for free.

If you are not then you have to check this out yourself.
For example see the pom.xml file for the the component which lists the
dependencies it uses.

Or see the maven reposts (which can be old as they are not updated very often)
http://camel.apache.org/maven/camel-atom/dependency-convergence.html
http://camel.apache.org/maven/index.html


>
>
> jpcook wrote:
>>
>> Got further.
>>
>> What version of abdera does this work with. I have downloaded the latest
>> which is 0.3.0 and it doesn't seem to contain the getInstance method used
>> here.
>>
>> Abdera.getInstance().getParser();
>>
>> if I use Abdera.getNewParser() it works ok.
>>
>> Any ideas please?
>>
>>
>> jpcook wrote:
>>>
>>> Strange just using the http endpoint works fine.
>>>
>>>
>>> jpcook wrote:
>>>>
>>>> Hi,
>>>>
>>>> So just trying a very simple example like:
>>>> from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");
>>>>
>>>> But it returns:
>>>> 2009-09-30 12:05:04,650 INFO [main] afterAtom - Exchange[BodyType:null,
>>>> Body:null]
>>>>
>>>> No errors in the console so not sure what I'm doing wrong?
>>>>
>>>>
>>>> Claus Ibsen-2 wrote:
>>>>>
>>>>> On Mon, Sep 28, 2009 at 10:05 AM, jpcook <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> One last probably silly question. What you initialise consumerTemplate
>>>>>> as or
>>>>>> do you extend from a particular class for this to work?
>>>>>>
>>>>>> We're not using Spring so can't do any injection as per another thread
>>>>>> I
>>>>>> saw.
>>>>>>
>>>>>
>>>>> You can create a consumer template from the CamelContext
>>>>> And then you can reuse it on subsequent invocations.
>>>>>
>>>>> If you use a bean/pojo you can just have a CamelContext parameter in
>>>>> the method signature and Camel will inject the context for you.
>>>>>
>>>>> If you use a processor then you can get hold of the CamelContext from
>>>>> the Exchange using getContext()
>>>>>
>>>>> public void doSomething(CamelContext context) {
>>>>> if (consumerTemplate == null) {
>>>>>   consumerTemplate = context.createConsumerTemplate();
>>>>> }
>>>>>
>>>>>
>>>>>> And then does the url actually contain the atom part? Looking at the
>>>>>> wiki
>>>>>> page it would indicate it should do eg) Exchange exchange =
>>>>>> consumerTemplate.receive("activemq:my.queue");
>>>>>>
>>>>>
>>>>> Yeah you need to use the complete URL
>>>>>  consumerTemplate.receive("atom://and some more here");
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Claus Ibsen-2 wrote:
>>>>>>>
>>>>>>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook
>>>>>>> <[hidden email]>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> thanks
>>>>>>>>
>>>>>>>> The atom endpoint already has consumer.delay which I think I would
>>>>>>>> use.
>>>>>>>
>>>>>>> Yeah but the atom endpoint will then be configured with a static
>>>>>>> endpoint
>>>>>>> URI
>>>>>>> from("atom:staticUriHere?consumer.delay=5000")...
>>>>>>>
>>>>>>> Where as if you use a processor/bean/ with a consumer template you
>>>>>>> can do
>>>>>>>
>>>>>>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>>>>>>
>>>>>>> And then in your POJO you can poll the atom endpoint using a dynamic
>>>>>>> URI
>>>>>>>
>>>>>>> private List<String> uris;
>>>>>>>
>>>>>>> public void doSomething() {
>>>>>>>    // loop the list of dynamic uris and get the content from it
>>>>>>>   // and then consume from the endpoint using consumer template
>>>>>>>   Exchange out = consumerTemplate.receive(uri, 1000);
>>>>>>> }
>>>>>>>
>>>>>>> And see that link Charles mentioned.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Is there an example of the consumerTemplate?
>>>>>>>>
>>>>>>>> Could I use the web-console to configure the dynamic uris or just a
>>>>>>>> normal
>>>>>>>> xml configuration file?
>>>>>>>>
>>>>>>>> About the last point, that is how I have done it in the past but
>>>>>>>> just
>>>>>>>> thought I'd check if there was something built into camel now. :)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> You can use a scheduler / timer to trigger a route at a certain
>>>>>>>>> interval (quartz or timer)
>>>>>>>>>
>>>>>>>>> And then use a processor / bean with a consumerTemplate to consume
>>>>>>>>> from the atom feeds.
>>>>>>>>> Then you can use dynamic URIs.
>>>>>>>>>
>>>>>>>>> And if you want that to route in parallel you can use the JDK
>>>>>>>>> concurrency API for that as well.
>>>>>>>>>
>>>>>>>>> Sometimes the easiest stuff is to do that using regular java in a
>>>>>>>>> POJO.
>>>>>>>>> Submit tasks to the JDK executor services and then afterwards route
>>>>>>>>> the result to a file endpoint to store the file.
>>>>>>>>> Or a "direct" endpoint so you can do additional routing.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook
>>>>>>>>> <[hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I have a requirement to pull 24 atom feeds, process them in the
>>>>>>>>>> same
>>>>>>>>>> way
>>>>>>>>>> via
>>>>>>>>>> xslt and then write the results to a file location which is
>>>>>>>>>> slightly
>>>>>>>>>> different for each feed. This is fairly straight forward.
>>>>>>>>>>
>>>>>>>>>> I was looking at the atom component as it looks almost perfect.
>>>>>>>>>> But I
>>>>>>>>>> wondered if there was a clever way I could maybe specify a list of
>>>>>>>>>> urls
>>>>>>>>>> to
>>>>>>>>>> the component and then it could process them concurrently as I
>>>>>>>>>> don't
>>>>>>>>>> want
>>>>>>>>>> to
>>>>>>>>>> have to do this synchronously? A bit like when you use the
>>>>>>>>>> splitter you
>>>>>>>>>> can
>>>>>>>>>> specify parallelProcessing()
>>>>>>>>>>
>>>>>>>>>> I guess another alternative would be to have a route for each feed
>>>>>>>>>> I
>>>>>>>>>> need
>>>>>>>>>> to
>>>>>>>>>> pull but this seemed a bit overkill as they would essentially be
>>>>>>>>>> all
>>>>>>>>>> the
>>>>>>>>>> same. Also I wanted to make the atom urls and the location that
>>>>>>>>>> the
>>>>>>>>>> result
>>>>>>>>>> gets written to configurable but we are not using the spring dsl
>>>>>>>>>> xml
>>>>>>>>>> configuration. As an alternative I could make these parameters
>>>>>>>>>> configurable
>>>>>>>>>> via my own configuration but I also wondered if I could perhaps
>>>>>>>>>> control
>>>>>>>>>> these parameters via JMX or even better via the web console?
>>>>>>>>>>
>>>>>>>>>> Any thoughts much appreciated. Thanks.
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25641681.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25681591.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
Thanks. Useful to know.
We aren't using maven unfortunately.

Claus Ibsen-2 wrote
On Wed, Sep 30, 2009 at 3:57 PM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Upgraded to the latest latest version 0.4.0 and it works. Might be useful to
> list the dependancies on the wiki? Camel just eats the exception as well.

Most people use maven and have the dependencies for free.

If you are not then you have to check this out yourself.
For example see the pom.xml file for the the component which lists the
dependencies it uses.

Or see the maven reposts (which can be old as they are not updated very often)
http://camel.apache.org/maven/camel-atom/dependency-convergence.html
http://camel.apache.org/maven/index.html


>
>
> jpcook wrote:
>>
>> Got further.
>>
>> What version of abdera does this work with. I have downloaded the latest
>> which is 0.3.0 and it doesn't seem to contain the getInstance method used
>> here.
>>
>> Abdera.getInstance().getParser();
>>
>> if I use Abdera.getNewParser() it works ok.
>>
>> Any ideas please?
>>
>>
>> jpcook wrote:
>>>
>>> Strange just using the http endpoint works fine.
>>>
>>>
>>> jpcook wrote:
>>>>
>>>> Hi,
>>>>
>>>> So just trying a very simple example like:
>>>> from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");
>>>>
>>>> But it returns:
>>>> 2009-09-30 12:05:04,650 INFO [main] afterAtom - Exchange[BodyType:null,
>>>> Body:null]
>>>>
>>>> No errors in the console so not sure what I'm doing wrong?
>>>>
>>>>
>>>> Claus Ibsen-2 wrote:
>>>>>
>>>>> On Mon, Sep 28, 2009 at 10:05 AM, jpcook <jonathan.cook@erars.plus.com>
>>>>> wrote:
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> One last probably silly question. What you initialise consumerTemplate
>>>>>> as or
>>>>>> do you extend from a particular class for this to work?
>>>>>>
>>>>>> We're not using Spring so can't do any injection as per another thread
>>>>>> I
>>>>>> saw.
>>>>>>
>>>>>
>>>>> You can create a consumer template from the CamelContext
>>>>> And then you can reuse it on subsequent invocations.
>>>>>
>>>>> If you use a bean/pojo you can just have a CamelContext parameter in
>>>>> the method signature and Camel will inject the context for you.
>>>>>
>>>>> If you use a processor then you can get hold of the CamelContext from
>>>>> the Exchange using getContext()
>>>>>
>>>>> public void doSomething(CamelContext context) {
>>>>> if (consumerTemplate == null) {
>>>>>   consumerTemplate = context.createConsumerTemplate();
>>>>> }
>>>>>
>>>>>
>>>>>> And then does the url actually contain the atom part? Looking at the
>>>>>> wiki
>>>>>> page it would indicate it should do eg) Exchange exchange =
>>>>>> consumerTemplate.receive("activemq:my.queue");
>>>>>>
>>>>>
>>>>> Yeah you need to use the complete URL
>>>>>  consumerTemplate.receive("atom://and some more here");
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Claus Ibsen-2 wrote:
>>>>>>>
>>>>>>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook
>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> thanks
>>>>>>>>
>>>>>>>> The atom endpoint already has consumer.delay which I think I would
>>>>>>>> use.
>>>>>>>
>>>>>>> Yeah but the atom endpoint will then be configured with a static
>>>>>>> endpoint
>>>>>>> URI
>>>>>>> from("atom:staticUriHere?consumer.delay=5000")...
>>>>>>>
>>>>>>> Where as if you use a processor/bean/ with a consumer template you
>>>>>>> can do
>>>>>>>
>>>>>>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>>>>>>
>>>>>>> And then in your POJO you can poll the atom endpoint using a dynamic
>>>>>>> URI
>>>>>>>
>>>>>>> private List<String> uris;
>>>>>>>
>>>>>>> public void doSomething() {
>>>>>>>    // loop the list of dynamic uris and get the content from it
>>>>>>>   // and then consume from the endpoint using consumer template
>>>>>>>   Exchange out = consumerTemplate.receive(uri, 1000);
>>>>>>> }
>>>>>>>
>>>>>>> And see that link Charles mentioned.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Is there an example of the consumerTemplate?
>>>>>>>>
>>>>>>>> Could I use the web-console to configure the dynamic uris or just a
>>>>>>>> normal
>>>>>>>> xml configuration file?
>>>>>>>>
>>>>>>>> About the last point, that is how I have done it in the past but
>>>>>>>> just
>>>>>>>> thought I'd check if there was something built into camel now. :)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> You can use a scheduler / timer to trigger a route at a certain
>>>>>>>>> interval (quartz or timer)
>>>>>>>>>
>>>>>>>>> And then use a processor / bean with a consumerTemplate to consume
>>>>>>>>> from the atom feeds.
>>>>>>>>> Then you can use dynamic URIs.
>>>>>>>>>
>>>>>>>>> And if you want that to route in parallel you can use the JDK
>>>>>>>>> concurrency API for that as well.
>>>>>>>>>
>>>>>>>>> Sometimes the easiest stuff is to do that using regular java in a
>>>>>>>>> POJO.
>>>>>>>>> Submit tasks to the JDK executor services and then afterwards route
>>>>>>>>> the result to a file endpoint to store the file.
>>>>>>>>> Or a "direct" endpoint so you can do additional routing.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook
>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I have a requirement to pull 24 atom feeds, process them in the
>>>>>>>>>> same
>>>>>>>>>> way
>>>>>>>>>> via
>>>>>>>>>> xslt and then write the results to a file location which is
>>>>>>>>>> slightly
>>>>>>>>>> different for each feed. This is fairly straight forward.
>>>>>>>>>>
>>>>>>>>>> I was looking at the atom component as it looks almost perfect.
>>>>>>>>>> But I
>>>>>>>>>> wondered if there was a clever way I could maybe specify a list of
>>>>>>>>>> urls
>>>>>>>>>> to
>>>>>>>>>> the component and then it could process them concurrently as I
>>>>>>>>>> don't
>>>>>>>>>> want
>>>>>>>>>> to
>>>>>>>>>> have to do this synchronously? A bit like when you use the
>>>>>>>>>> splitter you
>>>>>>>>>> can
>>>>>>>>>> specify parallelProcessing()
>>>>>>>>>>
>>>>>>>>>> I guess another alternative would be to have a route for each feed
>>>>>>>>>> I
>>>>>>>>>> need
>>>>>>>>>> to
>>>>>>>>>> pull but this seemed a bit overkill as they would essentially be
>>>>>>>>>> all
>>>>>>>>>> the
>>>>>>>>>> same. Also I wanted to make the atom urls and the location that
>>>>>>>>>> the
>>>>>>>>>> result
>>>>>>>>>> gets written to configurable but we are not using the spring dsl
>>>>>>>>>> xml
>>>>>>>>>> configuration. As an alternative I could make these parameters
>>>>>>>>>> configurable
>>>>>>>>>> via my own configuration but I also wondered if I could perhaps
>>>>>>>>>> control
>>>>>>>>>> these parameters via JMX or even better via the web console?
>>>>>>>>>>
>>>>>>>>>> Any thoughts much appreciated. Thanks.
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25641681.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25681591.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
Hi,

I was also wondering why you can't specify that you want all the entries sorted? It appears that sortEntries only works when splitEntries is true?

Thanks

jpcook wrote
Thanks. Useful to know.
We aren't using maven unfortunately.

Claus Ibsen-2 wrote
On Wed, Sep 30, 2009 at 3:57 PM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Upgraded to the latest latest version 0.4.0 and it works. Might be useful to
> list the dependancies on the wiki? Camel just eats the exception as well.

Most people use maven and have the dependencies for free.

If you are not then you have to check this out yourself.
For example see the pom.xml file for the the component which lists the
dependencies it uses.

Or see the maven reposts (which can be old as they are not updated very often)
http://camel.apache.org/maven/camel-atom/dependency-convergence.html
http://camel.apache.org/maven/index.html


>
>
> jpcook wrote:
>>
>> Got further.
>>
>> What version of abdera does this work with. I have downloaded the latest
>> which is 0.3.0 and it doesn't seem to contain the getInstance method used
>> here.
>>
>> Abdera.getInstance().getParser();
>>
>> if I use Abdera.getNewParser() it works ok.
>>
>> Any ideas please?
>>
>>
>> jpcook wrote:
>>>
>>> Strange just using the http endpoint works fine.
>>>
>>>
>>> jpcook wrote:
>>>>
>>>> Hi,
>>>>
>>>> So just trying a very simple example like:
>>>> from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");
>>>>
>>>> But it returns:
>>>> 2009-09-30 12:05:04,650 INFO [main] afterAtom - Exchange[BodyType:null,
>>>> Body:null]
>>>>
>>>> No errors in the console so not sure what I'm doing wrong?
>>>>
>>>>
>>>> Claus Ibsen-2 wrote:
>>>>>
>>>>> On Mon, Sep 28, 2009 at 10:05 AM, jpcook <jonathan.cook@erars.plus.com>
>>>>> wrote:
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> One last probably silly question. What you initialise consumerTemplate
>>>>>> as or
>>>>>> do you extend from a particular class for this to work?
>>>>>>
>>>>>> We're not using Spring so can't do any injection as per another thread
>>>>>> I
>>>>>> saw.
>>>>>>
>>>>>
>>>>> You can create a consumer template from the CamelContext
>>>>> And then you can reuse it on subsequent invocations.
>>>>>
>>>>> If you use a bean/pojo you can just have a CamelContext parameter in
>>>>> the method signature and Camel will inject the context for you.
>>>>>
>>>>> If you use a processor then you can get hold of the CamelContext from
>>>>> the Exchange using getContext()
>>>>>
>>>>> public void doSomething(CamelContext context) {
>>>>> if (consumerTemplate == null) {
>>>>>   consumerTemplate = context.createConsumerTemplate();
>>>>> }
>>>>>
>>>>>
>>>>>> And then does the url actually contain the atom part? Looking at the
>>>>>> wiki
>>>>>> page it would indicate it should do eg) Exchange exchange =
>>>>>> consumerTemplate.receive("activemq:my.queue");
>>>>>>
>>>>>
>>>>> Yeah you need to use the complete URL
>>>>>  consumerTemplate.receive("atom://and some more here");
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Claus Ibsen-2 wrote:
>>>>>>>
>>>>>>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook
>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> thanks
>>>>>>>>
>>>>>>>> The atom endpoint already has consumer.delay which I think I would
>>>>>>>> use.
>>>>>>>
>>>>>>> Yeah but the atom endpoint will then be configured with a static
>>>>>>> endpoint
>>>>>>> URI
>>>>>>> from("atom:staticUriHere?consumer.delay=5000")...
>>>>>>>
>>>>>>> Where as if you use a processor/bean/ with a consumer template you
>>>>>>> can do
>>>>>>>
>>>>>>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>>>>>>
>>>>>>> And then in your POJO you can poll the atom endpoint using a dynamic
>>>>>>> URI
>>>>>>>
>>>>>>> private List<String> uris;
>>>>>>>
>>>>>>> public void doSomething() {
>>>>>>>    // loop the list of dynamic uris and get the content from it
>>>>>>>   // and then consume from the endpoint using consumer template
>>>>>>>   Exchange out = consumerTemplate.receive(uri, 1000);
>>>>>>> }
>>>>>>>
>>>>>>> And see that link Charles mentioned.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Is there an example of the consumerTemplate?
>>>>>>>>
>>>>>>>> Could I use the web-console to configure the dynamic uris or just a
>>>>>>>> normal
>>>>>>>> xml configuration file?
>>>>>>>>
>>>>>>>> About the last point, that is how I have done it in the past but
>>>>>>>> just
>>>>>>>> thought I'd check if there was something built into camel now. :)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> You can use a scheduler / timer to trigger a route at a certain
>>>>>>>>> interval (quartz or timer)
>>>>>>>>>
>>>>>>>>> And then use a processor / bean with a consumerTemplate to consume
>>>>>>>>> from the atom feeds.
>>>>>>>>> Then you can use dynamic URIs.
>>>>>>>>>
>>>>>>>>> And if you want that to route in parallel you can use the JDK
>>>>>>>>> concurrency API for that as well.
>>>>>>>>>
>>>>>>>>> Sometimes the easiest stuff is to do that using regular java in a
>>>>>>>>> POJO.
>>>>>>>>> Submit tasks to the JDK executor services and then afterwards route
>>>>>>>>> the result to a file endpoint to store the file.
>>>>>>>>> Or a "direct" endpoint so you can do additional routing.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook
>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I have a requirement to pull 24 atom feeds, process them in the
>>>>>>>>>> same
>>>>>>>>>> way
>>>>>>>>>> via
>>>>>>>>>> xslt and then write the results to a file location which is
>>>>>>>>>> slightly
>>>>>>>>>> different for each feed. This is fairly straight forward.
>>>>>>>>>>
>>>>>>>>>> I was looking at the atom component as it looks almost perfect.
>>>>>>>>>> But I
>>>>>>>>>> wondered if there was a clever way I could maybe specify a list of
>>>>>>>>>> urls
>>>>>>>>>> to
>>>>>>>>>> the component and then it could process them concurrently as I
>>>>>>>>>> don't
>>>>>>>>>> want
>>>>>>>>>> to
>>>>>>>>>> have to do this synchronously? A bit like when you use the
>>>>>>>>>> splitter you
>>>>>>>>>> can
>>>>>>>>>> specify parallelProcessing()
>>>>>>>>>>
>>>>>>>>>> I guess another alternative would be to have a route for each feed
>>>>>>>>>> I
>>>>>>>>>> need
>>>>>>>>>> to
>>>>>>>>>> pull but this seemed a bit overkill as they would essentially be
>>>>>>>>>> all
>>>>>>>>>> the
>>>>>>>>>> same. Also I wanted to make the atom urls and the location that
>>>>>>>>>> the
>>>>>>>>>> result
>>>>>>>>>> gets written to configurable but we are not using the spring dsl
>>>>>>>>>> xml
>>>>>>>>>> configuration. As an alternative I could make these parameters
>>>>>>>>>> configurable
>>>>>>>>>> via my own configuration but I also wondered if I could perhaps
>>>>>>>>>> control
>>>>>>>>>> these parameters via JMX or even better via the web console?
>>>>>>>>>>
>>>>>>>>>> Any thoughts much appreciated. Thanks.
>>>>>>>>>> --
>>>>>>>>>> View this message in context:
>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25641681.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25681591.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

Claus Ibsen-2
On Mon, Oct 5, 2009 at 5:01 PM, jpcook <[hidden email]> wrote:
>
> Hi,
>
> I was also wondering why you can't specify that you want all the entries
> sorted? It appears that sortEntries only works when splitEntries is true?

Create a ticket and we love contributions. So please try to fix it
yourself and submit a patch to us.

>
> Thanks
>
>
> jpcook wrote:
>>
>> Thanks. Useful to know.
>> We aren't using maven unfortunately.
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> On Wed, Sep 30, 2009 at 3:57 PM, jpcook <[hidden email]>
>>> wrote:
>>>>
>>>> Upgraded to the latest latest version 0.4.0 and it works. Might be
>>>> useful to
>>>> list the dependancies on the wiki? Camel just eats the exception as
>>>> well.
>>>
>>> Most people use maven and have the dependencies for free.
>>>
>>> If you are not then you have to check this out yourself.
>>> For example see the pom.xml file for the the component which lists the
>>> dependencies it uses.
>>>
>>> Or see the maven reposts (which can be old as they are not updated very
>>> often)
>>> http://camel.apache.org/maven/camel-atom/dependency-convergence.html
>>> http://camel.apache.org/maven/index.html
>>>
>>>
>>>>
>>>>
>>>> jpcook wrote:
>>>>>
>>>>> Got further.
>>>>>
>>>>> What version of abdera does this work with. I have downloaded the
>>>>> latest
>>>>> which is 0.3.0 and it doesn't seem to contain the getInstance method
>>>>> used
>>>>> here.
>>>>>
>>>>> Abdera.getInstance().getParser();
>>>>>
>>>>> if I use Abdera.getNewParser() it works ok.
>>>>>
>>>>> Any ideas please?
>>>>>
>>>>>
>>>>> jpcook wrote:
>>>>>>
>>>>>> Strange just using the http endpoint works fine.
>>>>>>
>>>>>>
>>>>>> jpcook wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> So just trying a very simple example like:
>>>>>>> from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");
>>>>>>>
>>>>>>> But it returns:
>>>>>>> 2009-09-30 12:05:04,650 INFO [main] afterAtom -
>>>>>>> Exchange[BodyType:null,
>>>>>>> Body:null]
>>>>>>>
>>>>>>> No errors in the console so not sure what I'm doing wrong?
>>>>>>>
>>>>>>>
>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>
>>>>>>>> On Mon, Sep 28, 2009 at 10:05 AM, jpcook
>>>>>>>> <[hidden email]>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>> One last probably silly question. What you initialise
>>>>>>>>> consumerTemplate
>>>>>>>>> as or
>>>>>>>>> do you extend from a particular class for this to work?
>>>>>>>>>
>>>>>>>>> We're not using Spring so can't do any injection as per another
>>>>>>>>> thread
>>>>>>>>> I
>>>>>>>>> saw.
>>>>>>>>>
>>>>>>>>
>>>>>>>> You can create a consumer template from the CamelContext
>>>>>>>> And then you can reuse it on subsequent invocations.
>>>>>>>>
>>>>>>>> If you use a bean/pojo you can just have a CamelContext parameter in
>>>>>>>> the method signature and Camel will inject the context for you.
>>>>>>>>
>>>>>>>> If you use a processor then you can get hold of the CamelContext
>>>>>>>> from
>>>>>>>> the Exchange using getContext()
>>>>>>>>
>>>>>>>> public void doSomething(CamelContext context) {
>>>>>>>> if (consumerTemplate == null) {
>>>>>>>>   consumerTemplate = context.createConsumerTemplate();
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>> And then does the url actually contain the atom part? Looking at
>>>>>>>>> the
>>>>>>>>> wiki
>>>>>>>>> page it would indicate it should do eg) Exchange exchange =
>>>>>>>>> consumerTemplate.receive("activemq:my.queue");
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yeah you need to use the complete URL
>>>>>>>>  consumerTemplate.receive("atom://and some more here");
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook
>>>>>>>>>> <[hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> thanks
>>>>>>>>>>>
>>>>>>>>>>> The atom endpoint already has consumer.delay which I think I
>>>>>>>>>>> would
>>>>>>>>>>> use.
>>>>>>>>>>
>>>>>>>>>> Yeah but the atom endpoint will then be configured with a static
>>>>>>>>>> endpoint
>>>>>>>>>> URI
>>>>>>>>>> from("atom:staticUriHere?consumer.delay=5000")...
>>>>>>>>>>
>>>>>>>>>> Where as if you use a processor/bean/ with a consumer template you
>>>>>>>>>> can do
>>>>>>>>>>
>>>>>>>>>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>>>>>>>>>
>>>>>>>>>> And then in your POJO you can poll the atom endpoint using a
>>>>>>>>>> dynamic
>>>>>>>>>> URI
>>>>>>>>>>
>>>>>>>>>> private List<String> uris;
>>>>>>>>>>
>>>>>>>>>> public void doSomething() {
>>>>>>>>>>    // loop the list of dynamic uris and get the content from it
>>>>>>>>>>   // and then consume from the endpoint using consumer template
>>>>>>>>>>   Exchange out = consumerTemplate.receive(uri, 1000);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> And see that link Charles mentioned.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Is there an example of the consumerTemplate?
>>>>>>>>>>>
>>>>>>>>>>> Could I use the web-console to configure the dynamic uris or just
>>>>>>>>>>> a
>>>>>>>>>>> normal
>>>>>>>>>>> xml configuration file?
>>>>>>>>>>>
>>>>>>>>>>> About the last point, that is how I have done it in the past but
>>>>>>>>>>> just
>>>>>>>>>>> thought I'd check if there was something built into camel now. :)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi
>>>>>>>>>>>>
>>>>>>>>>>>> You can use a scheduler / timer to trigger a route at a certain
>>>>>>>>>>>> interval (quartz or timer)
>>>>>>>>>>>>
>>>>>>>>>>>> And then use a processor / bean with a consumerTemplate to
>>>>>>>>>>>> consume
>>>>>>>>>>>> from the atom feeds.
>>>>>>>>>>>> Then you can use dynamic URIs.
>>>>>>>>>>>>
>>>>>>>>>>>> And if you want that to route in parallel you can use the JDK
>>>>>>>>>>>> concurrency API for that as well.
>>>>>>>>>>>>
>>>>>>>>>>>> Sometimes the easiest stuff is to do that using regular java in
>>>>>>>>>>>> a
>>>>>>>>>>>> POJO.
>>>>>>>>>>>> Submit tasks to the JDK executor services and then afterwards
>>>>>>>>>>>> route
>>>>>>>>>>>> the result to a file endpoint to store the file.
>>>>>>>>>>>> Or a "direct" endpoint so you can do additional routing.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook
>>>>>>>>>>>> <[hidden email]>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have a requirement to pull 24 atom feeds, process them in the
>>>>>>>>>>>>> same
>>>>>>>>>>>>> way
>>>>>>>>>>>>> via
>>>>>>>>>>>>> xslt and then write the results to a file location which is
>>>>>>>>>>>>> slightly
>>>>>>>>>>>>> different for each feed. This is fairly straight forward.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was looking at the atom component as it looks almost perfect.
>>>>>>>>>>>>> But I
>>>>>>>>>>>>> wondered if there was a clever way I could maybe specify a list
>>>>>>>>>>>>> of
>>>>>>>>>>>>> urls
>>>>>>>>>>>>> to
>>>>>>>>>>>>> the component and then it could process them concurrently as I
>>>>>>>>>>>>> don't
>>>>>>>>>>>>> want
>>>>>>>>>>>>> to
>>>>>>>>>>>>> have to do this synchronously? A bit like when you use the
>>>>>>>>>>>>> splitter you
>>>>>>>>>>>>> can
>>>>>>>>>>>>> specify parallelProcessing()
>>>>>>>>>>>>>
>>>>>>>>>>>>> I guess another alternative would be to have a route for each
>>>>>>>>>>>>> feed
>>>>>>>>>>>>> I
>>>>>>>>>>>>> need
>>>>>>>>>>>>> to
>>>>>>>>>>>>> pull but this seemed a bit overkill as they would essentially
>>>>>>>>>>>>> be
>>>>>>>>>>>>> all
>>>>>>>>>>>>> the
>>>>>>>>>>>>> same. Also I wanted to make the atom urls and the location that
>>>>>>>>>>>>> the
>>>>>>>>>>>>> result
>>>>>>>>>>>>> gets written to configurable but we are not using the spring
>>>>>>>>>>>>> dsl
>>>>>>>>>>>>> xml
>>>>>>>>>>>>> configuration. As an alternative I could make these parameters
>>>>>>>>>>>>> configurable
>>>>>>>>>>>>> via my own configuration but I also wondered if I could perhaps
>>>>>>>>>>>>> control
>>>>>>>>>>>>> these parameters via JMX or even better via the web console?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any thoughts much appreciated. Thanks.
>>>>>>>>>>>>> --
>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>>
>>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Claus Ibsen
>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>
>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25641681.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Atom-Component-tp25609495p25681591.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25752445.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
I knew you were going to say that :) I will!
Claus Ibsen-2 wrote
On Mon, Oct 5, 2009 at 5:01 PM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Hi,
>
> I was also wondering why you can't specify that you want all the entries
> sorted? It appears that sortEntries only works when splitEntries is true?

Create a ticket and we love contributions. So please try to fix it
yourself and submit a patch to us.

>
> Thanks
>
>
> jpcook wrote:
>>
>> Thanks. Useful to know.
>> We aren't using maven unfortunately.
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> On Wed, Sep 30, 2009 at 3:57 PM, jpcook <jonathan.cook@erars.plus.com>
>>> wrote:
>>>>
>>>> Upgraded to the latest latest version 0.4.0 and it works. Might be
>>>> useful to
>>>> list the dependancies on the wiki? Camel just eats the exception as
>>>> well.
>>>
>>> Most people use maven and have the dependencies for free.
>>>
>>> If you are not then you have to check this out yourself.
>>> For example see the pom.xml file for the the component which lists the
>>> dependencies it uses.
>>>
>>> Or see the maven reposts (which can be old as they are not updated very
>>> often)
>>> http://camel.apache.org/maven/camel-atom/dependency-convergence.html
>>> http://camel.apache.org/maven/index.html
>>>
>>>
>>>>
>>>>
>>>> jpcook wrote:
>>>>>
>>>>> Got further.
>>>>>
>>>>> What version of abdera does this work with. I have downloaded the
>>>>> latest
>>>>> which is 0.3.0 and it doesn't seem to contain the getInstance method
>>>>> used
>>>>> here.
>>>>>
>>>>> Abdera.getInstance().getParser();
>>>>>
>>>>> if I use Abdera.getNewParser() it works ok.
>>>>>
>>>>> Any ideas please?
>>>>>
>>>>>
>>>>> jpcook wrote:
>>>>>>
>>>>>> Strange just using the http endpoint works fine.
>>>>>>
>>>>>>
>>>>>> jpcook wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> So just trying a very simple example like:
>>>>>>> from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");
>>>>>>>
>>>>>>> But it returns:
>>>>>>> 2009-09-30 12:05:04,650 INFO [main] afterAtom -
>>>>>>> Exchange[BodyType:null,
>>>>>>> Body:null]
>>>>>>>
>>>>>>> No errors in the console so not sure what I'm doing wrong?
>>>>>>>
>>>>>>>
>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>
>>>>>>>> On Mon, Sep 28, 2009 at 10:05 AM, jpcook
>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>> One last probably silly question. What you initialise
>>>>>>>>> consumerTemplate
>>>>>>>>> as or
>>>>>>>>> do you extend from a particular class for this to work?
>>>>>>>>>
>>>>>>>>> We're not using Spring so can't do any injection as per another
>>>>>>>>> thread
>>>>>>>>> I
>>>>>>>>> saw.
>>>>>>>>>
>>>>>>>>
>>>>>>>> You can create a consumer template from the CamelContext
>>>>>>>> And then you can reuse it on subsequent invocations.
>>>>>>>>
>>>>>>>> If you use a bean/pojo you can just have a CamelContext parameter in
>>>>>>>> the method signature and Camel will inject the context for you.
>>>>>>>>
>>>>>>>> If you use a processor then you can get hold of the CamelContext
>>>>>>>> from
>>>>>>>> the Exchange using getContext()
>>>>>>>>
>>>>>>>> public void doSomething(CamelContext context) {
>>>>>>>> if (consumerTemplate == null) {
>>>>>>>>   consumerTemplate = context.createConsumerTemplate();
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>> And then does the url actually contain the atom part? Looking at
>>>>>>>>> the
>>>>>>>>> wiki
>>>>>>>>> page it would indicate it should do eg) Exchange exchange =
>>>>>>>>> consumerTemplate.receive("activemq:my.queue");
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yeah you need to use the complete URL
>>>>>>>>  consumerTemplate.receive("atom://and some more here");
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook
>>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> thanks
>>>>>>>>>>>
>>>>>>>>>>> The atom endpoint already has consumer.delay which I think I
>>>>>>>>>>> would
>>>>>>>>>>> use.
>>>>>>>>>>
>>>>>>>>>> Yeah but the atom endpoint will then be configured with a static
>>>>>>>>>> endpoint
>>>>>>>>>> URI
>>>>>>>>>> from("atom:staticUriHere?consumer.delay=5000")...
>>>>>>>>>>
>>>>>>>>>> Where as if you use a processor/bean/ with a consumer template you
>>>>>>>>>> can do
>>>>>>>>>>
>>>>>>>>>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>>>>>>>>>
>>>>>>>>>> And then in your POJO you can poll the atom endpoint using a
>>>>>>>>>> dynamic
>>>>>>>>>> URI
>>>>>>>>>>
>>>>>>>>>> private List<String> uris;
>>>>>>>>>>
>>>>>>>>>> public void doSomething() {
>>>>>>>>>>    // loop the list of dynamic uris and get the content from it
>>>>>>>>>>   // and then consume from the endpoint using consumer template
>>>>>>>>>>   Exchange out = consumerTemplate.receive(uri, 1000);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> And see that link Charles mentioned.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Is there an example of the consumerTemplate?
>>>>>>>>>>>
>>>>>>>>>>> Could I use the web-console to configure the dynamic uris or just
>>>>>>>>>>> a
>>>>>>>>>>> normal
>>>>>>>>>>> xml configuration file?
>>>>>>>>>>>
>>>>>>>>>>> About the last point, that is how I have done it in the past but
>>>>>>>>>>> just
>>>>>>>>>>> thought I'd check if there was something built into camel now. :)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi
>>>>>>>>>>>>
>>>>>>>>>>>> You can use a scheduler / timer to trigger a route at a certain
>>>>>>>>>>>> interval (quartz or timer)
>>>>>>>>>>>>
>>>>>>>>>>>> And then use a processor / bean with a consumerTemplate to
>>>>>>>>>>>> consume
>>>>>>>>>>>> from the atom feeds.
>>>>>>>>>>>> Then you can use dynamic URIs.
>>>>>>>>>>>>
>>>>>>>>>>>> And if you want that to route in parallel you can use the JDK
>>>>>>>>>>>> concurrency API for that as well.
>>>>>>>>>>>>
>>>>>>>>>>>> Sometimes the easiest stuff is to do that using regular java in
>>>>>>>>>>>> a
>>>>>>>>>>>> POJO.
>>>>>>>>>>>> Submit tasks to the JDK executor services and then afterwards
>>>>>>>>>>>> route
>>>>>>>>>>>> the result to a file endpoint to store the file.
>>>>>>>>>>>> Or a "direct" endpoint so you can do additional routing.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook
>>>>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have a requirement to pull 24 atom feeds, process them in the
>>>>>>>>>>>>> same
>>>>>>>>>>>>> way
>>>>>>>>>>>>> via
>>>>>>>>>>>>> xslt and then write the results to a file location which is
>>>>>>>>>>>>> slightly
>>>>>>>>>>>>> different for each feed. This is fairly straight forward.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was looking at the atom component as it looks almost perfect.
>>>>>>>>>>>>> But I
>>>>>>>>>>>>> wondered if there was a clever way I could maybe specify a list
>>>>>>>>>>>>> of
>>>>>>>>>>>>> urls
>>>>>>>>>>>>> to
>>>>>>>>>>>>> the component and then it could process them concurrently as I
>>>>>>>>>>>>> don't
>>>>>>>>>>>>> want
>>>>>>>>>>>>> to
>>>>>>>>>>>>> have to do this synchronously? A bit like when you use the
>>>>>>>>>>>>> splitter you
>>>>>>>>>>>>> can
>>>>>>>>>>>>> specify parallelProcessing()
>>>>>>>>>>>>>
>>>>>>>>>>>>> I guess another alternative would be to have a route for each
>>>>>>>>>>>>> feed
>>>>>>>>>>>>> I
>>>>>>>>>>>>> need
>>>>>>>>>>>>> to
>>>>>>>>>>>>> pull but this seemed a bit overkill as they would essentially
>>>>>>>>>>>>> be
>>>>>>>>>>>>> all
>>>>>>>>>>>>> the
>>>>>>>>>>>>> same. Also I wanted to make the atom urls and the location that
>>>>>>>>>>>>> the
>>>>>>>>>>>>> result
>>>>>>>>>>>>> gets written to configurable but we are not using the spring
>>>>>>>>>>>>> dsl
>>>>>>>>>>>>> xml
>>>>>>>>>>>>> configuration. As an alternative I could make these parameters
>>>>>>>>>>>>> configurable
>>>>>>>>>>>>> via my own configuration but I also wondered if I could perhaps
>>>>>>>>>>>>> control
>>>>>>>>>>>>> these parameters via JMX or even better via the web console?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any thoughts much appreciated. Thanks.
>>>>>>>>>>>>> --
>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>>
>>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Claus Ibsen
>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>
>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25641681.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Atom-Component-tp25609495p25681591.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25752445.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
In reply to this post by Claus Ibsen-2
Hello,

I ended up solving this problem with the follow configure method:

private void configureNewsTrackerByTeamHtml() throws Exception {
               
                for(String url : urls){
                        from("atom://"+url+"?splitEntries=false&consumer.delay="+consumerDelay)
                                .to("seda:footballNewsTrackerByTeamHtml");
                }
               
                from("seda:footballNewsTrackerByTeamHtml").convertBodyTo(String.class)
                        .process(new SearchFeedProcessor()).convertBodyTo(String.class)
                        .process(XsltBuilder.xslt(new File("xslt//football//newstracker//html//football_newstracker.html.xslt")))
                        .setHeader(FileComponent.HEADER_FILE_NAME,
                                        constant("/feeds/football/teams/")
                                        .append(header(FeedConsumerConstants.SEARCHFEED_TEAM_ID))
                                        .append("/news_tracker.inc"))                
                        .to("file://#config(output.content)?append=false&noop=true&fileEncoding=ISO-8859-1");
        }

Not sure if this has many disadvantages over the solution you proposed using a pojo and consumer template but it works nicely?

Thanks

Claus Ibsen-2 wrote
On Mon, Oct 5, 2009 at 5:01 PM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Hi,
>
> I was also wondering why you can't specify that you want all the entries
> sorted? It appears that sortEntries only works when splitEntries is true?

Create a ticket and we love contributions. So please try to fix it
yourself and submit a patch to us.

>
> Thanks
>
>
> jpcook wrote:
>>
>> Thanks. Useful to know.
>> We aren't using maven unfortunately.
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> On Wed, Sep 30, 2009 at 3:57 PM, jpcook <jonathan.cook@erars.plus.com>
>>> wrote:
>>>>
>>>> Upgraded to the latest latest version 0.4.0 and it works. Might be
>>>> useful to
>>>> list the dependancies on the wiki? Camel just eats the exception as
>>>> well.
>>>
>>> Most people use maven and have the dependencies for free.
>>>
>>> If you are not then you have to check this out yourself.
>>> For example see the pom.xml file for the the component which lists the
>>> dependencies it uses.
>>>
>>> Or see the maven reposts (which can be old as they are not updated very
>>> often)
>>> http://camel.apache.org/maven/camel-atom/dependency-convergence.html
>>> http://camel.apache.org/maven/index.html
>>>
>>>
>>>>
>>>>
>>>> jpcook wrote:
>>>>>
>>>>> Got further.
>>>>>
>>>>> What version of abdera does this work with. I have downloaded the
>>>>> latest
>>>>> which is 0.3.0 and it doesn't seem to contain the getInstance method
>>>>> used
>>>>> here.
>>>>>
>>>>> Abdera.getInstance().getParser();
>>>>>
>>>>> if I use Abdera.getNewParser() it works ok.
>>>>>
>>>>> Any ideas please?
>>>>>
>>>>>
>>>>> jpcook wrote:
>>>>>>
>>>>>> Strange just using the http endpoint works fine.
>>>>>>
>>>>>>
>>>>>> jpcook wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> So just trying a very simple example like:
>>>>>>> from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");
>>>>>>>
>>>>>>> But it returns:
>>>>>>> 2009-09-30 12:05:04,650 INFO [main] afterAtom -
>>>>>>> Exchange[BodyType:null,
>>>>>>> Body:null]
>>>>>>>
>>>>>>> No errors in the console so not sure what I'm doing wrong?
>>>>>>>
>>>>>>>
>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>
>>>>>>>> On Mon, Sep 28, 2009 at 10:05 AM, jpcook
>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>> One last probably silly question. What you initialise
>>>>>>>>> consumerTemplate
>>>>>>>>> as or
>>>>>>>>> do you extend from a particular class for this to work?
>>>>>>>>>
>>>>>>>>> We're not using Spring so can't do any injection as per another
>>>>>>>>> thread
>>>>>>>>> I
>>>>>>>>> saw.
>>>>>>>>>
>>>>>>>>
>>>>>>>> You can create a consumer template from the CamelContext
>>>>>>>> And then you can reuse it on subsequent invocations.
>>>>>>>>
>>>>>>>> If you use a bean/pojo you can just have a CamelContext parameter in
>>>>>>>> the method signature and Camel will inject the context for you.
>>>>>>>>
>>>>>>>> If you use a processor then you can get hold of the CamelContext
>>>>>>>> from
>>>>>>>> the Exchange using getContext()
>>>>>>>>
>>>>>>>> public void doSomething(CamelContext context) {
>>>>>>>> if (consumerTemplate == null) {
>>>>>>>>   consumerTemplate = context.createConsumerTemplate();
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>> And then does the url actually contain the atom part? Looking at
>>>>>>>>> the
>>>>>>>>> wiki
>>>>>>>>> page it would indicate it should do eg) Exchange exchange =
>>>>>>>>> consumerTemplate.receive("activemq:my.queue");
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yeah you need to use the complete URL
>>>>>>>>  consumerTemplate.receive("atom://and some more here");
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook
>>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> thanks
>>>>>>>>>>>
>>>>>>>>>>> The atom endpoint already has consumer.delay which I think I
>>>>>>>>>>> would
>>>>>>>>>>> use.
>>>>>>>>>>
>>>>>>>>>> Yeah but the atom endpoint will then be configured with a static
>>>>>>>>>> endpoint
>>>>>>>>>> URI
>>>>>>>>>> from("atom:staticUriHere?consumer.delay=5000")...
>>>>>>>>>>
>>>>>>>>>> Where as if you use a processor/bean/ with a consumer template you
>>>>>>>>>> can do
>>>>>>>>>>
>>>>>>>>>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>>>>>>>>>
>>>>>>>>>> And then in your POJO you can poll the atom endpoint using a
>>>>>>>>>> dynamic
>>>>>>>>>> URI
>>>>>>>>>>
>>>>>>>>>> private List<String> uris;
>>>>>>>>>>
>>>>>>>>>> public void doSomething() {
>>>>>>>>>>    // loop the list of dynamic uris and get the content from it
>>>>>>>>>>   // and then consume from the endpoint using consumer template
>>>>>>>>>>   Exchange out = consumerTemplate.receive(uri, 1000);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> And see that link Charles mentioned.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Is there an example of the consumerTemplate?
>>>>>>>>>>>
>>>>>>>>>>> Could I use the web-console to configure the dynamic uris or just
>>>>>>>>>>> a
>>>>>>>>>>> normal
>>>>>>>>>>> xml configuration file?
>>>>>>>>>>>
>>>>>>>>>>> About the last point, that is how I have done it in the past but
>>>>>>>>>>> just
>>>>>>>>>>> thought I'd check if there was something built into camel now. :)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi
>>>>>>>>>>>>
>>>>>>>>>>>> You can use a scheduler / timer to trigger a route at a certain
>>>>>>>>>>>> interval (quartz or timer)
>>>>>>>>>>>>
>>>>>>>>>>>> And then use a processor / bean with a consumerTemplate to
>>>>>>>>>>>> consume
>>>>>>>>>>>> from the atom feeds.
>>>>>>>>>>>> Then you can use dynamic URIs.
>>>>>>>>>>>>
>>>>>>>>>>>> And if you want that to route in parallel you can use the JDK
>>>>>>>>>>>> concurrency API for that as well.
>>>>>>>>>>>>
>>>>>>>>>>>> Sometimes the easiest stuff is to do that using regular java in
>>>>>>>>>>>> a
>>>>>>>>>>>> POJO.
>>>>>>>>>>>> Submit tasks to the JDK executor services and then afterwards
>>>>>>>>>>>> route
>>>>>>>>>>>> the result to a file endpoint to store the file.
>>>>>>>>>>>> Or a "direct" endpoint so you can do additional routing.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook
>>>>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have a requirement to pull 24 atom feeds, process them in the
>>>>>>>>>>>>> same
>>>>>>>>>>>>> way
>>>>>>>>>>>>> via
>>>>>>>>>>>>> xslt and then write the results to a file location which is
>>>>>>>>>>>>> slightly
>>>>>>>>>>>>> different for each feed. This is fairly straight forward.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was looking at the atom component as it looks almost perfect.
>>>>>>>>>>>>> But I
>>>>>>>>>>>>> wondered if there was a clever way I could maybe specify a list
>>>>>>>>>>>>> of
>>>>>>>>>>>>> urls
>>>>>>>>>>>>> to
>>>>>>>>>>>>> the component and then it could process them concurrently as I
>>>>>>>>>>>>> don't
>>>>>>>>>>>>> want
>>>>>>>>>>>>> to
>>>>>>>>>>>>> have to do this synchronously? A bit like when you use the
>>>>>>>>>>>>> splitter you
>>>>>>>>>>>>> can
>>>>>>>>>>>>> specify parallelProcessing()
>>>>>>>>>>>>>
>>>>>>>>>>>>> I guess another alternative would be to have a route for each
>>>>>>>>>>>>> feed
>>>>>>>>>>>>> I
>>>>>>>>>>>>> need
>>>>>>>>>>>>> to
>>>>>>>>>>>>> pull but this seemed a bit overkill as they would essentially
>>>>>>>>>>>>> be
>>>>>>>>>>>>> all
>>>>>>>>>>>>> the
>>>>>>>>>>>>> same. Also I wanted to make the atom urls and the location that
>>>>>>>>>>>>> the
>>>>>>>>>>>>> result
>>>>>>>>>>>>> gets written to configurable but we are not using the spring
>>>>>>>>>>>>> dsl
>>>>>>>>>>>>> xml
>>>>>>>>>>>>> configuration. As an alternative I could make these parameters
>>>>>>>>>>>>> configurable
>>>>>>>>>>>>> via my own configuration but I also wondered if I could perhaps
>>>>>>>>>>>>> control
>>>>>>>>>>>>> these parameters via JMX or even better via the web console?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any thoughts much appreciated. Thanks.
>>>>>>>>>>>>> --
>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>>
>>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Claus Ibsen
>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>
>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25641681.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Atom-Component-tp25609495p25681591.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25752445.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Atom Component

jpcook01
Hi,

Just wondered if anyone had read the below?
Thanks

jpcook wrote
Hello,

I ended up solving this problem with the follow configure method:

private void configureNewsTrackerByTeamHtml() throws Exception {
               
                for(String url : urls){
                        from("atom://"+url+"?splitEntries=false&consumer.delay="+consumerDelay)
                                .to("seda:footballNewsTrackerByTeamHtml");
                }
               
                from("seda:footballNewsTrackerByTeamHtml").convertBodyTo(String.class)
                        .process(new SearchFeedProcessor()).convertBodyTo(String.class)
                        .process(XsltBuilder.xslt(new File("xslt//football//newstracker//html//football_newstracker.html.xslt")))
                        .setHeader(FileComponent.HEADER_FILE_NAME,
                                        constant("/feeds/football/teams/")
                                        .append(header(FeedConsumerConstants.SEARCHFEED_TEAM_ID))
                                        .append("/news_tracker.inc"))                
                        .to("file://#config(output.content)?append=false&noop=true&fileEncoding=ISO-8859-1");
        }

Not sure if this has many disadvantages over the solution you proposed using a pojo and consumer template but it works nicely?

Thanks

Claus Ibsen-2 wrote
On Mon, Oct 5, 2009 at 5:01 PM, jpcook <jonathan.cook@erars.plus.com> wrote:
>
> Hi,
>
> I was also wondering why you can't specify that you want all the entries
> sorted? It appears that sortEntries only works when splitEntries is true?

Create a ticket and we love contributions. So please try to fix it
yourself and submit a patch to us.

>
> Thanks
>
>
> jpcook wrote:
>>
>> Thanks. Useful to know.
>> We aren't using maven unfortunately.
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> On Wed, Sep 30, 2009 at 3:57 PM, jpcook <jonathan.cook@erars.plus.com>
>>> wrote:
>>>>
>>>> Upgraded to the latest latest version 0.4.0 and it works. Might be
>>>> useful to
>>>> list the dependancies on the wiki? Camel just eats the exception as
>>>> well.
>>>
>>> Most people use maven and have the dependencies for free.
>>>
>>> If you are not then you have to check this out yourself.
>>> For example see the pom.xml file for the the component which lists the
>>> dependencies it uses.
>>>
>>> Or see the maven reposts (which can be old as they are not updated very
>>> often)
>>> http://camel.apache.org/maven/camel-atom/dependency-convergence.html
>>> http://camel.apache.org/maven/index.html
>>>
>>>
>>>>
>>>>
>>>> jpcook wrote:
>>>>>
>>>>> Got further.
>>>>>
>>>>> What version of abdera does this work with. I have downloaded the
>>>>> latest
>>>>> which is 0.3.0 and it doesn't seem to contain the getInstance method
>>>>> used
>>>>> here.
>>>>>
>>>>> Abdera.getInstance().getParser();
>>>>>
>>>>> if I use Abdera.getNewParser() it works ok.
>>>>>
>>>>> Any ideas please?
>>>>>
>>>>>
>>>>> jpcook wrote:
>>>>>>
>>>>>> Strange just using the http endpoint works fine.
>>>>>>
>>>>>>
>>>>>> jpcook wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> So just trying a very simple example like:
>>>>>>> from("atom://http://macstrac.blogspot.com/feeds/posts/default").to("log:afterAtom");
>>>>>>>
>>>>>>> But it returns:
>>>>>>> 2009-09-30 12:05:04,650 INFO [main] afterAtom -
>>>>>>> Exchange[BodyType:null,
>>>>>>> Body:null]
>>>>>>>
>>>>>>> No errors in the console so not sure what I'm doing wrong?
>>>>>>>
>>>>>>>
>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>
>>>>>>>> On Mon, Sep 28, 2009 at 10:05 AM, jpcook
>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>>
>>>>>>>>> One last probably silly question. What you initialise
>>>>>>>>> consumerTemplate
>>>>>>>>> as or
>>>>>>>>> do you extend from a particular class for this to work?
>>>>>>>>>
>>>>>>>>> We're not using Spring so can't do any injection as per another
>>>>>>>>> thread
>>>>>>>>> I
>>>>>>>>> saw.
>>>>>>>>>
>>>>>>>>
>>>>>>>> You can create a consumer template from the CamelContext
>>>>>>>> And then you can reuse it on subsequent invocations.
>>>>>>>>
>>>>>>>> If you use a bean/pojo you can just have a CamelContext parameter in
>>>>>>>> the method signature and Camel will inject the context for you.
>>>>>>>>
>>>>>>>> If you use a processor then you can get hold of the CamelContext
>>>>>>>> from
>>>>>>>> the Exchange using getContext()
>>>>>>>>
>>>>>>>> public void doSomething(CamelContext context) {
>>>>>>>> if (consumerTemplate == null) {
>>>>>>>>   consumerTemplate = context.createConsumerTemplate();
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>> And then does the url actually contain the atom part? Looking at
>>>>>>>>> the
>>>>>>>>> wiki
>>>>>>>>> page it would indicate it should do eg) Exchange exchange =
>>>>>>>>> consumerTemplate.receive("activemq:my.queue");
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yeah you need to use the complete URL
>>>>>>>>  consumerTemplate.receive("atom://and some more here");
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>
>>>>>>>>>> On Fri, Sep 25, 2009 at 1:02 PM, jpcook
>>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> thanks
>>>>>>>>>>>
>>>>>>>>>>> The atom endpoint already has consumer.delay which I think I
>>>>>>>>>>> would
>>>>>>>>>>> use.
>>>>>>>>>>
>>>>>>>>>> Yeah but the atom endpoint will then be configured with a static
>>>>>>>>>> endpoint
>>>>>>>>>> URI
>>>>>>>>>> from("atom:staticUriHere?consumer.delay=5000")...
>>>>>>>>>>
>>>>>>>>>> Where as if you use a processor/bean/ with a consumer template you
>>>>>>>>>> can do
>>>>>>>>>>
>>>>>>>>>> from("timer://foo?delay=5000").beanRef("myBean", "doSomething)
>>>>>>>>>>
>>>>>>>>>> And then in your POJO you can poll the atom endpoint using a
>>>>>>>>>> dynamic
>>>>>>>>>> URI
>>>>>>>>>>
>>>>>>>>>> private List<String> uris;
>>>>>>>>>>
>>>>>>>>>> public void doSomething() {
>>>>>>>>>>    // loop the list of dynamic uris and get the content from it
>>>>>>>>>>   // and then consume from the endpoint using consumer template
>>>>>>>>>>   Exchange out = consumerTemplate.receive(uri, 1000);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> And see that link Charles mentioned.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Is there an example of the consumerTemplate?
>>>>>>>>>>>
>>>>>>>>>>> Could I use the web-console to configure the dynamic uris or just
>>>>>>>>>>> a
>>>>>>>>>>> normal
>>>>>>>>>>> xml configuration file?
>>>>>>>>>>>
>>>>>>>>>>> About the last point, that is how I have done it in the past but
>>>>>>>>>>> just
>>>>>>>>>>> thought I'd check if there was something built into camel now. :)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Claus Ibsen-2 wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi
>>>>>>>>>>>>
>>>>>>>>>>>> You can use a scheduler / timer to trigger a route at a certain
>>>>>>>>>>>> interval (quartz or timer)
>>>>>>>>>>>>
>>>>>>>>>>>> And then use a processor / bean with a consumerTemplate to
>>>>>>>>>>>> consume
>>>>>>>>>>>> from the atom feeds.
>>>>>>>>>>>> Then you can use dynamic URIs.
>>>>>>>>>>>>
>>>>>>>>>>>> And if you want that to route in parallel you can use the JDK
>>>>>>>>>>>> concurrency API for that as well.
>>>>>>>>>>>>
>>>>>>>>>>>> Sometimes the easiest stuff is to do that using regular java in
>>>>>>>>>>>> a
>>>>>>>>>>>> POJO.
>>>>>>>>>>>> Submit tasks to the JDK executor services and then afterwards
>>>>>>>>>>>> route
>>>>>>>>>>>> the result to a file endpoint to store the file.
>>>>>>>>>>>> Or a "direct" endpoint so you can do additional routing.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Sep 25, 2009 at 12:02 PM, jpcook
>>>>>>>>>>>> <jonathan.cook@erars.plus.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have a requirement to pull 24 atom feeds, process them in the
>>>>>>>>>>>>> same
>>>>>>>>>>>>> way
>>>>>>>>>>>>> via
>>>>>>>>>>>>> xslt and then write the results to a file location which is
>>>>>>>>>>>>> slightly
>>>>>>>>>>>>> different for each feed. This is fairly straight forward.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I was looking at the atom component as it looks almost perfect.
>>>>>>>>>>>>> But I
>>>>>>>>>>>>> wondered if there was a clever way I could maybe specify a list
>>>>>>>>>>>>> of
>>>>>>>>>>>>> urls
>>>>>>>>>>>>> to
>>>>>>>>>>>>> the component and then it could process them concurrently as I
>>>>>>>>>>>>> don't
>>>>>>>>>>>>> want
>>>>>>>>>>>>> to
>>>>>>>>>>>>> have to do this synchronously? A bit like when you use the
>>>>>>>>>>>>> splitter you
>>>>>>>>>>>>> can
>>>>>>>>>>>>> specify parallelProcessing()
>>>>>>>>>>>>>
>>>>>>>>>>>>> I guess another alternative would be to have a route for each
>>>>>>>>>>>>> feed
>>>>>>>>>>>>> I
>>>>>>>>>>>>> need
>>>>>>>>>>>>> to
>>>>>>>>>>>>> pull but this seemed a bit overkill as they would essentially
>>>>>>>>>>>>> be
>>>>>>>>>>>>> all
>>>>>>>>>>>>> the
>>>>>>>>>>>>> same. Also I wanted to make the atom urls and the location that
>>>>>>>>>>>>> the
>>>>>>>>>>>>> result
>>>>>>>>>>>>> gets written to configurable but we are not using the spring
>>>>>>>>>>>>> dsl
>>>>>>>>>>>>> xml
>>>>>>>>>>>>> configuration. As an alternative I could make these parameters
>>>>>>>>>>>>> configurable
>>>>>>>>>>>>> via my own configuration but I also wondered if I could perhaps
>>>>>>>>>>>>> control
>>>>>>>>>>>>> these parameters via JMX or even better via the web console?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any thoughts much appreciated. Thanks.
>>>>>>>>>>>>> --
>>>>>>>>>>>>> View this message in context:
>>>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25609495.html
>>>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Claus Ibsen
>>>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>>>
>>>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> View this message in context:
>>>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25610124.html
>>>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Claus Ibsen
>>>>>>>>>> Apache Camel Committer
>>>>>>>>>>
>>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context:
>>>>>>>>> http://www.nabble.com/Atom-Component-tp25609495p25641681.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Atom-Component-tp25609495p25681591.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/Atom-Component-tp25609495p25752445.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



--
Claus Ibsen
Apache Camel Committer

Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus