[DESIGN] - Camel 2.x - Triggering scheduled consumer

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[DESIGN] - Camel 2.x - Triggering scheduled consumer

Claus Ibsen-2
Hi

In lack of a better subject I named it "Triggering scheduled consumers"

An end user had a very decent question last day that was a bit of a
challenge for me to do nicely in Camel (and still is).
See nabble: http://www.nabble.com/Trigger-causes-Poll-from-ftp-server-to-sftp-td20934240s22882.html

The use-case is that he wants to poll a file from a FTP server from
within a route. And how do you do that in Camel?
As the FTP consumer is a scheduled consumer its only to be used in the
from type.

But he wants to trigger this when a message arrives on a JMS queue
      from(jms:queue).XXXXX.to(somewhere else)
where XXX is where we download the file from the FTP server

I do think that downloading a single file from a FTP server, or
reading a file from a file system should be easy and possible from
within a route.
But for both the FTP consumer and the File consumer they are scheduled
based. And this is because they can by interval scan folders for files
to consume and fire them in the route.

However reusing these components for a use-case: I need to download a
file or I need to read a file is a valid question that end users of
Camel will raise and try/expect it to be able to.

So do we have other Camel components where end users might wanna do
something like this? For instance Jon's new RSS component?

So I am wondering how we should support this for the future?

Using endpoint.createPollingConsumer() was my first thought of a nice
solution but then again it expects a single Exchange as return and the
FTP and File consumers is event based firing a new exchange per file
they process. But this API is probably where we should go, so end
users can do

PollingConsumer pc = endpoint.createPollingConsumer();
Exchange exchange = pc.receive();


For now I am just spinning of this thread for starters.



--

/Claus Ibsen
Apache Camel Committer
Blog: http://davsclaus.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: [DESIGN] - Camel 2.x - Triggering scheduled consumer

jstrachan
2008/12/11 Claus Ibsen <[hidden email]>:

> Hi
>
> In lack of a better subject I named it "Triggering scheduled consumers"
>
> An end user had a very decent question last day that was a bit of a
> challenge for me to do nicely in Camel (and still is).
> See nabble: http://www.nabble.com/Trigger-causes-Poll-from-ftp-server-to-sftp-td20934240s22882.html
>
> The use-case is that he wants to poll a file from a FTP server from
> within a route. And how do you do that in Camel?
> As the FTP consumer is a scheduled consumer its only to be used in the
> from type.
>
> But he wants to trigger this when a message arrives on a JMS queue
>      from(jms:queue).XXXXX.to(somewhere else)
> where XXX is where we download the file from the FTP server
>
> I do think that downloading a single file from a FTP server, or
> reading a file from a file system should be easy and possible from
> within a route.
> But for both the FTP consumer and the File consumer they are scheduled
> based. And this is because they can by interval scan folders for files
> to consume and fire them in the route.
>
> However reusing these components for a use-case: I need to download a
> file or I need to read a file is a valid question that end users of
> Camel will raise and try/expect it to be able to.
>
> So do we have other Camel components where end users might wanna do
> something like this? For instance Jon's new RSS component?

Yeah - being able to easily poll an endpoint is required quite often
I'd say. Ditto HTTP/FTP and databases too

One approach could be we use ExchangePattern?

e.g.

* InOnly for a File endpoint could write the file. (bit like a HTTP PUT?)
* InOut could write and read it back? (bit like a HTTP POST?)
* OutOnly could just read the file thats there (bit like a HTTP GET)



--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/
Reply | Threaded
Open this post in threaded view
|

Re: [DESIGN] - Camel 2.x - Triggering scheduled consumer

Ramon Buckland
In reply to this post by Claus Ibsen-2
>
> So do we have other Camel components where end users might wanna do
> something like this? For instance Jon's new RSS component?
>

Yes I have two components in the pipe line which specifically use / will
need this pattern.

(see

http://markmail.org/search/?q=camel%20ramon%20foo#query:camel%20ramon%20foo+page:1+mid:xbyvrht2bjc5qr57+state:results
)

more details once I am closer :-)
Reply | Threaded
Open this post in threaded view
|

Re: [DESIGN] - Camel 2.x - Triggering scheduled consumer

Jon Anstey
This would be a really nice feature. It would make merging multiple rss
sources much nicer. It could also be used in Fintan's new enricher feature
(CAMEL-1106).

On Thu, Dec 11, 2008 at 11:17 AM, Ramon Buckland <[hidden email]>wrote:

> >
> > So do we have other Camel components where end users might wanna do
> > something like this? For instance Jon's new RSS component?
> >
>
> Yes I have two components in the pipe line which specifically use / will
> need this pattern.
>
> (see
>
>
> http://markmail.org/search/?q=camel%20ramon%20foo#query:camel%20ramon%20foo+page:1+mid:xbyvrht2bjc5qr57+state:results
> )
>
> more details once I am closer :-)
>



--
Cheers,
Jon

http://janstey.blogspot.com/
Reply | Threaded
Open this post in threaded view
|

Re: [DESIGN] - Camel 2.x - Triggering scheduled consumer

Claus Ibsen-2
Hi

BTW I got the ftp sample much cleaner and nicer. My first attempt at
it somehow got messed, but now it's a textbox Camel API example
                        PollingConsumer consumer = ftp.createPollingConsumer();
                        consumer.start();
                        // receive the remote ftp without timeout
                        Exchange result = consumer.receive();
                        // we must stop the consumer
                        consumer.stop();



On Thu, Dec 11, 2008 at 5:18 PM, Jon Anstey <[hidden email]> wrote:

> This would be a really nice feature. It would make merging multiple rss
> sources much nicer. It could also be used in Fintan's new enricher feature
> (CAMEL-1106).
>
> On Thu, Dec 11, 2008 at 11:17 AM, Ramon Buckland <[hidden email]>wrote:
>
>> >
>> > So do we have other Camel components where end users might wanna do
>> > something like this? For instance Jon's new RSS component?
>> >
>>
>> Yes I have two components in the pipe line which specifically use / will
>> need this pattern.
>>
>> (see
>>
>>
>> http://markmail.org/search/?q=camel%20ramon%20foo#query:camel%20ramon%20foo+page:1+mid:xbyvrht2bjc5qr57+state:results
>> )
>>
>> more details once I am closer :-)
>>
>
>
>
> --
> Cheers,
> Jon
>
> http://janstey.blogspot.com/
>



--

/Claus Ibsen
Apache Camel Committer
Blog: http://davsclaus.blogspot.com/