Jetty -> AHC route producing a lot of threads

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Jetty -> AHC route producing a lot of threads

Les Novell
Hello, I am creating a simple route to test asynchronous message passing in Camel.  When testing the route by creating 1000 simultaneous connections I see that around 2000 threads are created to handle requests I'm forwarding on using AHC component.  I am surprised, I thought that Jetty+AHC component would use non-blocking asynchronous I/O, and a low number of threads.

Basic Route:
        restConfiguration().component("jetty").host("localhost").port(8081);
        rest("/ud").get("/wait/{timeout}")
                .to("direct:mockEndpoint").bindingMode(RestBindingMode.off);
        from("direct:mockEndpoint")
                .removeHeaders("CamelHttpPath")
                .toD("ahc:http://localhost:9000/wait/${headers.timeout}?bridgeEndpoint=true");

It appears that the AHC component is creating worker threads to manage outgoing connections to localhost:9000. One for every connection made to localhost:9090.  Am I configuring something wrong, or is this a defect in AHC?

Camel version is 2.16-SNAPSHOT

Thanks,
Les
Reply | Threaded
Open this post in threaded view
|

Re: Jetty -> AHC route producing a lot of threads

Claus Ibsen-2
Hi

This would be expected as the endpoint is dynamic and therefore each
endpoint has their own producer.

You should try to use the same endpoint, and if possible if AHC /
camel-ahc allows dynamic timeout per request. I am not sure if that is
possible / supported.



On Mon, Aug 31, 2015 at 11:13 PM, Les Novell
<[hidden email]> wrote:

> Hello, I am creating a simple route to test asynchronous message passing in
> Camel.  When testing the route by creating 1000 simultaneous connections I
> see that around 2000 threads are created to handle requests I'm forwarding
> on using AHC component.  I am surprised, I thought that Jetty+AHC component
> would use non-blocking asynchronous I/O, and a low number of threads.
>
> Basic Route:
>         restConfiguration().component("jetty").host("localhost").port(8081);
>         rest("/ud").get("/wait/{timeout}")
>                 .to("direct:mockEndpoint").bindingMode(RestBindingMode.off);
>         from("direct:mockEndpoint")
>                 .removeHeaders("CamelHttpPath")
>
> .toD("ahc:http://localhost:9000/wait/${headers.timeout}?bridgeEndpoint=true");
>
> It appears that the AHC component is creating worker threads to manage
> outgoing connections to localhost:9000. One for every connection made to
> localhost:9090.  Am I configuring something wrong, or is this a defect in
> AHC?
>
> Camel version is 2.16-SNAPSHOT
>
> Thanks,
> Les
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Jetty-AHC-route-producing-a-lot-of-threads-tp5771106.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2nd edition: http://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: Jetty -> AHC route producing a lot of threads

Claus Ibsen-2
Hi

Ah the wait was part of the content path. Instead of <toD> you can
build the url as a header with the key Exchange.HTTP_URI, then the ahc
endpoint can be hardcoded and the header will be used as url instead
from the endpoint. Then you can reuse the endpoint/producers.



On Thu, Sep 3, 2015 at 5:21 PM, Claus Ibsen <[hidden email]> wrote:

> Hi
>
> This would be expected as the endpoint is dynamic and therefore each
> endpoint has their own producer.
>
> You should try to use the same endpoint, and if possible if AHC /
> camel-ahc allows dynamic timeout per request. I am not sure if that is
> possible / supported.
>
>
>
> On Mon, Aug 31, 2015 at 11:13 PM, Les Novell
> <[hidden email]> wrote:
>> Hello, I am creating a simple route to test asynchronous message passing in
>> Camel.  When testing the route by creating 1000 simultaneous connections I
>> see that around 2000 threads are created to handle requests I'm forwarding
>> on using AHC component.  I am surprised, I thought that Jetty+AHC component
>> would use non-blocking asynchronous I/O, and a low number of threads.
>>
>> Basic Route:
>>         restConfiguration().component("jetty").host("localhost").port(8081);
>>         rest("/ud").get("/wait/{timeout}")
>>                 .to("direct:mockEndpoint").bindingMode(RestBindingMode.off);
>>         from("direct:mockEndpoint")
>>                 .removeHeaders("CamelHttpPath")
>>
>> .toD("ahc:http://localhost:9000/wait/${headers.timeout}?bridgeEndpoint=true");
>>
>> It appears that the AHC component is creating worker threads to manage
>> outgoing connections to localhost:9000. One for every connection made to
>> localhost:9090.  Am I configuring something wrong, or is this a defect in
>> AHC?
>>
>> Camel version is 2.16-SNAPSHOT
>>
>> Thanks,
>> Les
>>
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Jetty-AHC-route-producing-a-lot-of-threads-tp5771106.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2nd edition: http://www.manning.com/ibsen2



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2nd edition: http://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: Jetty -> AHC route producing a lot of threads

Claus Ibsen-2
And btw jetty also supports producer, so you can use jetty instead of ahc.

On Sat, Sep 5, 2015 at 9:03 AM, Claus Ibsen <[hidden email]> wrote:

> Hi
>
> Ah the wait was part of the content path. Instead of <toD> you can
> build the url as a header with the key Exchange.HTTP_URI, then the ahc
> endpoint can be hardcoded and the header will be used as url instead
> from the endpoint. Then you can reuse the endpoint/producers.
>
>
>
> On Thu, Sep 3, 2015 at 5:21 PM, Claus Ibsen <[hidden email]> wrote:
>> Hi
>>
>> This would be expected as the endpoint is dynamic and therefore each
>> endpoint has their own producer.
>>
>> You should try to use the same endpoint, and if possible if AHC /
>> camel-ahc allows dynamic timeout per request. I am not sure if that is
>> possible / supported.
>>
>>
>>
>> On Mon, Aug 31, 2015 at 11:13 PM, Les Novell
>> <[hidden email]> wrote:
>>> Hello, I am creating a simple route to test asynchronous message passing in
>>> Camel.  When testing the route by creating 1000 simultaneous connections I
>>> see that around 2000 threads are created to handle requests I'm forwarding
>>> on using AHC component.  I am surprised, I thought that Jetty+AHC component
>>> would use non-blocking asynchronous I/O, and a low number of threads.
>>>
>>> Basic Route:
>>>         restConfiguration().component("jetty").host("localhost").port(8081);
>>>         rest("/ud").get("/wait/{timeout}")
>>>                 .to("direct:mockEndpoint").bindingMode(RestBindingMode.off);
>>>         from("direct:mockEndpoint")
>>>                 .removeHeaders("CamelHttpPath")
>>>
>>> .toD("ahc:http://localhost:9000/wait/${headers.timeout}?bridgeEndpoint=true");
>>>
>>> It appears that the AHC component is creating worker threads to manage
>>> outgoing connections to localhost:9000. One for every connection made to
>>> localhost:9090.  Am I configuring something wrong, or is this a defect in
>>> AHC?
>>>
>>> Camel version is 2.16-SNAPSHOT
>>>
>>> Thanks,
>>> Les
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context: http://camel.465427.n5.nabble.com/Jetty-AHC-route-producing-a-lot-of-threads-tp5771106.html
>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
>>
>> --
>> Claus Ibsen
>> -----------------
>> http://davsclaus.com @davsclaus
>> Camel in Action 2nd edition: http://www.manning.com/ibsen2
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2nd edition: http://www.manning.com/ibsen2



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2nd edition: http://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: Jetty -> AHC route producing a lot of threads

Les Novell
Thank you, I'll give this a try without dynamic routes and let you know the results!

Les
Reply | Threaded
Open this post in threaded view
|

Re: Jetty -> AHC route producing a lot of threads

Les Novell
In reply to this post by Claus Ibsen-2
Hi Claus, it works very well with a non-dynamic route. Thank you :-)

Btw, I also tried jetty as a producer compared to ahc.  Jetty component threw a lot of exceptions during the load test, whereas ahc worked flawlessly.

Les