Multiple remote connection to the same host but different users

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Multiple remote connection to the same host but different users

manoj.sahu
Hi there,
I am using camel HTTP component to do an outbound connection to a remote server.  However, depending on my inbound connection I need to communicate to the remote server using different credentials.  The remote server uses basicauth.  What I have noticed that camel tries to reuse the already authenticated connection in the pool even though the subsequent request may need to be reauthenticated using new credential.  What's the best way to configure this in camel.  I was thinking to shutdown the connection after each request even though it has performance penalties.  Can someone please give me some pointers.

Here is my route:

from("direct:inbound")
    .to("bean:requestMessageTranslator")
    .setHeader("CamelHttpMethod", constant("POST"))
    .to("https://host/secure/ssl-gateway?authMethod=Basic&authUsername=username&authPassword=password")
...

The username and password gets replaced dynamically with the correct values for each request.  But camel still tries to reuse an prior authenticated connection, which is already in the pool I suppose.  I have not overridden connection manager or connection manager params etc.  Using the defaults provider by http component.

Thanks in advance for your time!
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

manoj.sahu
I just want to make it clear that I have seen this behavior when I am running integration level tests using JUnit4.  I have not tested yet running this inside a container (Tomcat...)
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

Ashwin Karpe
In reply to this post by manoj.sahu
Hi,

If we are talking about 2 separate connections, can you not set up 2 different endpoints. Obviously, this will require you to set up a different HTTP URL (by adding another leaf maybe) for each of the endpoints on the server.
For example: https://host/secure/ssl-gateway/service1?authMethod=Basic&authUsername=username&authPassword=password"  and https://host/secure/ssl-gateway/service2?authMethod=Basic&authUsername=username&authPassword=password"

This should enable you to create a secure connection with different credentials and then use that to communicate with the server.. Of course, you may need to modify your route or add a separate route to communicate with the remote server and enable the other connection to be used.

I have not personally tried this before, however, I do not see why it would not work!!!

Cheers,

Ashwin...

 
---------------------------------------------------------
Ashwin Karpe
Apache Camel Committer & Sr Principal Consultant
Red Hat
http://www.redhat.com

Blog:http://opensourceknowledge.blogspot.com
---------------------------------------------------------
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

manoj.sahu
Hi Ashiwin,
Thanks for your response.  Unfortunately I cannot do that.  First of all the remote server is from a Service Provider.  Their URL is fixed. Only connecting users need to be different.  It's the same service offered at a fixed endpoint.

The re-authentication is needed for many different users. Hence I cannot statically defined in the routes. It can be n.

I do understand that for connection pools it's probably not the ideal scenario.  Can I just close/shutdown the connection after each request in the worst case.  Do I have to write custom code for that.  I did not see any parameter that I could configure on the service route URL.  
Thanks
   
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

Claus Ibsen-2
When you need a dynamic uri for an endpoint you should use the
recipient list EIP pattern.
http://camel.apache.org/recipient-list.html


On Mon, Apr 18, 2011 at 4:44 AM, manoj.sahu <[hidden email]> wrote:

> Hi Ashiwin,
> Thanks for your response.  Unfortunately I cannot do that.  First of all the
> remote server is from a Service Provider.  Their URL is fixed. Only
> connecting users need to be different.  It's the same service offered at a
> fixed endpoint.
>
> The re-authentication is needed for many different users. Hence I cannot
> statically defined in the routes. It can be n.
>
> I do understand that for connection pools it's probably not the ideal
> scenario.  Can I just close/shutdown the connection after each request in
> the worst case.  Do I have to write custom code for that.  I did not see any
> parameter that I could configure on the service route URL.
> Thanks
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Multiple-remote-connection-to-the-same-host-but-different-users-tp4309456p4309883.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>



--
Claus Ibsen
-----------------
FuseSource
Email: [hidden email]
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

manoj.sahu
This post was updated on .
Hi Claus,
That's exactly what I use. I use recipient list pattern. Basically I have many providers that I need to connect and get response.  

Here is the code in the route :
from("direct:providers") .recipientList(header(Constants.PROVIDERS_RECIPIENT_LIST).tokenize(",")).stopOnException().ignoreInvalidEndpoints().aggregationStrategy(new ProviderResultsAggregationStrategy())
;


For each individual provider , I compute the end point based the user name/pwd etc...as mentioned above then call recipientlist again, but this time just for one provider.

from("direct:provider1")
    .setHeader("CamelHttpMethod", constant("POST"))
    .recipientList(header(Constants.PROVIDER_ENDPOINT_URL))
    .convertBodyTo(...)
...


That's working as expected.
 

The problem I am having is on the actual transport to an individual provider using HTTP component. User A is using an authenticated connection of User B  to the same endpoint.  It's a backend webserivce.  I am okay to reestablish connection on every request. I am not sure how to shutdown the connection after each request is complete.


Thanks

Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

manoj.sahu
After further analysis, it appears to me as a bug in apache camel.  

The bug remains in ProducerCache.doGetProducer() where a producer is instantiated or retrieved from the cache. At the time of instantiating producer everything is initialized properly.

Basically ProducerCache caches the Producers.  However when it retrieves the producer from the cache it ignores the fact that the producer may have stale credential.

In other words following endpoints will potentially use the same  producer:
https://host/ssl-gateway?authMethod=Basic&authUsername=user1&authPassword=pwd1
https://host/ssl-gateway?authMethod=Basic&authUsername=user2&authPassword=pwd2

I have patched the ProducerCache to fix this problem locally. Here is the fix I have put.  This gets executed when the producer is retrieved from the cache.

    protected synchronized Producer doGetProducer(Endpoint endpoint, boolean pooled) {
        String key = endpoint.getEndpointUri();
        Producer answer = producers.get(key);
        if (pooled && answer == null) {
            // try acquire from connection pool
            answer = pool.acquire(endpoint);
        }

        if (answer == null) {
            // create a new producer
            try {
                answer = endpoint.createProducer();
                // must then start service so producer is ready to be used
                ServiceHelper.startService(answer);
            } catch (Exception e) {
                throw new FailedToCreateProducerException(endpoint, e);
            }

            // add producer to cache or pool if applicable
            if (pooled && answer instanceof ServicePoolAware) {
                if (LOG.isDebugEnabled()) {
                    LOG.debug("Adding to producer service pool with key: " + endpoint + " for producer: " + answer);
                }
                answer = pool.addAndAcquire(endpoint, answer);
            } else if (answer.isSingleton()) {
                if (LOG.isDebugEnabled()) {
                    LOG.debug("Adding to producer cache with key: " + endpoint + " for producer: " + answer);
                }
                producers.put(key, answer);
            }
        } else {
// PATCH BEGIN
        if (endpoint instanceof HttpEndpoint && answer instanceof HttpProducer){
        if (((HttpEndpoint)endpoint).getHttpClientConfigurer() != null){
            ((HttpEndpoint)endpoint).getHttpClientConfigurer().configureHttpClient(((HttpProducer)answer).getHttpClient());  
            LOG.debug("Patched ProducerCache : " + " Refreshing the authentication credential for the cached HTTP producer");
        }
        }
        }
// PATCH END
        return answer;
    }

 
I am not happy about the patch due to the coupling of HttpEndpoint and HttpProducer classes with the ProducerCache.  But I could not get it to work otherwise.   I tried putting the fix in HttpEndpoint but it did not work consistently.  

Claus/Ashwin, should this be treated as a bug? Can this be addressed in the future version of the camel.  I am using version 2.4.0. However, I checked the code from the latest version and it seems that the problem exists.  I am sure that there is a better place other than ProducerCache where the fix can be addressed.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

Claus Ibsen-2
Hi

Thanks for reporting back.

Try instead in the camel-http component to patch by doing this:

Add
    implements ServicePoolAware
to HttpProducer

And then report back if that fixes your issue.


On Tue, May 3, 2011 at 4:27 AM, manoj.sahu <[hidden email]> wrote:

> After further analysis, it appears to me as a bug in apache camel.
>
> The bug remains in ProducerCache.doGetProducer() where a producer is
> instantiated or retrieved from the cache. At the time of instantiating
> producer
>
> Basically ProducerCache caches the Producers.  However when it retrieves the
> producer from the cache it ignores the fact that the producer may have stale
> credential.
>
> In other words following endpoints will potentially use the same  producer:
> https://host/ssl-gateway?authMethod=Basic&authUsername=user1&authPassword=pwd1
> https://host/ssl-gateway?authMethod=Basic&authUsername=user2&authPassword=pwd2
>
> I have patched the ProducerCache to fix this problem locally. Here is the
> fix I have put.  This gets executed when the producer is retrieved from the
> cache.
>
>    protected synchronized Producer doGetProducer(Endpoint endpoint, boolean
> pooled) {
>        String key = endpoint.getEndpointUri();
>        Producer answer = producers.get(key);
>        if (pooled && answer == null) {
>            // try acquire from connection pool
>            answer = pool.acquire(endpoint);
>        }
>
>        if (answer == null) {
>            // create a new producer
>            try {
>                answer = endpoint.createProducer();
>                // must then start service so producer is ready to be used
>                ServiceHelper.startService(answer);
>            } catch (Exception e) {
>                throw new FailedToCreateProducerException(endpoint, e);
>            }
>
>            // add producer to cache or pool if applicable
>            if (pooled && answer instanceof ServicePoolAware) {
>                if (LOG.isDebugEnabled()) {
>                    LOG.debug("Adding to producer service pool with key: " +
> endpoint + " for producer: " + answer);
>                }
>                answer = pool.addAndAcquire(endpoint, answer);
>            } else if (answer.isSingleton()) {
>                if (LOG.isDebugEnabled()) {
>                    LOG.debug("Adding to producer cache with key: " +
> endpoint + " for producer: " + answer);
>                }
>                producers.put(key, answer);
>            }
>        } else {
> // PATCH BEGIN
>                if (endpoint instanceof HttpEndpoint && answer instanceof
> HttpProducer){
>                        if (((HttpEndpoint)endpoint).getHttpClientConfigurer() != null){
>
> ((HttpEndpoint)endpoint).getHttpClientConfigurer().configureHttpClient(((HttpProducer)answer).getHttpClient());
>                        LOG.debug("Patched ProducerCache : " + " Refreshing the
> authentication credential for the cached HTTP producer");
>                        }
>                }
>        }
> // PATCH END
>        return answer;
>    }
>
>
> I am not happy about the patch due to the coupling of HttpEndpoint and
> HttpProducer classes with the ProducerCache.  But I could not get it to work
> otherwise.   I tried putting the fix in HttpEndpoint but it did not work
> consistently.
>
> Claus/Ashwin, should this be treated as a bug? Can this be addressed in the
> future version of the camel.  I am using version 2.4.0. However, I checked
> the code from the latest version and it seems that the problem exists.  I am
> sure that there is a better place other than ProducerCache where the fix can
> be addressed.
>
> Thanks
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Multiple-remote-connection-to-the-same-host-but-different-users-tp4309456p4366212.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>



--
Claus Ibsen
-----------------
FuseSource
Email: [hidden email]
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

Claus Ibsen-2
Any update on this? Did you try my suggestion?


On Tue, May 3, 2011 at 8:55 AM, Claus Ibsen <[hidden email]> wrote:

> Hi
>
> Thanks for reporting back.
>
> Try instead in the camel-http component to patch by doing this:
>
> Add
>    implements ServicePoolAware
> to HttpProducer
>
> And then report back if that fixes your issue.
>
>
> On Tue, May 3, 2011 at 4:27 AM, manoj.sahu <[hidden email]> wrote:
>> After further analysis, it appears to me as a bug in apache camel.
>>
>> The bug remains in ProducerCache.doGetProducer() where a producer is
>> instantiated or retrieved from the cache. At the time of instantiating
>> producer
>>
>> Basically ProducerCache caches the Producers.  However when it retrieves the
>> producer from the cache it ignores the fact that the producer may have stale
>> credential.
>>
>> In other words following endpoints will potentially use the same  producer:
>> https://host/ssl-gateway?authMethod=Basic&authUsername=user1&authPassword=pwd1
>> https://host/ssl-gateway?authMethod=Basic&authUsername=user2&authPassword=pwd2
>>
>> I have patched the ProducerCache to fix this problem locally. Here is the
>> fix I have put.  This gets executed when the producer is retrieved from the
>> cache.
>>
>>    protected synchronized Producer doGetProducer(Endpoint endpoint, boolean
>> pooled) {
>>        String key = endpoint.getEndpointUri();
>>        Producer answer = producers.get(key);
>>        if (pooled && answer == null) {
>>            // try acquire from connection pool
>>            answer = pool.acquire(endpoint);
>>        }
>>
>>        if (answer == null) {
>>            // create a new producer
>>            try {
>>                answer = endpoint.createProducer();
>>                // must then start service so producer is ready to be used
>>                ServiceHelper.startService(answer);
>>            } catch (Exception e) {
>>                throw new FailedToCreateProducerException(endpoint, e);
>>            }
>>
>>            // add producer to cache or pool if applicable
>>            if (pooled && answer instanceof ServicePoolAware) {
>>                if (LOG.isDebugEnabled()) {
>>                    LOG.debug("Adding to producer service pool with key: " +
>> endpoint + " for producer: " + answer);
>>                }
>>                answer = pool.addAndAcquire(endpoint, answer);
>>            } else if (answer.isSingleton()) {
>>                if (LOG.isDebugEnabled()) {
>>                    LOG.debug("Adding to producer cache with key: " +
>> endpoint + " for producer: " + answer);
>>                }
>>                producers.put(key, answer);
>>            }
>>        } else {
>> // PATCH BEGIN
>>                if (endpoint instanceof HttpEndpoint && answer instanceof
>> HttpProducer){
>>                        if (((HttpEndpoint)endpoint).getHttpClientConfigurer() != null){
>>
>> ((HttpEndpoint)endpoint).getHttpClientConfigurer().configureHttpClient(((HttpProducer)answer).getHttpClient());
>>                        LOG.debug("Patched ProducerCache : " + " Refreshing the
>> authentication credential for the cached HTTP producer");
>>                        }
>>                }
>>        }
>> // PATCH END
>>        return answer;
>>    }
>>
>>
>> I am not happy about the patch due to the coupling of HttpEndpoint and
>> HttpProducer classes with the ProducerCache.  But I could not get it to work
>> otherwise.   I tried putting the fix in HttpEndpoint but it did not work
>> consistently.
>>
>> Claus/Ashwin, should this be treated as a bug? Can this be addressed in the
>> future version of the camel.  I am using version 2.4.0. However, I checked
>> the code from the latest version and it seems that the problem exists.  I am
>> sure that there is a better place other than ProducerCache where the fix can
>> be addressed.
>>
>> Thanks
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble.com/Multiple-remote-connection-to-the-same-host-but-different-users-tp4309456p4366212.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>
>
>
> --
> Claus Ibsen
> -----------------
> FuseSource
> Email: [hidden email]
> Web: http://fusesource.com
> CamelOne 2011: http://fusesource.com/camelone2011/
> Twitter: davsclaus
> Blog: http://davsclaus.blogspot.com/
> Author of Camel in Action: http://www.manning.com/ibsen/
>



--
Claus Ibsen
-----------------
FuseSource
Email: [hidden email]
Web: http://fusesource.com
CamelOne 2011: http://fusesource.com/camelone2011/
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

manoj.sahu
Claus,
My apologies for the delayed response.  I got pulled out of that project.  Anyway, I did the following

public class HttpProducer extends DefaultProducer implements ServicePoolAware {
...

I hope that's all you wanted me to test.

I reran my tests and it failed with the same error as before.  The ugly fix in ProducerCache class is working.

Please let me know if you would like me to try anything else.
Thanks

Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

Claus Ibsen-2
Hi

Thanks. I have created a ticket to track this
https://issues.apache.org/jira/browse/CAMEL-4096

On Tue, May 24, 2011 at 6:43 AM, manoj.sahu <[hidden email]> wrote:

> Claus,
> My apologies for the delayed response.  I got pulled out of that project.
> Anyway, I did the following
>
> public class HttpProducer extends DefaultProducer implements
> ServicePoolAware {
> ...
>
> I hope that's all you wanted me to test.
>
> I reran my tests and it failed with the same error as before.  The ugly fix
> in ProducerCache class is working.
>
> Please let me know if you would like me to try anything else.
> Thanks
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Multiple-remote-connection-to-the-same-host-but-different-users-tp4309456p4420999.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>



--
Claus Ibsen
-----------------
FuseSource
Email: [hidden email]
Web: http://fusesource.com
Twitter: davsclaus, fusenews
Blog: http://davsclaus.blogspot.com/
Author of Camel in Action: http://www.manning.com/ibsen/
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

boday
In reply to this post by manoj.sahu
I'm looking into this, but I don't see the issue in the code (2.8-SNAPSHOT).  The producer cache uses the endpoint URI as the key and this is unique because the user/password are part of the URI...

I wrote a simple unit test and it is instantiating a different producer for each unique endpoint URI, etc.  Can you provide a unit test that fails or perhaps more details about the issue?

manoj.sahu wrote
After further analysis, it appears to me as a bug in apache camel.  

The bug remains in ProducerCache.doGetProducer() where a producer is instantiated or retrieved from the cache. At the time of instantiating producer everything is initialized properly.

Basically ProducerCache caches the Producers.  However when it retrieves the producer from the cache it ignores the fact that the producer may have stale credential.

In other words following endpoints will potentially use the same  producer:
https://host/ssl-gateway?authMethod=Basic&authUsername=user1&authPassword=pwd1
https://host/ssl-gateway?authMethod=Basic&authUsername=user2&authPassword=pwd2

I have patched the ProducerCache to fix this problem locally. Here is the fix I have put.  This gets executed when the producer is retrieved from the cache.

    protected synchronized Producer doGetProducer(Endpoint endpoint, boolean pooled) {
        String key = endpoint.getEndpointUri();
        Producer answer = producers.get(key);
        if (pooled && answer == null) {
            // try acquire from connection pool
            answer = pool.acquire(endpoint);
        }

        if (answer == null) {
            // create a new producer
            try {
                answer = endpoint.createProducer();
                // must then start service so producer is ready to be used
                ServiceHelper.startService(answer);
            } catch (Exception e) {
                throw new FailedToCreateProducerException(endpoint, e);
            }

            // add producer to cache or pool if applicable
            if (pooled && answer instanceof ServicePoolAware) {
                if (LOG.isDebugEnabled()) {
                    LOG.debug("Adding to producer service pool with key: " + endpoint + " for producer: " + answer);
                }
                answer = pool.addAndAcquire(endpoint, answer);
            } else if (answer.isSingleton()) {
                if (LOG.isDebugEnabled()) {
                    LOG.debug("Adding to producer cache with key: " + endpoint + " for producer: " + answer);
                }
                producers.put(key, answer);
            }
        } else {
// PATCH BEGIN
        if (endpoint instanceof HttpEndpoint && answer instanceof HttpProducer){
        if (((HttpEndpoint)endpoint).getHttpClientConfigurer() != null){
            ((HttpEndpoint)endpoint).getHttpClientConfigurer().configureHttpClient(((HttpProducer)answer).getHttpClient());  
            LOG.debug("Patched ProducerCache : " + " Refreshing the authentication credential for the cached HTTP producer");
        }
        }
        }
// PATCH END
        return answer;
    }

 
I am not happy about the patch due to the coupling of HttpEndpoint and HttpProducer classes with the ProducerCache.  But I could not get it to work otherwise.   I tried putting the fix in HttpEndpoint but it did not work consistently.  

Claus/Ashwin, should this be treated as a bug? Can this be addressed in the future version of the camel.  I am using version 2.4.0. However, I checked the code from the latest version and it seems that the problem exists.  I am sure that there is a better place other than ProducerCache where the fix can be addressed.

Thanks
Ben O'Day
IT Consultant -http://consulting-notes.com
Reply | Threaded
Open this post in threaded view
|

Re: Multiple remote connection to the same host but different users

boday
this was fixed in version 2.5...see CAMEL-2945

boday wrote
I'm looking into this, but I don't see the issue in the code (2.8-SNAPSHOT).  The producer cache uses the endpoint URI as the key and this is unique because the user/password are part of the URI...

I wrote a simple unit test and it is instantiating a different producer for each unique endpoint URI, etc.  Can you provide a unit test that fails or perhaps more details about the issue?
Ben O'Day
IT Consultant -http://consulting-notes.com