KafkaComponent: Authentication failed

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

KafkaComponent: Authentication failed

Klug Andreas (CI/OSI3)
Hello everybody,

I'm facing an issue with Camel 2.25.0 and Spring Boot 2.2.5 (as well with Camel 2.22.1 with Spring Boot 2.0.7) running on OpenJDK 8u242 within Docker (Azure container instances).

Utilizing the Kafka component I'm producing messages with the following statement
        .recipientList(simple(route.getDestination()+"${header.KafkaTopic}?"+route.getDestinationOptions()))
where route.getDestination() names a bean defined as
        <bean id="someKafkaCluster" class="org.apache.camel.component.kafka.KafkaComponent" primary="true">
            <property name="brokers" value="somehost.azure.confluent.cloud:9092"/>
        </bean>
and route.getDestinationOptions() resolves from a YAML file to
        "saslMechanism=PLAIN&sslProtocol=TLSv1.2&securityProtocol=SASL_SSL&sslEndpointAlgorithm=https&sslEnabledProtocols=TLSv1.2&saslJaasConfig=org.apache.kafka.common.security.plain.PlainLoginModule required username=\"username\" password=\"password\";"

Even though the initial login seems to work ("JmsConsumer[testqueue] [0;39m org.apache.kafka.common.security.authenticator.AbstractLogin - Successfully logged in")
sending the first message results in
2020-03-24 10:54:40,994 [1;31mERROR  [0;39m [35mer-network-thread | producer-1 [0;39m org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (somehost.azure.confluent.cloud/255.255.255.255:9092) failed authentication due to: Authentication failed: Invalid username or password

The producer logs that it got the Jaas config:
org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values:
 ...
 sasl.jaas.config = [hidden]

The strange thing now is that if I remove the sasljaasConfig from the URI and copy it to a Jaas config file, wrapped with "KafkaClient { ... }", referenced via "-Djava.security.auth.login.config=...", everything works fine.

Apparently, user2966021 seems to have at least a very similar issue according to a comment on StackOverflow:
https://stackoverflow.com/questions/50090226/how-to-implement-camel-kafka-producer-and-consumer-which-is-using-kafka-brokers

The password contains characters + and /, but as I'm using double quoted YAML strings, I don't expect this to be an issue.

Anybody having any hints for me?

Thanks in advance!

Best Regards
Andreas


Robert Bosch GmbH | Postfach 30 02 20 | 70442 Stuttgart | GERMANY | www.bosch.com

Registered Office: Stuttgart, Registration Court: Amtsgericht Stuttgart, HRB 14000;
Chairman of the Supervisory Board: Franz Fehrenbach; Managing Directors: Dr. Volkmar Denner,
Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian Fischer, Dr. Stefan Hartung,
Dr. Markus Heyn, Harald Kröger, Christoph Kübel, Rolf Najork, Uwe Raschke, Peter Tyroller

Reply | Threaded
Open this post in threaded view
|

Re: KafkaComponent: Authentication failed

Pasquale Congiusti
Hey Andreas,
Could it be possible you're trying to use TLS over a PLAINTEXT broker?

I can see your broker is running PLAINTEXT default port (9092), while
typically the SSL is run over 9093.

<property name="brokers" value="somehost.azure.confluent.cloud:9092"/>

Cheers,
Pasquale.



On Tue, Mar 24, 2020 at 11:23 AM Klug Andreas (CI/OSI3)
<[hidden email]> wrote:

> Hello everybody,
>
> I'm facing an issue with Camel 2.25.0 and Spring Boot 2.2.5 (as well with
> Camel 2.22.1 with Spring Boot 2.0.7) running on OpenJDK 8u242 within Docker
> (Azure container instances).
>
> Utilizing the Kafka component I'm producing messages with the following
> statement
>
> .recipientList(simple(route.getDestination()+"${header.KafkaTopic}?"+route.getDestinationOptions()))
> where route.getDestination() names a bean defined as
>         <bean id="someKafkaCluster"
> class="org.apache.camel.component.kafka.KafkaComponent" primary="true">
>             <property name="brokers"
> value="somehost.azure.confluent.cloud:9092"/>
>         </bean>
> and route.getDestinationOptions() resolves from a YAML file to
>
> "saslMechanism=PLAIN&sslProtocol=TLSv1.2&securityProtocol=SASL_SSL&sslEndpointAlgorithm=https&sslEnabledProtocols=TLSv1.2&saslJaasConfig=org.apache.kafka.common.security.plain.PlainLoginModule
> required username=\"username\" password=\"password\";"
>
> Even though the initial login seems to work ("JmsConsumer[testqueue]
> [0;39m org.apache.kafka.common.security.authenticator.AbstractLogin -
> Successfully logged in")
> sending the first message results in
> 2020-03-24 10:54:40,994 [1;31mERROR  [0;39m [35mer-network-thread |
> producer-1 [0;39m org.apache.kafka.clients.NetworkClient - [Producer
> clientId=producer-1] Connection to node -1 (somehost.azure.confluent.cloud/
> 255.255.255.255:9092) failed authentication due to: Authentication
> failed: Invalid username or password
>
> The producer logs that it got the Jaas config:
> org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values:
>  ...
>  sasl.jaas.config = [hidden]
>
> The strange thing now is that if I remove the sasljaasConfig from the URI
> and copy it to a Jaas config file, wrapped with "KafkaClient { ... }",
> referenced via "-Djava.security.auth.login.config=...", everything works
> fine.
>
> Apparently, user2966021 seems to have at least a very similar issue
> according to a comment on StackOverflow:
>
> https://stackoverflow.com/questions/50090226/how-to-implement-camel-kafka-producer-and-consumer-which-is-using-kafka-brokers
>
> The password contains characters + and /, but as I'm using double quoted
> YAML strings, I don't expect this to be an issue.
>
> Anybody having any hints for me?
>
> Thanks in advance!
>
> Best Regards
> Andreas
>
>
> Robert Bosch GmbH | Postfach 30 02 20 | 70442 Stuttgart | GERMANY |
> www.bosch.com
>
> Registered Office: Stuttgart, Registration Court: Amtsgericht Stuttgart,
> HRB 14000;
> Chairman of the Supervisory Board: Franz Fehrenbach; Managing Directors:
> Dr. Volkmar Denner,
> Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian
> Fischer, Dr. Stefan Hartung,
> Dr. Markus Heyn, Harald Kröger, Christoph Kübel, Rolf Najork, Uwe Raschke,
> Peter Tyroller
>
>
Reply | Threaded
Open this post in threaded view
|

RE: KafkaComponent: Authentication failed

Klug Andreas (CI/OSI3)
Hi Pasquale,

Thanks for the quick answer - you spotted that well, but the port answers with 0x15 0x03 0x03 0x00 0x02 0x02 0x50 - TLSv1.2. Seems to be a specialty in the Confluent Cloud offering. They're generating the configuration automatically anyhow, thus I assume the port is correct.

I would have been very surprised, too, as everything works fine with providing the Jaas configuration via "-Djava.security.auth.login.config". Do you think there might be an issue with passing the sasl.jaas.config via URI to the Kafka component?

Unfortunately the approach by using the Java system property has a lot of disadvantages, e. g. just able to address one Kafka cluster, not able to use the Spring Azure Key Vault integration, ...

Cheers,
Andreas

-----Original Message-----
From: Pasquale Congiusti <[hidden email]>
Sent: Dienstag, 24. März 2020 16:30
To: [hidden email]
Subject: Re: KafkaComponent: Authentication failed

Hey Andreas,
Could it be possible you're trying to use TLS over a PLAINTEXT broker?

I can see your broker is running PLAINTEXT default port (9092), while typically the SSL is run over 9093.

<property name="brokers" value="somehost.azure.confluent.cloud:9092"/>

Cheers,
Pasquale.



On Tue, Mar 24, 2020 at 11:23 AM Klug Andreas (CI/OSI3) <[hidden email]> wrote:

> Hello everybody,
>
> I'm facing an issue with Camel 2.25.0 and Spring Boot 2.2.5 (as well
> with Camel 2.22.1 with Spring Boot 2.0.7) running on OpenJDK 8u242
> within Docker (Azure container instances).
>
> Utilizing the Kafka component I'm producing messages with the
> following statement
>
> .recipientList(simple(route.getDestination()+"${header.KafkaTopic}?"+r
> oute.getDestinationOptions())) where route.getDestination() names a
> bean defined as
>         <bean id="someKafkaCluster"
> class="org.apache.camel.component.kafka.KafkaComponent" primary="true">
>             <property name="brokers"
> value="somehost.azure.confluent.cloud:9092"/>
>         </bean>
> and route.getDestinationOptions() resolves from a YAML file to
>
> "saslMechanism=PLAIN&sslProtocol=TLSv1.2&securityProtocol=SASL_SSL&ssl
> EndpointAlgorithm=https&sslEnabledProtocols=TLSv1.2&saslJaasConfig=org
> .apache.kafka.common.security.plain.PlainLoginModule
> required username=\"username\" password=\"password\";"
>
> Even though the initial login seems to work ("JmsConsumer[testqueue]
> [0;39m org.apache.kafka.common.security.authenticator.AbstractLogin -
> Successfully logged in") sending the first message results in
> 2020-03-24 10:54:40,994 [1;31mERROR  [0;39m [35mer-network-thread |
> producer-1 [0;39m org.apache.kafka.clients.NetworkClient - [Producer
> clientId=producer-1] Connection to node -1
> (somehost.azure.confluent.cloud/
> 255.255.255.255:9092) failed authentication due to: Authentication
> failed: Invalid username or password
>
> The producer logs that it got the Jaas config:
> org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values:
>  ...
>  sasl.jaas.config = [hidden]
>
> The strange thing now is that if I remove the sasljaasConfig from the
> URI and copy it to a Jaas config file, wrapped with "KafkaClient { ...
> }", referenced via "-Djava.security.auth.login.config=...", everything
> works fine.
>
> Apparently, user2966021 seems to have at least a very similar issue
> according to a comment on StackOverflow:
>
> https://stackoverflow.com/questions/50090226/how-to-implement-camel-ka
> fka-producer-and-consumer-which-is-using-kafka-brokers
>
> The password contains characters + and /, but as I'm using double
> quoted YAML strings, I don't expect this to be an issue.
>
> Anybody having any hints for me?
>
> Thanks in advance!
>
> Best Regards
> Andreas
>
>
> Robert Bosch GmbH | Postfach 30 02 20 | 70442 Stuttgart | GERMANY |
> www.bosch.com
>
> Registered Office: Stuttgart, Registration Court: Amtsgericht
> Stuttgart, HRB 14000; Chairman of the Supervisory Board: Franz
> Fehrenbach; Managing Directors:
> Dr. Volkmar Denner,
> Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian
> Fischer, Dr. Stefan Hartung, Dr. Markus Heyn, Harald Kröger, Christoph
> Kübel, Rolf Najork, Uwe Raschke, Peter Tyroller
>
>
Reply | Threaded
Open this post in threaded view
|

Re: KafkaComponent: Authentication failed

Pasquale Congiusti
Yeah, I've had a look and Confluent uses 9091 for plain and 9092 for ssl,
it's so nice to see that a default is not used as it should...

Then, according to the issue it's quite strange as it is supposed to work
as you're doing. This is the case opened for its development some time ago:
https://issues.apache.org/jira/browse/CAMEL-11682

I am wondering if Confluent Cloud is strict with the way such parameter is
provided. I can see that in the file, it usually have new lines for user
and password parameters. I suggest you to play with the file and try to
provide it plain, as it is configured in the route to see if it fails as
well when you provide it in the same fashion as it is sent through URI.

Cheers,
Pasquale.

On Wed, Mar 25, 2020 at 4:58 PM Klug Andreas (CI/OSI3)
<[hidden email]> wrote:

> Hi Pasquale,
>
> Thanks for the quick answer - you spotted that well, but the port answers
> with 0x15 0x03 0x03 0x00 0x02 0x02 0x50 - TLSv1.2. Seems to be a specialty
> in the Confluent Cloud offering. They're generating the configuration
> automatically anyhow, thus I assume the port is correct.
>
> I would have been very surprised, too, as everything works fine with
> providing the Jaas configuration via "-Djava.security.auth.login.config".
> Do you think there might be an issue with passing the sasl.jaas.config via
> URI to the Kafka component?
>
> Unfortunately the approach by using the Java system property has a lot of
> disadvantages, e. g. just able to address one Kafka cluster, not able to
> use the Spring Azure Key Vault integration, ...
>
> Cheers,
> Andreas
>
> -----Original Message-----
> From: Pasquale Congiusti <[hidden email]>
> Sent: Dienstag, 24. März 2020 16:30
> To: [hidden email]
> Subject: Re: KafkaComponent: Authentication failed
>
> Hey Andreas,
> Could it be possible you're trying to use TLS over a PLAINTEXT broker?
>
> I can see your broker is running PLAINTEXT default port (9092), while
> typically the SSL is run over 9093.
>
> <property name="brokers" value="somehost.azure.confluent.cloud:9092"/>
>
> Cheers,
> Pasquale.
>
>
>
> On Tue, Mar 24, 2020 at 11:23 AM Klug Andreas (CI/OSI3) <
> [hidden email]> wrote:
>
> > Hello everybody,
> >
> > I'm facing an issue with Camel 2.25.0 and Spring Boot 2.2.5 (as well
> > with Camel 2.22.1 with Spring Boot 2.0.7) running on OpenJDK 8u242
> > within Docker (Azure container instances).
> >
> > Utilizing the Kafka component I'm producing messages with the
> > following statement
> >
> > .recipientList(simple(route.getDestination()+"${header.KafkaTopic}?"+r
> > oute.getDestinationOptions())) where route.getDestination() names a
> > bean defined as
> >         <bean id="someKafkaCluster"
> > class="org.apache.camel.component.kafka.KafkaComponent" primary="true">
> >             <property name="brokers"
> > value="somehost.azure.confluent.cloud:9092"/>
> >         </bean>
> > and route.getDestinationOptions() resolves from a YAML file to
> >
> > "saslMechanism=PLAIN&sslProtocol=TLSv1.2&securityProtocol=SASL_SSL&ssl
> > EndpointAlgorithm=https&sslEnabledProtocols=TLSv1.2&saslJaasConfig=org
> > .apache.kafka.common.security.plain.PlainLoginModule
> > required username=\"username\" password=\"password\";"
> >
> > Even though the initial login seems to work ("JmsConsumer[testqueue]
> > [0;39m org.apache.kafka.common.security.authenticator.AbstractLogin -
> > Successfully logged in") sending the first message results in
> > 2020-03-24 10:54:40,994 [1;31mERROR  [0;39m [35mer-network-thread |
> > producer-1 [0;39m org.apache.kafka.clients.NetworkClient - [Producer
> > clientId=producer-1] Connection to node -1
> > (somehost.azure.confluent.cloud/
> > 255.255.255.255:9092) failed authentication due to: Authentication
> > failed: Invalid username or password
> >
> > The producer logs that it got the Jaas config:
> > org.apache.kafka.clients.producer.ProducerConfig - ProducerConfig values:
> >  ...
> >  sasl.jaas.config = [hidden]
> >
> > The strange thing now is that if I remove the sasljaasConfig from the
> > URI and copy it to a Jaas config file, wrapped with "KafkaClient { ...
> > }", referenced via "-Djava.security.auth.login.config=...", everything
> > works fine.
> >
> > Apparently, user2966021 seems to have at least a very similar issue
> > according to a comment on StackOverflow:
> >
> > https://stackoverflow.com/questions/50090226/how-to-implement-camel-ka
> > fka-producer-and-consumer-which-is-using-kafka-brokers
> >
> > The password contains characters + and /, but as I'm using double
> > quoted YAML strings, I don't expect this to be an issue.
> >
> > Anybody having any hints for me?
> >
> > Thanks in advance!
> >
> > Best Regards
> > Andreas
> >
> >
> > Robert Bosch GmbH | Postfach 30 02 20 | 70442 Stuttgart | GERMANY |
> > www.bosch.com
> >
> > Registered Office: Stuttgart, Registration Court: Amtsgericht
> > Stuttgart, HRB 14000; Chairman of the Supervisory Board: Franz
> > Fehrenbach; Managing Directors:
> > Dr. Volkmar Denner,
> > Prof. Dr. Stefan Asenkerschbaumer, Dr. Michael Bolle, Dr. Christian
> > Fischer, Dr. Stefan Hartung, Dr. Markus Heyn, Harald Kröger, Christoph
> > Kübel, Rolf Najork, Uwe Raschke, Peter Tyroller
> >
> >
>