JVM runs into heap memory issue, upon switching to SSL kafka cluster using apache camel

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

JVM runs into heap memory issue, upon switching to SSL kafka cluster using apache camel

akash b
Hello,

I am using apache camel Spring DSL to connect to kafka topics. Everything
works fine while using non SSL kafka cluster but the moment I switch it to
SSL cluster, JVM heap will keep on increasing and eventually kill the
service in some time. However I am able to post and consume messages
successfully but due to heap issue, the service will shutdown frequently. I
tried by increasing the memory from 1gb to 2gb and 3gb, still the process
will consume all the available space.

Requesting for the help to resolve this.

Further Details:
Apache Camel Version: 2.19.0,
Kafka Client: 0.10.1.1,
Spring Version: 4.3.7.RELEASE,
Connecting through spring DSL:
<camel:to uri =
"kafka:[[kafka.zookeeper]]?securityProtocol=SSL&sslProtocol=TLSv1.2&topic=[[kafka.topic]]&groupId=[[
kafka.consumer.group.id]]&brokers=[[kafka.brokers]]&sslTruststoreLocation=path/truststore.jks&sslTruststorePassword=password&sslKeystoreLocation=path/keystore.jks&sslKeystorePassword=password&sslKeyPassword=password"
/>


*Error I am getting from logs.*

**Uncaught exception in kafka-producer-network-thread | producer-8:

java.lang.OutOfMemoryError: Java heap space

        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
~[na:1.8.0_25]

        at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_25]

        at
org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:93)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:154)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:135)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:343)
~[kafka-clients-0.10.1.1.jar:na]

        at org.apache.kafka.common.network.Selector.poll(Selector.java:291)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:236)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:135)
~[kafka-clients-0.10.1.1.jar:na]

        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_25]

2020-03-31 07:12:00,013--INFO Class:: org.apache.camel.util.CamelLogger
Method :: log :159
Reply | Threaded
Open this post in threaded view
|

Re: JVM runs into heap memory issue, upon switching to SSL kafka cluster using apache camel

Andrea Cosentino-2
Please use a newer version, 2.19.0 is really old and we don't release that branch anymore.

--
Andrea Cosentino 
----------------------------------
Apache Camel PMC Chair
Apache Karaf Committer
Apache Servicemix PMC Member
Email: [hidden email]
Twitter: @oscerd2
Github: oscerd






On Wednesday, April 1, 2020, 09:11:14 AM GMT+2, akash b <[hidden email]> wrote:





Hello,

I am using apache camel Spring DSL to connect to kafka topics. Everything
works fine while using non SSL kafka cluster but the moment I switch it to
SSL cluster, JVM heap will keep on increasing and eventually kill the
service in some time. However I am able to post and consume messages
successfully but due to heap issue, the service will shutdown frequently. I
tried by increasing the memory from 1gb to 2gb and 3gb, still the process
will consume all the available space.

Requesting for the help to resolve this.

Further Details:
Apache Camel Version: 2.19.0,
Kafka Client: 0.10.1.1,
Spring Version: 4.3.7.RELEASE,
Connecting through spring DSL:
<camel:to uri =
"kafka:[[kafka.zookeeper]]?securityProtocol=SSL&sslProtocol=TLSv1.2&topic=[[kafka.topic]]&groupId=[[
kafka.consumer.group.id]]&brokers=[[kafka.brokers]]&sslTruststoreLocation=path/truststore.jks&sslTruststorePassword=password&sslKeystoreLocation=path/keystore.jks&sslKeystorePassword=password&sslKeyPassword=password"
/>


*Error I am getting from logs.*

**Uncaught exception in kafka-producer-network-thread | producer-8:

java.lang.OutOfMemoryError: Java heap space

        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
~[na:1.8.0_25]

        at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) ~[na:1.8.0_25]

        at
org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:93)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:154)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:135)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:343)
~[kafka-clients-0.10.1.1.jar:na]

        at org.apache.kafka.common.network.Selector.poll(Selector.java:291)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:260)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:236)
~[kafka-clients-0.10.1.1.jar:na]

        at
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:135)
~[kafka-clients-0.10.1.1.jar:na]

        at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_25]

2020-03-31 07:12:00,013--INFO Class:: org.apache.camel.util.CamelLogger
Method :: log :159