ConcurrentModificationException when using recipientList with Strings in parallel

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ConcurrentModificationException when using recipientList with Strings in parallel

Zohhak
Dear Camel Devs,

I encountered a ConcurrentModificationException in Camel:

java.util.ConcurrentModificationException: null
                at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1134)
                at org.apache.camel.util.Scanner.cachePattern(Scanner.java:305)
                at org.apache.camel.util.Scanner.<init>(Scanner.java:82)
                at org.apache.camel.support.ObjectHelper.lambda$createIterable$3(ObjectHelper.java:593)
                at org.apache.camel.support.ObjectHelper$$Lambda$1118.0000000000000000.iterator(Unknown Source)
                at org.apache.camel.support.ObjectHelper.createIterator(ObjectHelper.java:435)
                at org.apache.camel.support.ObjectHelper.createIterator(ObjectHelper.java:412)
                at org.apache.camel.processor.RecipientList.sendToRecipientList(RecipientList.java:137)
                at org.apache.camel.processor.RecipientList.process(RecipientList.java:125)
                at org.apache.camel.processor.Pipeline.doProcess(Pipeline.java:103)
                at org.apache.camel.processor.Pipeline.lambda$null$2(Pipeline.java:104)
                at org.apache.camel.processor.Pipeline$$Lambda$1096.0000000000000000.run(Unknown Source)
                at org.apache.camel.impl.engine.DefaultReactiveExecutor$3.run(DefaultReactiveExecutor.java:116)
                at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:185)
                at org.apache.camel.impl.engine.DefaultReactiveExecutor.schedule(DefaultReactiveExecutor.java:67)
                at org.apache.camel.spi.ReactiveExecutor.schedule(ReactiveExecutor.java:32)
                at org.apache.camel.processor.MulticastProcessor.lambda$schedule$1(MulticastProcessor.java:249)
                at org.apache.camel.processor.MulticastProcessor$$Lambda$1099.0000000000000000.run(Unknown Source)
                at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
                at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
                at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
                at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
                at java.base/java.lang.Thread.run(Thread.java:831)

Basic Setup: Java 11, Camel3-RC3
My simplified route is as follows. The input is a list of objects, these are split, and based on the object the recipientList is different. When there are two recipients in the String provided by „getRecipients“, the named exception is thrown. Sadly not consistently.

  from("seda:input")
      .split(method(TestClass.class, "split"), AggregationStrategies.groupedBody())
        .parallelProcessing()
        .recipientList(method(TestClass.class, "getRecipients"))
          .parallelProcessing()

It looks like the Scanner uses an internal map of compiled Patterns, but this map (LinkedHashMap) is not threadsafe. For now i changed the returnvalue of „getRecipients“ to a Set, but maybe someone else needs this. Shall i provide a patch for this, and if yes, do you know a threadsafe variant of LinkedHashMap, that provides the the circular behaviour, or should i just synchronize  the „cachePattern“ function?

Thanks for any responses.

Best regards,

Christian Mohr
Reply | Threaded
Open this post in threaded view
|

Re: ConcurrentModificationException when using recipientList with Strings in parallel

Claus Ibsen-2
Hi Christian

Yeah its a bug in Camel 3, you are surely welcome to log a JIRA ticket
and work on a patch.
I can see we (gnodet) introduced a custom fast scanner implementation,
but its not thread safe.

We could add synchronization block to that compute if absent. And for
those constant patterns WHITE_SPACE, FIND_ANY etc we can make them
static
as java.util.Pattern is thread-safe and create them once in a class
static block.



On Sat, Nov 2, 2019 at 12:32 PM Christian Mohr <[hidden email]> wrote:

>
> Dear Camel Devs,
>
> I encountered a ConcurrentModificationException in Camel:
>
> java.util.ConcurrentModificationException: null
>                 at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1134)
>                 at org.apache.camel.util.Scanner.cachePattern(Scanner.java:305)
>                 at org.apache.camel.util.Scanner.<init>(Scanner.java:82)
>                 at org.apache.camel.support.ObjectHelper.lambda$createIterable$3(ObjectHelper.java:593)
>                 at org.apache.camel.support.ObjectHelper$$Lambda$1118.0000000000000000.iterator(Unknown Source)
>                 at org.apache.camel.support.ObjectHelper.createIterator(ObjectHelper.java:435)
>                 at org.apache.camel.support.ObjectHelper.createIterator(ObjectHelper.java:412)
>                 at org.apache.camel.processor.RecipientList.sendToRecipientList(RecipientList.java:137)
>                 at org.apache.camel.processor.RecipientList.process(RecipientList.java:125)
>                 at org.apache.camel.processor.Pipeline.doProcess(Pipeline.java:103)
>                 at org.apache.camel.processor.Pipeline.lambda$null$2(Pipeline.java:104)
>                 at org.apache.camel.processor.Pipeline$$Lambda$1096.0000000000000000.run(Unknown Source)
>                 at org.apache.camel.impl.engine.DefaultReactiveExecutor$3.run(DefaultReactiveExecutor.java:116)
>                 at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:185)
>                 at org.apache.camel.impl.engine.DefaultReactiveExecutor.schedule(DefaultReactiveExecutor.java:67)
>                 at org.apache.camel.spi.ReactiveExecutor.schedule(ReactiveExecutor.java:32)
>                 at org.apache.camel.processor.MulticastProcessor.lambda$schedule$1(MulticastProcessor.java:249)
>                 at org.apache.camel.processor.MulticastProcessor$$Lambda$1099.0000000000000000.run(Unknown Source)
>                 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>                 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>                 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>                 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>                 at java.base/java.lang.Thread.run(Thread.java:831)
>
> Basic Setup: Java 11, Camel3-RC3
> My simplified route is as follows. The input is a list of objects, these are split, and based on the object the recipientList is different. When there are two recipients in the String provided by „getRecipients“, the named exception is thrown. Sadly not consistently.
>
>   from("seda:input")
>       .split(method(TestClass.class, "split"), AggregationStrategies.groupedBody())
>         .parallelProcessing()
>         .recipientList(method(TestClass.class, "getRecipients"))
>           .parallelProcessing()
>
> It looks like the Scanner uses an internal map of compiled Patterns, but this map (LinkedHashMap) is not threadsafe. For now i changed the returnvalue of „getRecipients“ to a Set, but maybe someone else needs this. Shall i provide a patch for this, and if yes, do you know a threadsafe variant of LinkedHashMap, that provides the the circular behaviour, or should i just synchronize  the „cachePattern“ function?
>
> Thanks for any responses.
>
> Best regards,
>
> Christian Mohr



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2