DefaultTimeoutMap vs Caffeine

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

DefaultTimeoutMap vs Caffeine

drekbour
TimeoutMap (primarily used by AggregateProcessor with completionTimeout
) is not well suited to large aggregation maps as it does an O(n) scan
across all registered timeouts every 1000ms. I intended to take a look
at improving this but it's also not great example of open-close either,
all users must extend DefaultTimeoutMap to get any use at all meaning
it's difficult to substitute an experimental implementation.  The
further the thought process goes the more it turns into "why not use
Caffeine"

* Already in use in camel-core (LRUCache etc)

* Synchronous eviction listeners -
https://github.com/ben-manes/caffeine/wiki/Writer

* Supports O(1) per-entry timeouts -
https://github.com/ben-manes/caffeine/issues/114

* Has active eviction/refresh capabilities which can (probably) be used
to trigger EIP completed-by-timeout

Thoughts?

Marc


Reply | Threaded
Open this post in threaded view
|

Re: DefaultTimeoutMap vs Caffeine

Claus Ibsen-2
Hi Marc

Just curious how large aggregation maps do you have?

You are surely welcome to work on an experiment, and log a JIRA ticket.
Mind that we are working on Camel 3 development, and such a change
would need to be applicable there, and this change is potentially only
being a candidate for 3.x, as 2.x development is slowing down, and 2.x
has been using the current way always.


On Sat, Feb 9, 2019 at 12:01 PM Marc Carter <[hidden email]> wrote:

>
> TimeoutMap (primarily used by AggregateProcessor with completionTimeout
> ) is not well suited to large aggregation maps as it does an O(n) scan
> across all registered timeouts every 1000ms. I intended to take a look
> at improving this but it's also not great example of open-close either,
> all users must extend DefaultTimeoutMap to get any use at all meaning
> it's difficult to substitute an experimental implementation.  The
> further the thought process goes the more it turns into "why not use
> Caffeine"
>
> * Already in use in camel-core (LRUCache etc)
>
> * Synchronous eviction listeners -
> https://github.com/ben-manes/caffeine/wiki/Writer
>
> * Supports O(1) per-entry timeouts -
> https://github.com/ben-manes/caffeine/issues/114
>
> * Has active eviction/refresh capabilities which can (probably) be used
> to trigger EIP completed-by-timeout
>
> Thoughts?
>
> Marc
>
>


--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

R: DefaultTimeoutMap vs Caffeine

Andrea Cosentino-2
In reply to this post by drekbour
We have also a Caffeine component btw

Inviato da Yahoo Mail su Android
 
  Il sab, 9 feb, 2019 alle 12:01, Marc Carter<[hidden email]> ha scritto:   TimeoutMap (primarily used by AggregateProcessor with completionTimeout
) is not well suited to large aggregation maps as it does an O(n) scan
across all registered timeouts every 1000ms. I intended to take a look
at improving this but it's also not great example of open-close either,
all users must extend DefaultTimeoutMap to get any use at all meaning
it's difficult to substitute an experimental implementation.  The
further the thought process goes the more it turns into "why not use
Caffeine"

* Already in use in camel-core (LRUCache etc)

* Synchronous eviction listeners -
https://github.com/ben-manes/caffeine/wiki/Writer

* Supports O(1) per-entry timeouts -
https://github.com/ben-manes/caffeine/issues/114

* Has active eviction/refresh capabilities which can (probably) be used
to trigger EIP completed-by-timeout

Thoughts?

Marc