Camel Tracer impedes performance.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Camel Tracer impedes performance.

Naira & Kobo
Hi,

I noticed that extending the camel tracer to log exchange data into the database impedes performance on the ESB.

I followed the example in the camel samples, where I defined a direct endpoint

<bean id="camelTracer" class="org.apache.camel.processor.interceptor.Tracer">
            <property name="destination" ref="traceEndpoint"/>
            <property name="logLevel" value="OFF"/>
        </bean>       

<osgi:camelContext xmlns="http://camel.apache.org/schema/spring" trace="true">   
  <endpoint id="traceEndpoint" uri="direct:traceRoute"/>
 
        <route>
              <from ....../>
               ...............
              <to ............./>
        </route>
   
        <route>
    <from uri="direct:traceRoute"/>
      <bean ref="tracer" method="Log"/>
    </route> 
  </osgi:camelContext>

What I did was to create a custom bean that logs this tracemessage to database. I assume the process of calling this route is off the main route (meaning I assume its like an interceptor and should not in any way slow down the time it takes to travel from one node to another). But it seems it does in reality. As whenever I turn of the custom trace, it takes considerable  less time to complete a transaction and when its on, it takes more time.

A background to this problem that might be helpful is the fact that, exchange message body type is a bean (and not string or xml). I noticed each time its going from one node to another, it implicitly tries to convert from bean to String, before handling the message to the direct endpoint. (May be its just my imagination)

It is also noteworthy that the main route uses vm and the preferred endpoint uri while the log interfaces uses the direct uri.

Is there another way, I can efficiently do this?

kr.
Reply | Threaded
Open this post in threaded view
|

Re: Camel Tracer impedes performance.

Jim Talbut-2
On 17/02/2011 10:01, Naira & Kobo wrote:

> Hi,
>
> I noticed that extending the camel tracer to log exchange data into the
> database impedes performance on the ESB.
>
> I followed the example in the camel samples, where I defined a direct
> endpoint
>
> <bean id="camelTracer"
> class="org.apache.camel.processor.interceptor.Tracer">
>     <property name="destination" ref="traceEndpoint"/>
>     <property name="logLevel" value="OFF"/>
> </bean>
>
> <osgi:camelContext xmlns="http://camel.apache.org/schema/spring"
> trace="true">
>     <endpoint id="traceEndpoint" uri="direct:traceRoute"/>
>    
>          <route>
>                <from ....../>
>                 ...............
>                <to ............./>
>          </route>
>
> <route>
>       <from uri="direct:traceRoute"/>
>         <bean ref="tracer" method="Log"/>
>      </route>
>    </osgi:camelContext>
>
> What I did was to create a custom bean that logs this tracemessage to
> database. I assume the process of calling this route is off the main route
> (meaning I assume its like an interceptor and should not in any way slow
> down the time it takes to travel from one node to another). But it seems it
> does in reality. As whenever I turn of the custom trace, it takes
> considerable  less time to complete a transaction and when its on, it takes
> more time.
>
> A background to this problem that might be helpful is the fact that,
> exchange message body type is a bean (and not string or xml). I noticed each
> time its going from one node to another, it implicitly tries to convert from
> bean to String, before handling the message to the direct endpoint. (May be
> its just my imagination)
>
> It is also noteworthy that the main route uses vm and the preferred endpoint
> uri while the log interfaces uses the direct uri.
>
> Is there another way, I can efficiently do this?
>
> kr.
>    
Of course tracing impedes performance, it's got more to do :)

However the default tracing is particularly inefficient (because it's
safe and copies the Exchange) and there are more efficient options if
you don't mind taking more responsibility for what happens.
Look at writing a custom TraceEventHandler
(http://fusesource.com/docs/router/2.5/apidoc/org/apache/camel/processor/interceptor/TraceEventHandler.html)
which you can then set with Tracer::setTraceHandler and take complete
control of how tracing works.

There may be other, simpler, ways to get enough performance from the
default tracing, but a custom TraceEventHandler should be the fastest
you can get.

Jim
Reply | Threaded
Open this post in threaded view
|

Re: Camel Tracer impedes performance.

Naira & Kobo
Thanks a lot for your response.

Whats is the name of the default trace event handler used by camel tracer?

I 'd like to take a look at that class and see how I can optimize it.

I will actually prefer if messages intercepted and logged in a separate thread form the main exchange thread. This way, the main exchange can keep running without having to be blocked/impeded by the trace handler.

I am curious, is it the TraceEventHandler that impedes the performance or the TraceInceptor  or the InterceptStrategy?

If the differences in the usage of these classes are clear to you, can you please explain to me how they differ?

I assume, when there is a new exchange on a node, I assume the TraceInterceptor intercepts the message using a strategy defined by the InterceptStrategy (This I assume should actually create a new thread and get a copy of the exchange. At this point I assume the main Exchange, should not try to transform exchange message to suite the interceptor). I also assume it is after the Trace Interceptor accepts the message, then the TraceEventHandler 'd attempt to "handle" or process the trace event.

At this time, the main exchange should not be impeded (meaning, messages should be forwarded to subsequent nodes/endpoints without it being blocked by the the trace processing).

Again, these are my assumptions. Kindly help clarify what are, and what are not.
Reply | Threaded
Open this post in threaded view
|

Re: Camel Tracer impedes performance.

Jim Talbut-2
  On 17/02/2011 11:21, Naira & Kobo wrote:
> Thanks a lot for your response.
>
> Whats is the name of the default trace event handler used by camel tracer?
>
> I 'd like to take a look at that class and see how I can optimize it.
>
> I will actually prefer if messages intercepted and logged in a separate
> thread form the main exchange thread. This way, the main exchange can keep
> running without having to be blocked/impeded by the trace handler.
There isn't one.
Have a look at
org\apache\camel\processor\interceptor\TraceInterceptor.java (line 299+)
to see what happens, if there is a traceHandler it gets used otherwise
the default processing takes place.
You can replace the TraceInterceptor, but that's even more work 'cos the
interceptor does useful stuff (the TraceEventHandler is an opportunity
to use the useful stuff from the TraceInterceptor without having to copy
it).

If you want the processing on a different thread then I think you will
have to pay the price of copy the Exchange (which may not be a big
penalty in your case, I would expect that DB writes are the worst perf hit).

Jim

Reply | Threaded
Open this post in threaded view
|

Re: Camel Tracer impedes performance.

Naira & Kobo

I am curious, is it the TraceEventHandler that impedes the performance or the TraceInceptor  or the InterceptStrategy?

If the differences in the usage of these classes are clear to you, can you please explain to me how they differ?

I assume, when there is a new exchange on a node, the TraceInterceptor intercepts the message using a strategy defined by the InterceptStrategy (This I assume should actually create a new thread and get a copy of the exchange. At this point I assume the main Exchange, should not try to transform exchange message to suite the interceptor). I also assume it is after the Trace Interceptor accepts the message, then the TraceEventHandler 'd attempt to "handle" or process the trace event.

At this time, the main exchange should not be impeded (meaning, messages should be forwarded to subsequent nodes/endpoints without it being blocked by the the trace processing).

Again, these are my assumptions. Kindly help clarify what are, and what are not.  Basically, all I want to do is, allow original message to continue being processed then use a separate thread handle traces. This way, my original transaction will not be blocked by the tracer and it can still be completed as if the tracer were not turned on. How long its takes traces to be log is not a big deal as it is a trace, so it can take for as long it wants to - reasonably though.
Reply | Threaded
Open this post in threaded view
|

AW: Camel Tracer impedes performance.

Christian Schneider-2
Perhaps you should use a seda queue

<endpoint id="traceEndpoint" uri="seda:traceRoute? waitForTaskToComplete=Never "/>

<route>
    <from uri="seda:traceRoute"/>
      <bean ref="tracer" method="Log"/>
 </route>

The direct endpoint you used is always synchronous so this may cause the performance hit.
You may also want to use a jms topic for tracing. This way you can even combine the traces of several servers.
You can then attach one listener that writes to the db but also easily plug in e.g. a live monitoring console.

Christian


-----Urspr√ľngliche Nachricht-----
Von: Naira & Kobo [mailto:[hidden email]]
Gesendet: Donnerstag, 17. Februar 2011 12:59
An: [hidden email]
Betreff: Re: Camel Tracer impedes performance.



I am curious, is it the TraceEventHandler that impedes the performance or the TraceInceptor  or the InterceptStrategy?

If the differences in the usage of these classes are clear to you, can you please explain to me how they differ?

I assume, when there is a new exchange on a node, the TraceInterceptor intercepts the message using a strategy defined by the InterceptStrategy (This I assume should actually create a new thread and get a copy of the exchange. At this point I assume the main Exchange, should not try to transform exchange message to suite the interceptor). I also assume it is after the Trace Interceptor accepts the message, then the TraceEventHandler 'd attempt to "handle" or process the trace event.

At this time, the main exchange should not be impeded (meaning, messages should be forwarded to subsequent nodes/endpoints without it being blocked by the the trace processing).

Again, these are my assumptions. Kindly help clarify what are, and what are not.  Basically, all I want to do is, allow original message to continue being processed then use a separate thread handle traces. This way, my original transaction will not be blocked by the tracer and it can still be completed as if the tracer were not turned on. How long its takes traces to be log is not a big deal as it is a trace, so it can take for as long it wants to - reasonably though.
--
View this message in context: http://camel.465427.n5.nabble.com/Camel-Tracer-impedes-performance-tp3389173p3389307.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: Camel Tracer impedes performance.

Jim Talbut-2
In reply to this post by Naira & Kobo
  On 17/02/2011 11:58, Naira & Kobo wrote:
> I am curious, is it the TraceEventHandler that impedes the performance or
> the TraceInceptor  or the InterceptStrategy?
Well it's definitely not the TraceEventHandler, because unless you
provide one, there isn't one.
Beyond saying that if you want a definitive answer you should profile it
(I can guarantee that tracing has an impact, but whether it's even
measurable or not will depend on a great many factors).

Also everything else that follows is shaky and may be completely wrong
(I hope someone will point it out if it is).

The InterceptStrategy for tracing is called Tracer
(org.apache.camel.processor.interceptor.Tracer).
The strategy doesn't really do very much, it just sits there as
configuration device - it holds the parameters that you can set for
tracing and it creates trace interceptors for any routes that it is told
about.
So if you were to profile it you wouldn't spend much time in Tracer.

A TraceInterceptor is set up for every node in the route and any
performance impact from tracing must come within it's "process" method.
The process method does nothing for some nodes, and the rest of it is
concerned with either setting up some tracking data or calling the right
tracing methods.
Most of the tracing methods call both logExchange and traceExchange,
either of which could be expensive (but one would hope that logging is
cheap, so I'm going to ignore it).

So the guts of it all is the traceExchange function, called in the
TraceInterceptor of ever 'important' node (typically being those you set
up) in your route.

The traceExchange function copies the exchange and then feeds it to the
trace route (which, by default, is in the same thread, I think).
So the expensive bit is likely to be in that route processing, and if
you want to reduce the impact I think you need to reduce the latency of
that route.

Jim

> If the differences in the usage of these classes are clear to you, can you
> please explain to me how they differ?
>
> I assume, when there is a new exchange on a node, the TraceInterceptor
> intercepts the message using a strategy defined by the InterceptStrategy
> (This I assume should actually create a new thread and get a copy of the
> exchange. At this point I assume the main Exchange, should not try to
> transform exchange message to suite the interceptor). I also assume it is
> after the Trace Interceptor accepts the message, then the TraceEventHandler
> 'd attempt to "handle" or process the trace event.
>
> At this time, the main exchange should not be impeded (meaning, messages
> should be forwarded to subsequent nodes/endpoints without it being blocked
> by the the trace processing).
>
> Again, these are my assumptions. Kindly help clarify what are, and what are
> not.  Basically, all I want to do is, allow original message to continue
> being processed then use a separate thread handle traces. This way, my
> original transaction will not be blocked by the tracer and it can still be
> completed as if the tracer were not turned on. How long its takes traces to
> be log is not a big deal as it is a trace, so it can take for as long it
> wants to - reasonably though.