Camel + ActiveMQ shutdown timeout problem

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Camel + ActiveMQ shutdown timeout problem


We are using ServiceMix as a container, with an embedded ActiveMQ broker and some Camel routes deployed as JBI service assemblies. ServiceMix is registered as a Windows service, so when the server machine shuts down, the service is also shut down. The current timeout set on ServiceMix is 10 seconds, so if all the components can't be shut down before the timeout, they are somehow dropped/discarded.

We wouldn't care about this if we didn't have situations when, after a restart with TimeoutException, some messages are not delivered by the broker to the Camel endpoints anymore. They are stuck in kahaDB and until the index is rebuilt or kahaDB is deleted, messages keep accumulating. Some are delivered after the broker is restarted, but not all of them.

Our research pointed out the fact that the shutdown of the Camel routes is taking very long, between 20 and 30 seconds, so many times the logging shows how some Camel routes are "preparing to shutdown complete" but there is no "shutdown complete" afterwards. Probably some routes are not even prepared when Windows shuts down.

So, if the routes are in the process of shutting down, could this damage the kahaDB or the kahaDB index? I am suspecting that Windows kills all the running threads suddenly, which includes possible threads which are running and updating kahaDB or the index.

The default timeout for shutting down a Camel route is 300 seconds, if I lower it to 1 second what could go wrong?

My questions is linked with:

Camel version is 2.7.3-fuse-00-89.

Many thanks,