Some thoughts about the architecture of camel

classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Some thoughts about the architecture of camel

Schneider Christian
Hi all,

I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.

As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:

"The camel components should know as little as possible about camel core"

"The classes needed to setup camel should be separate from the things needed at run time"

So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.

So my idea would be to split camel-core into three parts:

api, builder, impl

These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.

As allowed depdencies I propose ( "->" means may use, depends on):

* -> api
end user config -> builder
builder -> impl

I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.

What do you think about these ideas?

Thanks

Christian

Christian Schneider
Informationsverarbeitung
Business Solutions
Handel und Dispatching

Tel : +49-(0)721-63-15482

EnBW Systeme Infrastruktur Support GmbH
Sitz der Gesellschaft: Karlsruhe
Handelsregister: Amtsgericht Mannheim ‑ HRB 108550
Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
Geschäftsführer: Jochen Adenau, Hans-Günther Meier



Reply | Threaded
Open this post in threaded view
|

[DISCUSS] Some thoughts about the architecture of camel

hadrian
I changed the thread name to [discuss].

I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.

What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.

Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.

Cheers,
Hadrian

[1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap



On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:

> Hi all,
>
> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>
> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>
> "The camel components should know as little as possible about camel core"
>
> "The classes needed to setup camel should be separate from the things needed at run time"
>
> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>
> So my idea would be to split camel-core into three parts:
>
> api, builder, impl
>
> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>
> As allowed depdencies I propose ( "->" means may use, depends on):
>
> * -> api
> end user config -> builder
> builder -> impl
>
> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>
> What do you think about these ideas?
>
> Thanks
>
> Christian
>
> Christian Schneider
> Informationsverarbeitung
> Business Solutions
> Handel und Dispatching
>
> Tel : +49-(0)721-63-15482
>
> EnBW Systeme Infrastruktur Support GmbH
> Sitz der Gesellschaft: Karlsruhe
> Handelsregister: Amtsgericht Mannheim ‑ HRB 108550
> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Claus Ibsen-2
Hi

I think the idea is really great, but I think the timing for this is
*not* the right spot.

And by saying that I mean the goal of Camel 3.0 is to have a short
development cycle (not like 2.0 which took a long time).
And as a minimum (IMHO):
- To upgrade to JDK 1.6+,
- Spring 3.0+,
- To optimize the router internally,
- And to switch to slf4j logger (*)
- Keeping backwards compatibility as much as possible with 2.x is paramount

Switching to slf4j instead of commons logging, allows us to use the
MDC logging feature.
This allows us to push information to the logs such as message id,
transaction id etc. which can more easily correlate logs, not only
with Camel alone, but also with other projects such as ActiveMQ, SMX
etc.


On top of that we now have many 3rd party projects which integrate out
of the box with Camel, so changing the package structure in camel-core
will break their integration. Which means they may not take up the
effort to support both Camel 2.x/3.x.

However I do think we should take the effort and pick up the low
hanging fruits. I am sure there could be a couple of tangles which can
be identified and fixed in Camel 3.0, without breaking backwards
compatibility.

I think doing this is something for Camel 4 or 5 or 6 (or whatever
future version it may be) when or if we change to use Scala and use
some other framework as foundation. There are cool stuff being
developed for ActiveMQ 6 which are potential as a backbone for route
messages. And it has a much better threading model which Camel can
benefit as well.

Anyway practical works beats theory, so setting up a branch in the
sandbox to do experiments is a great idea.

But its very important that we keep backwards compatibility with Camel
3.0. There are so many people started using Camel 2.x now so we should
keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
2.x but now on JDK 1.6 and with X other internal upgrades.

Okay my first cup of coffee is ready, so beware this mail was written
before I got "my first fix".



On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:

> I changed the thread name to [discuss].
>
> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.
>
> What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.
>
> Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.
>
> Cheers,
> Hadrian
>
> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>
>
>
> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>
>> Hi all,
>>
>> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>>
>> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>>
>> "The camel components should know as little as possible about camel core"
>>
>> "The classes needed to setup camel should be separate from the things needed at run time"
>>
>> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>>
>> So my idea would be to split camel-core into three parts:
>>
>> api, builder, impl
>>
>> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>>
>> As allowed depdencies I propose ( "->" means may use, depends on):
>>
>> * -> api
>> end user config -> builder
>> builder -> impl
>>
>> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>>
>> What do you think about these ideas?
>>
>> Thanks
>>
>> Christian
>>
>> Christian Schneider
>> Informationsverarbeitung
>> Business Solutions
>> Handel und Dispatching
>>
>> Tel : +49-(0)721-63-15482
>>
>> EnBW Systeme Infrastruktur Support GmbH
>> Sitz der Gesellschaft: Karlsruhe
>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>
>>
>>
>
>



--
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Johan Edstrom-2
Could you post a "LOW HANGING FRUIT"
Page that us mortals could start hitting?


On Oct 18, 2010, at 10:23 PM, Claus Ibsen wrote:

> Hi
>
> I think the idea is really great, but I think the timing for this is
> *not* the right spot.
>
> And by saying that I mean the goal of Camel 3.0 is to have a short
> development cycle (not like 2.0 which took a long time).
> And as a minimum (IMHO):
> - To upgrade to JDK 1.6+,
> - Spring 3.0+,
> - To optimize the router internally,
> - And to switch to slf4j logger (*)
> - Keeping backwards compatibility as much as possible with 2.x is paramount
>
> Switching to slf4j instead of commons logging, allows us to use the
> MDC logging feature.
> This allows us to push information to the logs such as message id,
> transaction id etc. which can more easily correlate logs, not only
> with Camel alone, but also with other projects such as ActiveMQ, SMX
> etc.
>
>
> On top of that we now have many 3rd party projects which integrate out
> of the box with Camel, so changing the package structure in camel-core
> will break their integration. Which means they may not take up the
> effort to support both Camel 2.x/3.x.
>
> However I do think we should take the effort and pick up the low
> hanging fruits. I am sure there could be a couple of tangles which can
> be identified and fixed in Camel 3.0, without breaking backwards
> compatibility.
>
> I think doing this is something for Camel 4 or 5 or 6 (or whatever
> future version it may be) when or if we change to use Scala and use
> some other framework as foundation. There are cool stuff being
> developed for ActiveMQ 6 which are potential as a backbone for route
> messages. And it has a much better threading model which Camel can
> benefit as well.
>
> Anyway practical works beats theory, so setting up a branch in the
> sandbox to do experiments is a great idea.
>
> But its very important that we keep backwards compatibility with Camel
> 3.0. There are so many people started using Camel 2.x now so we should
> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
> 2.x but now on JDK 1.6 and with X other internal upgrades.
>
> Okay my first cup of coffee is ready, so beware this mail was written
> before I got "my first fix".
>
>
>
> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
>> I changed the thread name to [discuss].
>>
>> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.
>>
>> What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.
>>
>> Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.
>>
>> Cheers,
>> Hadrian
>>
>> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>
>>
>>
>> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>
>>> Hi all,
>>>
>>> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>>>
>>> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>>>
>>> "The camel components should know as little as possible about camel core"
>>>
>>> "The classes needed to setup camel should be separate from the things needed at run time"
>>>
>>> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>>>
>>> So my idea would be to split camel-core into three parts:
>>>
>>> api, builder, impl
>>>
>>> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>>>
>>> As allowed depdencies I propose ( "->" means may use, depends on):
>>>
>>> * -> api
>>> end user config -> builder
>>> builder -> impl
>>>
>>> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>>>
>>> What do you think about these ideas?
>>>
>>> Thanks
>>>
>>> Christian
>>>
>>> Christian Schneider
>>> Informationsverarbeitung
>>> Business Solutions
>>> Handel und Dispatching
>>>
>>> Tel : +49-(0)721-63-15482
>>>
>>> EnBW Systeme Infrastruktur Support GmbH
>>> Sitz der Gesellschaft: Karlsruhe
>>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>>
>>>
>>>
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Johan Edstrom

[hidden email]

They that can give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.

Benjamin Franklin, Historical Review of Pennsylvania, 1759





Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Johan Edstrom-2
In reply to this post by Claus Ibsen-2
Since you did not get your Coffee :)
Would you mind putting the hindrances up to the mere mortals?


On Oct 18, 2010, at 10:23 PM, Claus Ibsen wrote:

> Hi
>
> I think the idea is really great, but I think the timing for this is
> *not* the right spot.
>
> And by saying that I mean the goal of Camel 3.0 is to have a short
> development cycle (not like 2.0 which took a long time).
> And as a minimum (IMHO):
> - To upgrade to JDK 1.6+,
> - Spring 3.0+,
> - To optimize the router internally,
> - And to switch to slf4j logger (*)
> - Keeping backwards compatibility as much as possible with 2.x is paramount
>
> Switching to slf4j instead of commons logging, allows us to use the
> MDC logging feature.
> This allows us to push information to the logs such as message id,
> transaction id etc. which can more easily correlate logs, not only
> with Camel alone, but also with other projects such as ActiveMQ, SMX
> etc.
>
>
> On top of that we now have many 3rd party projects which integrate out
> of the box with Camel, so changing the package structure in camel-core
> will break their integration. Which means they may not take up the
> effort to support both Camel 2.x/3.x.
>
> However I do think we should take the effort and pick up the low
> hanging fruits. I am sure there could be a couple of tangles which can
> be identified and fixed in Camel 3.0, without breaking backwards
> compatibility.
>
> I think doing this is something for Camel 4 or 5 or 6 (or whatever
> future version it may be) when or if we change to use Scala and use
> some other framework as foundation. There are cool stuff being
> developed for ActiveMQ 6 which are potential as a backbone for route
> messages. And it has a much better threading model which Camel can
> benefit as well.
>
> Anyway practical works beats theory, so setting up a branch in the
> sandbox to do experiments is a great idea.
>
> But its very important that we keep backwards compatibility with Camel
> 3.0. There are so many people started using Camel 2.x now so we should
> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
> 2.x but now on JDK 1.6 and with X other internal upgrades.
>
> Okay my first cup of coffee is ready, so beware this mail was written
> before I got "my first fix".
>
>
>
> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
>> I changed the thread name to [discuss].
>>
>> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.
>>
>> What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.
>>
>> Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.
>>
>> Cheers,
>> Hadrian
>>
>> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>
>>
>>
>> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>
>>> Hi all,
>>>
>>> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>>>
>>> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>>>
>>> "The camel components should know as little as possible about camel core"
>>>
>>> "The classes needed to setup camel should be separate from the things needed at run time"
>>>
>>> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>>>
>>> So my idea would be to split camel-core into three parts:
>>>
>>> api, builder, impl
>>>
>>> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>>>
>>> As allowed depdencies I propose ( "->" means may use, depends on):
>>>
>>> * -> api
>>> end user config -> builder
>>> builder -> impl
>>>
>>> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>>>
>>> What do you think about these ideas?
>>>
>>> Thanks
>>>
>>> Christian
>>>
>>> Christian Schneider
>>> Informationsverarbeitung
>>> Business Solutions
>>> Handel und Dispatching
>>>
>>> Tel : +49-(0)721-63-15482
>>>
>>> EnBW Systeme Infrastruktur Support GmbH
>>> Sitz der Gesellschaft: Karlsruhe
>>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>>
>>>
>>>
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Johan Edstrom

[hidden email]

They that can give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.

Benjamin Franklin, Historical Review of Pennsylvania, 1759





Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Claus Ibsen-2
On Tue, Oct 19, 2010 at 6:40 AM, Johan Edstrom <[hidden email]> wrote:
> Since you did not get your Coffee :)
> Would you mind putting the hindrances up to the mere mortals?
>

What do you mean?

>
> On Oct 18, 2010, at 10:23 PM, Claus Ibsen wrote:
>
>> Hi
>>
>> I think the idea is really great, but I think the timing for this is
>> *not* the right spot.
>>
>> And by saying that I mean the goal of Camel 3.0 is to have a short
>> development cycle (not like 2.0 which took a long time).
>> And as a minimum (IMHO):
>> - To upgrade to JDK 1.6+,
>> - Spring 3.0+,
>> - To optimize the router internally,
>> - And to switch to slf4j logger (*)
>> - Keeping backwards compatibility as much as possible with 2.x is paramount
>>
>> Switching to slf4j instead of commons logging, allows us to use the
>> MDC logging feature.
>> This allows us to push information to the logs such as message id,
>> transaction id etc. which can more easily correlate logs, not only
>> with Camel alone, but also with other projects such as ActiveMQ, SMX
>> etc.
>>
>>
>> On top of that we now have many 3rd party projects which integrate out
>> of the box with Camel, so changing the package structure in camel-core
>> will break their integration. Which means they may not take up the
>> effort to support both Camel 2.x/3.x.
>>
>> However I do think we should take the effort and pick up the low
>> hanging fruits. I am sure there could be a couple of tangles which can
>> be identified and fixed in Camel 3.0, without breaking backwards
>> compatibility.
>>
>> I think doing this is something for Camel 4 or 5 or 6 (or whatever
>> future version it may be) when or if we change to use Scala and use
>> some other framework as foundation. There are cool stuff being
>> developed for ActiveMQ 6 which are potential as a backbone for route
>> messages. And it has a much better threading model which Camel can
>> benefit as well.
>>
>> Anyway practical works beats theory, so setting up a branch in the
>> sandbox to do experiments is a great idea.
>>
>> But its very important that we keep backwards compatibility with Camel
>> 3.0. There are so many people started using Camel 2.x now so we should
>> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
>> 2.x but now on JDK 1.6 and with X other internal upgrades.
>>
>> Okay my first cup of coffee is ready, so beware this mail was written
>> before I got "my first fix".
>>
>>
>>
>> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
>>> I changed the thread name to [discuss].
>>>
>>> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.
>>>
>>> What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.
>>>
>>> Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.
>>>
>>> Cheers,
>>> Hadrian
>>>
>>> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>>
>>>
>>>
>>> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>>
>>>> Hi all,
>>>>
>>>> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>>>>
>>>> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>>>>
>>>> "The camel components should know as little as possible about camel core"
>>>>
>>>> "The classes needed to setup camel should be separate from the things needed at run time"
>>>>
>>>> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>>>>
>>>> So my idea would be to split camel-core into three parts:
>>>>
>>>> api, builder, impl
>>>>
>>>> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>>>>
>>>> As allowed depdencies I propose ( "->" means may use, depends on):
>>>>
>>>> * -> api
>>>> end user config -> builder
>>>> builder -> impl
>>>>
>>>> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>>>>
>>>> What do you think about these ideas?
>>>>
>>>> Thanks
>>>>
>>>> Christian
>>>>
>>>> Christian Schneider
>>>> Informationsverarbeitung
>>>> Business Solutions
>>>> Handel und Dispatching
>>>>
>>>> Tel : +49-(0)721-63-15482
>>>>
>>>> EnBW Systeme Infrastruktur Support GmbH
>>>> Sitz der Gesellschaft: Karlsruhe
>>>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
> Johan Edstrom
>
> [hidden email]
>
> They that can give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.
>
> Benjamin Franklin, Historical Review of Pennsylvania, 1759
>
>
>
>
>
>



--
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Johan Edstrom-2
How do we help you?

On Oct 18, 2010, at 10:42 PM, Claus Ibsen wrote:

> On Tue, Oct 19, 2010 at 6:40 AM, Johan Edstrom <[hidden email]> wrote:
>> Since you did not get your Coffee :)
>> Would you mind putting the hindrances up to the mere mortals?
>>
>
> What do you mean?
>
>>
>> On Oct 18, 2010, at 10:23 PM, Claus Ibsen wrote:
>>
>>> Hi
>>>
>>> I think the idea is really great, but I think the timing for this is
>>> *not* the right spot.
>>>
>>> And by saying that I mean the goal of Camel 3.0 is to have a short
>>> development cycle (not like 2.0 which took a long time).
>>> And as a minimum (IMHO):
>>> - To upgrade to JDK 1.6+,
>>> - Spring 3.0+,
>>> - To optimize the router internally,
>>> - And to switch to slf4j logger (*)
>>> - Keeping backwards compatibility as much as possible with 2.x is paramount
>>>
>>> Switching to slf4j instead of commons logging, allows us to use the
>>> MDC logging feature.
>>> This allows us to push information to the logs such as message id,
>>> transaction id etc. which can more easily correlate logs, not only
>>> with Camel alone, but also with other projects such as ActiveMQ, SMX
>>> etc.
>>>
>>>
>>> On top of that we now have many 3rd party projects which integrate out
>>> of the box with Camel, so changing the package structure in camel-core
>>> will break their integration. Which means they may not take up the
>>> effort to support both Camel 2.x/3.x.
>>>
>>> However I do think we should take the effort and pick up the low
>>> hanging fruits. I am sure there could be a couple of tangles which can
>>> be identified and fixed in Camel 3.0, without breaking backwards
>>> compatibility.
>>>
>>> I think doing this is something for Camel 4 or 5 or 6 (or whatever
>>> future version it may be) when or if we change to use Scala and use
>>> some other framework as foundation. There are cool stuff being
>>> developed for ActiveMQ 6 which are potential as a backbone for route
>>> messages. And it has a much better threading model which Camel can
>>> benefit as well.
>>>
>>> Anyway practical works beats theory, so setting up a branch in the
>>> sandbox to do experiments is a great idea.
>>>
>>> But its very important that we keep backwards compatibility with Camel
>>> 3.0. There are so many people started using Camel 2.x now so we should
>>> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
>>> 2.x but now on JDK 1.6 and with X other internal upgrades.
>>>
>>> Okay my first cup of coffee is ready, so beware this mail was written
>>> before I got "my first fix".
>>>
>>>
>>>
>>> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
>>>> I changed the thread name to [discuss].
>>>>
>>>> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.
>>>>
>>>> What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.
>>>>
>>>> Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.
>>>>
>>>> Cheers,
>>>> Hadrian
>>>>
>>>> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>>>
>>>>
>>>>
>>>> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>>>>>
>>>>> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>>>>>
>>>>> "The camel components should know as little as possible about camel core"
>>>>>
>>>>> "The classes needed to setup camel should be separate from the things needed at run time"
>>>>>
>>>>> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>>>>>
>>>>> So my idea would be to split camel-core into three parts:
>>>>>
>>>>> api, builder, impl
>>>>>
>>>>> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>>>>>
>>>>> As allowed depdencies I propose ( "->" means may use, depends on):
>>>>>
>>>>> * -> api
>>>>> end user config -> builder
>>>>> builder -> impl
>>>>>
>>>>> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>>>>>
>>>>> What do you think about these ideas?
>>>>>
>>>>> Thanks
>>>>>
>>>>> Christian
>>>>>
>>>>> Christian Schneider
>>>>> Informationsverarbeitung
>>>>> Business Solutions
>>>>> Handel und Dispatching
>>>>>
>>>>> Tel : +49-(0)721-63-15482
>>>>>
>>>>> EnBW Systeme Infrastruktur Support GmbH
>>>>> Sitz der Gesellschaft: Karlsruhe
>>>>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>>>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>>>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>
>> Johan Edstrom
>>
>> [hidden email]
>>
>> They that can give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.
>>
>> Benjamin Franklin, Historical Review of Pennsylvania, 1759
>>
>>
>>
>>
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus

Johan Edstrom

[hidden email]

They that can give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.

Benjamin Franklin, Historical Review of Pennsylvania, 1759





Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Martin Krasser-3
In reply to this post by Claus Ibsen-2
  +1 for being fully backwards compatible. I've many projects running on
Camel 2.x and would like to benefit from Camel 3.x features without
doing any (bigger) changes.
+1 for making a bigger refactoring when going for a more idiomatic Scala
(DSL) support (with more idiomatic I mean more FP-style e.g. higher
order functions as combinators in the DSL etc.)

I think it might be useful if we're going to write some more load- and
stress tests for more frequently used Camel components (jetty, jms etc.)
and camel-core and run them on a weekly basis, for example. This is not
only to ensure correct behaviour under load: I've also had several
customers in the past that where asking for performance numbers. I know
that performance numbers make most sense for complete applications, but
having a reference for individual application parts (e.g. raw throughput
of jetty consumer endpoints) can be useful as well. It would also allow
us to detect performance decreases more easily (e.g. after refactorings).


Am 19.10.10 06:23, schrieb Claus Ibsen:

> Hi
>
> I think the idea is really great, but I think the timing for this is
> *not* the right spot.
>
> And by saying that I mean the goal of Camel 3.0 is to have a short
> development cycle (not like 2.0 which took a long time).
> And as a minimum (IMHO):
> - To upgrade to JDK 1.6+,
> - Spring 3.0+,
> - To optimize the router internally,
> - And to switch to slf4j logger (*)
> - Keeping backwards compatibility as much as possible with 2.x is paramount
>
> Switching to slf4j instead of commons logging, allows us to use the
> MDC logging feature.
> This allows us to push information to the logs such as message id,
> transaction id etc. which can more easily correlate logs, not only
> with Camel alone, but also with other projects such as ActiveMQ, SMX
> etc.
>
>
> On top of that we now have many 3rd party projects which integrate out
> of the box with Camel, so changing the package structure in camel-core
> will break their integration. Which means they may not take up the
> effort to support both Camel 2.x/3.x.
>
> However I do think we should take the effort and pick up the low
> hanging fruits. I am sure there could be a couple of tangles which can
> be identified and fixed in Camel 3.0, without breaking backwards
> compatibility.
>
> I think doing this is something for Camel 4 or 5 or 6 (or whatever
> future version it may be) when or if we change to use Scala and use
> some other framework as foundation. There are cool stuff being
> developed for ActiveMQ 6 which are potential as a backbone for route
> messages. And it has a much better threading model which Camel can
> benefit as well.
>
> Anyway practical works beats theory, so setting up a branch in the
> sandbox to do experiments is a great idea.
>
> But its very important that we keep backwards compatibility with Camel
> 3.0. There are so many people started using Camel 2.x now so we should
> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
> 2.x but now on JDK 1.6 and with X other internal upgrades.
>
> Okay my first cup of coffee is ready, so beware this mail was written
> before I got "my first fix".
>
>
>
> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea<[hidden email]>  wrote:
>> I changed the thread name to [discuss].
>>
>> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.
>>
>> What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.
>>
>> Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.
>>
>> Cheers,
>> Hadrian
>>
>> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>
>>
>>
>> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>
>>> Hi all,
>>>
>>> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>>>
>>> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>>>
>>> "The camel components should know as little as possible about camel core"
>>>
>>> "The classes needed to setup camel should be separate from the things needed at run time"
>>>
>>> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>>>
>>> So my idea would be to split camel-core into three parts:
>>>
>>> api, builder, impl
>>>
>>> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>>>
>>> As allowed depdencies I propose ( "->" means may use, depends on):
>>>
>>> * ->  api
>>> end user config ->  builder
>>> builder ->  impl
>>>
>>> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>>>
>>> What do you think about these ideas?
>>>
>>> Thanks
>>>
>>> Christian
>>>
>>> Christian Schneider
>>> Informationsverarbeitung
>>> Business Solutions
>>> Handel und Dispatching
>>>
>>> Tel : +49-(0)721-63-15482
>>>
>>> EnBW Systeme Infrastruktur Support GmbH
>>> Sitz der Gesellschaft: Karlsruhe
>>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>>
>>>
>>>
>>
>
>


--
Martin Krasser

blog:    http://krasserm.blogspot.com
code:    http://github.com/krasserm
twitter: http://twitter.com/mrt1nz

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Claus Ibsen-2
In reply to this post by Johan Edstrom-2
On Tue, Oct 19, 2010 at 6:44 AM, Johan Edstrom <[hidden email]> wrote:
> How do we help you?
>

About Camel 3.0?

Anyone who wants to contribute can of course help. We love
contributions at Apache.
And Johan just keep up a bit more and I am sure you got the attentions
of the PMCs to come onboard.


First I think we need Camel 2.5 to be shipped.

Then Hadrian can setup the 2.x branch and ensure trunk is ready for
Camel 3.0 development.

A rough plan for tasks to do could be something like this
- Then we can upgrade to JDK 1.6 and fix a few minor spots (eg
IOException now accepts nested exception, no need to use IOHelper)
- Removed @deprecated stuff from 2.x
- Upgrade to Spring 3.0
- Add SPEL (from Spring 3.0) as language to Camel
- There are potentially some code in the Camel spring namespace holder
which now may be obsolete by Spring 3 (check upon that)
- Upgrade to CXF 2.3 as people would like (eg it uses Jetty 7 and has
better continuations)
- Switch to use slf4j in camel-core, camel-spring and other important components
- And gradually switch to slf4j in the other camel components
- Use MDC in camel-core for enriching the logger with message id, tx
id (bread crumb, to trace message)
- Create camel-osgi-test for OSGi test kit

After all this work we can start looking into one of the tougher
pieces which is to optimize the internal router
- Reduce the need for copying the Exchange while routing
- Message facade with history and fix the useOriginalMessage
- Make the cross cutting concerns more dynamic (currently they are
statically woven into the route when the route is generated)
  Instead it ought to be dynamic applied, so its much easier to
inject/remove interceptors, error handlers, and whatnot

And a ton of stuff which I may forgot.
And of course we will still continue doing the usual work of fixing
bugs, adding new components, improvements and what else the community
reports.

In terms of "package tangle" there could be some low hanging fruits.
There is a tool which can visualize the packages. Structure 101 which
is really good at this.

There are stuff which can be started already now, such as the
camel-osgi-test which ought to be able to backport to Camel 2.x as
well. We got some rough code in tests/camel-itest-osgi which "just"
needs to be polished and moved into a new component.

Okay getting ready for the 2nd cup now.



> On Oct 18, 2010, at 10:42 PM, Claus Ibsen wrote:
>
>> On Tue, Oct 19, 2010 at 6:40 AM, Johan Edstrom <[hidden email]> wrote:
>>> Since you did not get your Coffee :)
>>> Would you mind putting the hindrances up to the mere mortals?
>>>
>>
>> What do you mean?
>>
>>>
>>> On Oct 18, 2010, at 10:23 PM, Claus Ibsen wrote:
>>>
>>>> Hi
>>>>
>>>> I think the idea is really great, but I think the timing for this is
>>>> *not* the right spot.
>>>>
>>>> And by saying that I mean the goal of Camel 3.0 is to have a short
>>>> development cycle (not like 2.0 which took a long time).
>>>> And as a minimum (IMHO):
>>>> - To upgrade to JDK 1.6+,
>>>> - Spring 3.0+,
>>>> - To optimize the router internally,
>>>> - And to switch to slf4j logger (*)
>>>> - Keeping backwards compatibility as much as possible with 2.x is paramount
>>>>
>>>> Switching to slf4j instead of commons logging, allows us to use the
>>>> MDC logging feature.
>>>> This allows us to push information to the logs such as message id,
>>>> transaction id etc. which can more easily correlate logs, not only
>>>> with Camel alone, but also with other projects such as ActiveMQ, SMX
>>>> etc.
>>>>
>>>>
>>>> On top of that we now have many 3rd party projects which integrate out
>>>> of the box with Camel, so changing the package structure in camel-core
>>>> will break their integration. Which means they may not take up the
>>>> effort to support both Camel 2.x/3.x.
>>>>
>>>> However I do think we should take the effort and pick up the low
>>>> hanging fruits. I am sure there could be a couple of tangles which can
>>>> be identified and fixed in Camel 3.0, without breaking backwards
>>>> compatibility.
>>>>
>>>> I think doing this is something for Camel 4 or 5 or 6 (or whatever
>>>> future version it may be) when or if we change to use Scala and use
>>>> some other framework as foundation. There are cool stuff being
>>>> developed for ActiveMQ 6 which are potential as a backbone for route
>>>> messages. And it has a much better threading model which Camel can
>>>> benefit as well.
>>>>
>>>> Anyway practical works beats theory, so setting up a branch in the
>>>> sandbox to do experiments is a great idea.
>>>>
>>>> But its very important that we keep backwards compatibility with Camel
>>>> 3.0. There are so many people started using Camel 2.x now so we should
>>>> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
>>>> 2.x but now on JDK 1.6 and with X other internal upgrades.
>>>>
>>>> Okay my first cup of coffee is ready, so beware this mail was written
>>>> before I got "my first fix".
>>>>
>>>>
>>>>
>>>> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
>>>>> I changed the thread name to [discuss].
>>>>>
>>>>> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.
>>>>>
>>>>> What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.
>>>>>
>>>>> Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.
>>>>>
>>>>> Cheers,
>>>>> Hadrian
>>>>>
>>>>> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>>>>
>>>>>
>>>>>
>>>>> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>>>>>>
>>>>>> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>>>>>>
>>>>>> "The camel components should know as little as possible about camel core"
>>>>>>
>>>>>> "The classes needed to setup camel should be separate from the things needed at run time"
>>>>>>
>>>>>> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>>>>>>
>>>>>> So my idea would be to split camel-core into three parts:
>>>>>>
>>>>>> api, builder, impl
>>>>>>
>>>>>> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>>>>>>
>>>>>> As allowed depdencies I propose ( "->" means may use, depends on):
>>>>>>
>>>>>> * -> api
>>>>>> end user config -> builder
>>>>>> builder -> impl
>>>>>>
>>>>>> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>>>>>>
>>>>>> What do you think about these ideas?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Christian
>>>>>>
>>>>>> Christian Schneider
>>>>>> Informationsverarbeitung
>>>>>> Business Solutions
>>>>>> Handel und Dispatching
>>>>>>
>>>>>> Tel : +49-(0)721-63-15482
>>>>>>
>>>>>> EnBW Systeme Infrastruktur Support GmbH
>>>>>> Sitz der Gesellschaft: Karlsruhe
>>>>>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>>>>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>>>>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>
>>> Johan Edstrom
>>>
>>> [hidden email]
>>>
>>> They that can give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.
>>>
>>> Benjamin Franklin, Historical Review of Pennsylvania, 1759
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>
> Johan Edstrom
>
> [hidden email]
>
> They that can give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety.
>
> Benjamin Franklin, Historical Review of Pennsylvania, 1759
>
>
>
>
>
>



--
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Claus Ibsen-2
In reply to this post by Martin Krasser-3
On Tue, Oct 19, 2010 at 6:58 AM, Martin Krasser <[hidden email]> wrote:

>  +1 for being fully backwards compatible. I've many projects running on
> Camel 2.x and would like to benefit from Camel 3.x features without doing
> any (bigger) changes.
> +1 for making a bigger refactoring when going for a more idiomatic Scala
> (DSL) support (with more idiomatic I mean more FP-style e.g. higher order
> functions as combinators in the DSL etc.)
>
> I think it might be useful if we're going to write some more load- and
> stress tests for more frequently used Camel components (jetty, jms etc.) and
> camel-core and run them on a weekly basis, for example. This is not only to
> ensure correct behaviour under load: I've also had several customers in the
> past that where asking for performance numbers. I know that performance
> numbers make most sense for complete applications, but having a reference
> for individual application parts (e.g. raw throughput of jetty consumer
> endpoints) can be useful as well. It would also allow us to detect
> performance decreases more easily (e.g. after refactorings).
>

Good idea about the load tests.
If we can't make them easily run on some CI servers we can at least
run them locally to keep an eye on it.



>
> Am 19.10.10 06:23, schrieb Claus Ibsen:
>>
>> Hi
>>
>> I think the idea is really great, but I think the timing for this is
>> *not* the right spot.
>>
>> And by saying that I mean the goal of Camel 3.0 is to have a short
>> development cycle (not like 2.0 which took a long time).
>> And as a minimum (IMHO):
>> - To upgrade to JDK 1.6+,
>> - Spring 3.0+,
>> - To optimize the router internally,
>> - And to switch to slf4j logger (*)
>> - Keeping backwards compatibility as much as possible with 2.x is
>> paramount
>>
>> Switching to slf4j instead of commons logging, allows us to use the
>> MDC logging feature.
>> This allows us to push information to the logs such as message id,
>> transaction id etc. which can more easily correlate logs, not only
>> with Camel alone, but also with other projects such as ActiveMQ, SMX
>> etc.
>>
>>
>> On top of that we now have many 3rd party projects which integrate out
>> of the box with Camel, so changing the package structure in camel-core
>> will break their integration. Which means they may not take up the
>> effort to support both Camel 2.x/3.x.
>>
>> However I do think we should take the effort and pick up the low
>> hanging fruits. I am sure there could be a couple of tangles which can
>> be identified and fixed in Camel 3.0, without breaking backwards
>> compatibility.
>>
>> I think doing this is something for Camel 4 or 5 or 6 (or whatever
>> future version it may be) when or if we change to use Scala and use
>> some other framework as foundation. There are cool stuff being
>> developed for ActiveMQ 6 which are potential as a backbone for route
>> messages. And it has a much better threading model which Camel can
>> benefit as well.
>>
>> Anyway practical works beats theory, so setting up a branch in the
>> sandbox to do experiments is a great idea.
>>
>> But its very important that we keep backwards compatibility with Camel
>> 3.0. There are so many people started using Camel 2.x now so we should
>> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
>> 2.x but now on JDK 1.6 and with X other internal upgrades.
>>
>> Okay my first cup of coffee is ready, so beware this mail was written
>> before I got "my first fix".
>>
>>
>>
>> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea<[hidden email]>
>>  wrote:
>>>
>>> I changed the thread name to [discuss].
>>>
>>> I like that idea and it's something we contemplated in the past. This
>>> will bring back the idea of getting the dsl out of core as well.
>>>
>>> What I'd propose Christian is to add your proposal to the roadmap [1]. I
>>> will do the same for the dsl idea. There at least 2 ideas for dsl's built on
>>> top of the camel dsl (scheduling and debugging) that make me even more
>>> interested in coming up with a better solution.
>>>
>>> Once we get some traction on the main refactoring ideas I'd suggest
>>> starting one (or more) branches and start hacking, because there's not a
>>> whole lot of time left if we want to meet our target.
>>>
>>> Cheers,
>>> Hadrian
>>>
>>> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>>
>>>
>>>
>>> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>>
>>>> Hi all,
>>>>
>>>> I will have some free time in december as I am changing my employer. So
>>>> I am planning to work a little on some architectural improvements for camel
>>>> 3.0.0. As these things are very critical to the stability of camel I would
>>>> like to get feedback before I start any substantial work.
>>>>
>>>> As you surely know currently camel-core is quite tangled. So it is quite
>>>> difficult where to start. Some time ago I proposed some improvements to
>>>> simply reduce those dependency cycles. As I now know a lot more about camel
>>>> I think that this simple aproach will not really work. So this time I want
>>>> to do this a little more structured. So I start with two simple goals:
>>>>
>>>> "The camel components should know as little as possible about camel
>>>> core"
>>>>
>>>> "The classes needed to setup camel should be separate from the things
>>>> needed at run time"
>>>>
>>>> So why should this be important? Currently components depend on
>>>> camel-core as a whole and there are no further rules which classes the
>>>> components should use and which classes should be private to core. Even
>>>> classes from the impl package are needed. So this means that any refactoring
>>>> we do in camel core could affect all components. As camel is growing
>>>> steadily this can become quite problematic.
>>>>
>>>> So my idea would be to split camel-core into three parts:
>>>>
>>>> api, builder, impl
>>>>
>>>> These should be structured in a way that these big building blocks do
>>>> not have cyclic dependencies. Any other cycles can be ignored in this step.
>>>>
>>>> As allowed depdencies I propose ( "->" means may use, depends on):
>>>>
>>>> * ->  api
>>>> end user config ->  builder
>>>> builder ->  impl
>>>>
>>>> I think the first thing we should do is to discuss and reach a consensus
>>>> about a basic architecure and rules like above. Then the next step is to
>>>> assign each package of core to one of the basic parts. Then the next step is
>>>> to resolve cycles between the parts.
>>>>
>>>> What do you think about these ideas?
>>>>
>>>> Thanks
>>>>
>>>> Christian
>>>>
>>>> Christian Schneider
>>>> Informationsverarbeitung
>>>> Business Solutions
>>>> Handel und Dispatching
>>>>
>>>> Tel : +49-(0)721-63-15482
>>>>
>>>> EnBW Systeme Infrastruktur Support GmbH
>>>> Sitz der Gesellschaft: Karlsruhe
>>>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>>>
>>>>
>>>>
>>>
>>
>>
>
>
> --
> Martin Krasser
>
> blog:    http://krasserm.blogspot.com
> code:    http://github.com/krasserm
> twitter: http://twitter.com/mrt1nz
>
>



--
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Christian Schneider
In reply to this post by Claus Ibsen-2
  If we agree that camel 3.0 aims to be compatible then the refactorings
are not prossible of course.

So I propose we try to agree on a future architecture for camel core and
also assign packages to the new architectural modules. Then we can go
for the low hanging fruits and keep the rest as it is.
Perhaps we could already use the new rules for classes we add so we
don´t make things worse for a later refactoring.

So let´s start with a discussion about the new architecture. Do we all
agree on the basic architecture I proposed or are there other proposals?

Thanks

Christian


Am 19.10.2010 06:23, schrieb Claus Ibsen:

> Hi
>
> I think the idea is really great, but I think the timing for this is
> *not* the right spot.
>
> And by saying that I mean the goal of Camel 3.0 is to have a short
> development cycle (not like 2.0 which took a long time).
> And as a minimum (IMHO):
> - To upgrade to JDK 1.6+,
> - Spring 3.0+,
> - To optimize the router internally,
> - And to switch to slf4j logger (*)
> - Keeping backwards compatibility as much as possible with 2.x is paramount
>
> Switching to slf4j instead of commons logging, allows us to use the
> MDC logging feature.
> This allows us to push information to the logs such as message id,
> transaction id etc. which can more easily correlate logs, not only
> with Camel alone, but also with other projects such as ActiveMQ, SMX
> etc.
>
>
> On top of that we now have many 3rd party projects which integrate out
> of the box with Camel, so changing the package structure in camel-core
> will break their integration. Which means they may not take up the
> effort to support both Camel 2.x/3.x.
>
> However I do think we should take the effort and pick up the low
> hanging fruits. I am sure there could be a couple of tangles which can
> be identified and fixed in Camel 3.0, without breaking backwards
> compatibility.
>
> I think doing this is something for Camel 4 or 5 or 6 (or whatever
> future version it may be) when or if we change to use Scala and use
> some other framework as foundation. There are cool stuff being
> developed for ActiveMQ 6 which are potential as a backbone for route
> messages. And it has a much better threading model which Camel can
> benefit as well.
>
> Anyway practical works beats theory, so setting up a branch in the
> sandbox to do experiments is a great idea.
>
> But its very important that we keep backwards compatibility with Camel
> 3.0. There are so many people started using Camel 2.x now so we should
> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
> 2.x but now on JDK 1.6 and with X other internal upgrades.
>
> Okay my first cup of coffee is ready, so beware this mail was written
> before I got "my first fix".
>
>
>
> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea<[hidden email]>  wrote:
>> I changed the thread name to [discuss].
>>
>> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.
>>
>> What I'd propose Christian is to add your proposal to the roadmap [1]. I will do the same for the dsl idea. There at least 2 ideas for dsl's built on top of the camel dsl (scheduling and debugging) that make me even more interested in coming up with a better solution.
>>
>> Once we get some traction on the main refactoring ideas I'd suggest starting one (or more) branches and start hacking, because there's not a whole lot of time left if we want to meet our target.
>>
>> Cheers,
>> Hadrian
>>
>> [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>
>>
>>
>> On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>
>>> Hi all,
>>>
>>> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>>>
>>> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>>>
>>> "The camel components should know as little as possible about camel core"
>>>
>>> "The classes needed to setup camel should be separate from the things needed at run time"
>>>
>>> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>>>
>>> So my idea would be to split camel-core into three parts:
>>>
>>> api, builder, impl
>>>
>>> These should be structured in a way that these big building blocks do not have cyclic dependencies. Any other cycles can be ignored in this step.
>>>
>>> As allowed depdencies I propose ( "->" means may use, depends on):
>>>
>>> * ->  api
>>> end user config ->  builder
>>> builder ->  impl
>>>
>>> I think the first thing we should do is to discuss and reach a consensus about a basic architecure and rules like above. Then the next step is to assign each package of core to one of the basic parts. Then the next step is to resolve cycles between the parts.
>>>
>>> What do you think about these ideas?
>>>
>>> Thanks
>>>
>>> Christian
>>>
>>> Christian Schneider
>>> Informationsverarbeitung
>>> Business Solutions
>>> Handel und Dispatching
>>>
>>> Tel : +49-(0)721-63-15482
>>>
>>> EnBW Systeme Infrastruktur Support GmbH
>>> Sitz der Gesellschaft: Karlsruhe
>>> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>>
>>>
>>>
>>
>
>

--
----
http://www.liquid-reality.de

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

dkulp@apache.org
In reply to this post by Claus Ibsen-2
On Tuesday 19 October 2010 12:23:27 am Claus Ibsen wrote:
> Hi
>
> I think the idea is really great, but I think the timing for this is
> *not* the right spot.


Why not?

>
> And by saying that I mean the goal of Camel 3.0 is to have a short
> development cycle (not like 2.0 which took a long time).
> And as a minimum (IMHO):
> - To upgrade to JDK 1.6+,
> - Spring 3.0+,
> - To optimize the router internally,
> - And to switch to slf4j logger (*)
> - Keeping backwards compatibility as much as possible with 2.x is paramount

Umm..  Everything listed there could go into a 2.6 release.   I don't see any
reason for that to be what defines a 3.0 release.

If we are going to have a 3.0, lets get the work done that would need to be
done to provide a stable platform for the next year or so and provides the
API's and feature that are requried for the various new and exciting ideas
people are proposing.

Dan


> Switching to slf4j instead of commons logging, allows us to use the
> MDC logging feature.
> This allows us to push information to the logs such as message id,
> transaction id etc. which can more easily correlate logs, not only
> with Camel alone, but also with other projects such as ActiveMQ, SMX
> etc.
>
>
> On top of that we now have many 3rd party projects which integrate out
> of the box with Camel, so changing the package structure in camel-core
> will break their integration. Which means they may not take up the
> effort to support both Camel 2.x/3.x.
>
> However I do think we should take the effort and pick up the low
> hanging fruits. I am sure there could be a couple of tangles which can
> be identified and fixed in Camel 3.0, without breaking backwards
> compatibility.
>
> I think doing this is something for Camel 4 or 5 or 6 (or whatever
> future version it may be) when or if we change to use Scala and use
> some other framework as foundation. There are cool stuff being
> developed for ActiveMQ 6 which are potential as a backbone for route
> messages. And it has a much better threading model which Camel can
> benefit as well.
>
> Anyway practical works beats theory, so setting up a branch in the
> sandbox to do experiments is a great idea.
>
> But its very important that we keep backwards compatibility with Camel
> 3.0. There are so many people started using Camel 2.x now so we should
> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
> 2.x but now on JDK 1.6 and with X other internal upgrades.
>
> Okay my first cup of coffee is ready, so beware this mail was written
> before I got "my first fix".
>
> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
> > I changed the thread name to [discuss].
> >
> > I like that idea and it's something we contemplated in the past. This
> > will bring back the idea of getting the dsl out of core as well.
> >
> > What I'd propose Christian is to add your proposal to the roadmap [1]. I
> > will do the same for the dsl idea. There at least 2 ideas for dsl's
> > built on top of the camel dsl (scheduling and debugging) that make me
> > even more interested in coming up with a better solution.
> >
> > Once we get some traction on the main refactoring ideas I'd suggest
> > starting one (or more) branches and start hacking, because there's not a
> > whole lot of time left if we want to meet our target.
> >
> > Cheers,
> > Hadrian
> >
> > [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
> >
> > On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
> >> Hi all,
> >>
> >> I will have some free time in december as I am changing my employer. So
> >> I am planning to work a little on some architectural improvements for
> >> camel 3.0.0. As these things are very critical to the stability of
> >> camel I would like to get feedback before I start any substantial work.
> >>
> >> As you surely know currently camel-core is quite tangled. So it is quite
> >> difficult where to start. Some time ago I proposed some improvements to
> >> simply reduce those dependency cycles. As I now know a lot more about
> >> camel I think that this simple aproach will not really work. So this
> >> time I want to do this a little more structured. So I start with two
> >> simple goals:
> >>
> >> "The camel components should know as little as possible about camel
> >> core"
> >>
> >> "The classes needed to setup camel should be separate from the things
> >> needed at run time"
> >>
> >> So why should this be important? Currently components depend on
> >> camel-core as a whole and there are no further rules which classes the
> >> components should use and which classes should be private to core. Even
> >> classes from the impl package are needed. So this means that any
> >> refactoring we do in camel core could affect all components. As camel
> >> is growing steadily this can become quite problematic.
> >>
> >> So my idea would be to split camel-core into three parts:
> >>
> >> api, builder, impl
> >>
> >> These should be structured in a way that these big building blocks do
> >> not have cyclic dependencies. Any other cycles can be ignored in this
> >> step.
> >>
> >> As allowed depdencies I propose ( "->" means may use, depends on):
> >>
> >> * -> api
> >> end user config -> builder
> >> builder -> impl
> >>
> >> I think the first thing we should do is to discuss and reach a consensus
> >> about a basic architecure and rules like above. Then the next step is
> >> to assign each package of core to one of the basic parts. Then the next
> >> step is to resolve cycles between the parts.
> >>
> >> What do you think about these ideas?
> >>
> >> Thanks
> >>
> >> Christian
> >>
> >> Christian Schneider
> >> Informationsverarbeitung
> >> Business Solutions
> >> Handel und Dispatching
> >>
> >> Tel : +49-(0)721-63-15482
> >>
> >> EnBW Systeme Infrastruktur Support GmbH
> >> Sitz der Gesellschaft: Karlsruhe
> >> Handelsregister: Amtsgericht Mannheim - HRB 108550
> >> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
> >> Geschäftsführer: Jochen Adenau, Hans-Günther Meier

--
Daniel Kulp
[hidden email]
http://dankulp.com/blog
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

Claus Ibsen-2
On Tue, Oct 19, 2010 at 1:47 PM, Daniel Kulp <[hidden email]> wrote:

> On Tuesday 19 October 2010 12:23:27 am Claus Ibsen wrote:
>> Hi
>>
>> I think the idea is really great, but I think the timing for this is
>> *not* the right spot.
>
>
> Why not?
>
>>
>> And by saying that I mean the goal of Camel 3.0 is to have a short
>> development cycle (not like 2.0 which took a long time).
>> And as a minimum (IMHO):
>> - To upgrade to JDK 1.6+,
>> - Spring 3.0+,
>> - To optimize the router internally,
>> - And to switch to slf4j logger (*)
>> - Keeping backwards compatibility as much as possible with 2.x is paramount
>
> Umm..  Everything listed there could go into a 2.6 release.   I don't see any
> reason for that to be what defines a 3.0 release.
>

Sorry but those changes are major. Changing the minimum supported JDK
platform is.
eg Spring 2.5 still supports JDK 1.4. Only Spring 3.0 is JDK 1.5+ only.

We can't just change that mid stream.

And we dont du usually do so many minor releases as you do with CXF,
eg 2.2.1 -> 2.2.11
At Camel we do 2.0, 2.1, 2.2, 2.3 etc.


> If we are going to have a 3.0, lets get the work done that would need to be
> done to provide a stable platform for the next year or so and provides the
> API's and feature that are requried for the various new and exciting ideas
> people are proposing.
>

Sorry but I think the community want an easy upgrade path for JDK 1.6
/ Spring 3.0 based on the current 2.x architecture.

It works really well, easy to write custom components. Hence why we
got so many now.
And many 3rd party projects have integrated out of the box with Camel.
And we would lose some if we keep changing the architecture.

We should not do a "Tapestry" with the Camel project, and change the
core at each major release.


> Dan
>
>
>> Switching to slf4j instead of commons logging, allows us to use the
>> MDC logging feature.
>> This allows us to push information to the logs such as message id,
>> transaction id etc. which can more easily correlate logs, not only
>> with Camel alone, but also with other projects such as ActiveMQ, SMX
>> etc.
>>
>>
>> On top of that we now have many 3rd party projects which integrate out
>> of the box with Camel, so changing the package structure in camel-core
>> will break their integration. Which means they may not take up the
>> effort to support both Camel 2.x/3.x.
>>
>> However I do think we should take the effort and pick up the low
>> hanging fruits. I am sure there could be a couple of tangles which can
>> be identified and fixed in Camel 3.0, without breaking backwards
>> compatibility.
>>
>> I think doing this is something for Camel 4 or 5 or 6 (or whatever
>> future version it may be) when or if we change to use Scala and use
>> some other framework as foundation. There are cool stuff being
>> developed for ActiveMQ 6 which are potential as a backbone for route
>> messages. And it has a much better threading model which Camel can
>> benefit as well.
>>
>> Anyway practical works beats theory, so setting up a branch in the
>> sandbox to do experiments is a great idea.
>>
>> But its very important that we keep backwards compatibility with Camel
>> 3.0. There are so many people started using Camel 2.x now so we should
>> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
>> 2.x but now on JDK 1.6 and with X other internal upgrades.
>>
>> Okay my first cup of coffee is ready, so beware this mail was written
>> before I got "my first fix".
>>
>> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
>> > I changed the thread name to [discuss].
>> >
>> > I like that idea and it's something we contemplated in the past. This
>> > will bring back the idea of getting the dsl out of core as well.
>> >
>> > What I'd propose Christian is to add your proposal to the roadmap [1]. I
>> > will do the same for the dsl idea. There at least 2 ideas for dsl's
>> > built on top of the camel dsl (scheduling and debugging) that make me
>> > even more interested in coming up with a better solution.
>> >
>> > Once we get some traction on the main refactoring ideas I'd suggest
>> > starting one (or more) branches and start hacking, because there's not a
>> > whole lot of time left if we want to meet our target.
>> >
>> > Cheers,
>> > Hadrian
>> >
>> > [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>> >
>> > On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>> >> Hi all,
>> >>
>> >> I will have some free time in december as I am changing my employer. So
>> >> I am planning to work a little on some architectural improvements for
>> >> camel 3.0.0. As these things are very critical to the stability of
>> >> camel I would like to get feedback before I start any substantial work.
>> >>
>> >> As you surely know currently camel-core is quite tangled. So it is quite
>> >> difficult where to start. Some time ago I proposed some improvements to
>> >> simply reduce those dependency cycles. As I now know a lot more about
>> >> camel I think that this simple aproach will not really work. So this
>> >> time I want to do this a little more structured. So I start with two
>> >> simple goals:
>> >>
>> >> "The camel components should know as little as possible about camel
>> >> core"
>> >>
>> >> "The classes needed to setup camel should be separate from the things
>> >> needed at run time"
>> >>
>> >> So why should this be important? Currently components depend on
>> >> camel-core as a whole and there are no further rules which classes the
>> >> components should use and which classes should be private to core. Even
>> >> classes from the impl package are needed. So this means that any
>> >> refactoring we do in camel core could affect all components. As camel
>> >> is growing steadily this can become quite problematic.
>> >>
>> >> So my idea would be to split camel-core into three parts:
>> >>
>> >> api, builder, impl
>> >>
>> >> These should be structured in a way that these big building blocks do
>> >> not have cyclic dependencies. Any other cycles can be ignored in this
>> >> step.
>> >>
>> >> As allowed depdencies I propose ( "->" means may use, depends on):
>> >>
>> >> * -> api
>> >> end user config -> builder
>> >> builder -> impl
>> >>
>> >> I think the first thing we should do is to discuss and reach a consensus
>> >> about a basic architecure and rules like above. Then the next step is
>> >> to assign each package of core to one of the basic parts. Then the next
>> >> step is to resolve cycles between the parts.
>> >>
>> >> What do you think about these ideas?
>> >>
>> >> Thanks
>> >>
>> >> Christian
>> >>
>> >> Christian Schneider
>> >> Informationsverarbeitung
>> >> Business Solutions
>> >> Handel und Dispatching
>> >>
>> >> Tel : +49-(0)721-63-15482
>> >>
>> >> EnBW Systeme Infrastruktur Support GmbH
>> >> Sitz der Gesellschaft: Karlsruhe
>> >> Handelsregister: Amtsgericht Mannheim - HRB 108550
>> >> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>> >> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>
> --
> Daniel Kulp
> [hidden email]
> http://dankulp.com/blog
>



--
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

AW: [DISCUSS] Some thoughts about the architecture of camel

Schneider Christian
In reply to this post by dkulp@apache.org
+1 For having a 2.6 release like Dan proposed. As Claus said the new release should be as backwards compatible as possbile so I think it is not necessary to increase the major release. Q1 2011 could be a good date for this release.

+1 For doing the more advanced stuff like architecture of camel-core like I proposed in the 3.0 release. As the 2.6 release would have the short term stuff the release date could even be after Q1 2011 in my opinion.

Best Regards

Christian
 




Christian Schneider
Informationsverarbeitung
Business Solutions
Handel und Dispatching

Tel : +49-(0)721-63-15482

EnBW Systeme Infrastruktur Support GmbH
Sitz der Gesellschaft: Karlsruhe
Handelsregister: Amtsgericht Mannheim ­ HRB 108550
Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
Geschäftsführer: Jochen Adenau, Hans-Günther Meier


-----Ursprüngliche Nachricht-----
Von: Daniel Kulp [mailto:[hidden email]]
Gesendet: Dienstag, 19. Oktober 2010 13:47
An: [hidden email]
Cc: Claus Ibsen
Betreff: Re: [DISCUSS] Some thoughts about the architecture of camel

On Tuesday 19 October 2010 12:23:27 am Claus Ibsen wrote:
> Hi
>
> I think the idea is really great, but I think the timing for this is
> *not* the right spot.


Why not?

>
> And by saying that I mean the goal of Camel 3.0 is to have a short
> development cycle (not like 2.0 which took a long time).
> And as a minimum (IMHO):
> - To upgrade to JDK 1.6+,
> - Spring 3.0+,
> - To optimize the router internally,
> - And to switch to slf4j logger (*)
> - Keeping backwards compatibility as much as possible with 2.x is paramount

Umm..  Everything listed there could go into a 2.6 release.   I don't see any
reason for that to be what defines a 3.0 release.

If we are going to have a 3.0, lets get the work done that would need to be
done to provide a stable platform for the next year or so and provides the
API's and feature that are requried for the various new and exciting ideas
people are proposing.

Dan


> Switching to slf4j instead of commons logging, allows us to use the
> MDC logging feature.
> This allows us to push information to the logs such as message id,
> transaction id etc. which can more easily correlate logs, not only
> with Camel alone, but also with other projects such as ActiveMQ, SMX
> etc.
>
>
> On top of that we now have many 3rd party projects which integrate out
> of the box with Camel, so changing the package structure in camel-core
> will break their integration. Which means they may not take up the
> effort to support both Camel 2.x/3.x.
>
> However I do think we should take the effort and pick up the low
> hanging fruits. I am sure there could be a couple of tangles which can
> be identified and fixed in Camel 3.0, without breaking backwards
> compatibility.
>
> I think doing this is something for Camel 4 or 5 or 6 (or whatever
> future version it may be) when or if we change to use Scala and use
> some other framework as foundation. There are cool stuff being
> developed for ActiveMQ 6 which are potential as a backbone for route
> messages. And it has a much better threading model which Camel can
> benefit as well.
>
> Anyway practical works beats theory, so setting up a branch in the
> sandbox to do experiments is a great idea.
>
> But its very important that we keep backwards compatibility with Camel
> 3.0. There are so many people started using Camel 2.x now so we should
> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
> 2.x but now on JDK 1.6 and with X other internal upgrades.
>
> Okay my first cup of coffee is ready, so beware this mail was written
> before I got "my first fix".
>
> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
> > I changed the thread name to [discuss].
> >
> > I like that idea and it's something we contemplated in the past. This
> > will bring back the idea of getting the dsl out of core as well.
> >
> > What I'd propose Christian is to add your proposal to the roadmap [1]. I
> > will do the same for the dsl idea. There at least 2 ideas for dsl's
> > built on top of the camel dsl (scheduling and debugging) that make me
> > even more interested in coming up with a better solution.
> >
> > Once we get some traction on the main refactoring ideas I'd suggest
> > starting one (or more) branches and start hacking, because there's not a
> > whole lot of time left if we want to meet our target.
> >
> > Cheers,
> > Hadrian
> >
> > [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
> >
> > On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
> >> Hi all,
> >>
> >> I will have some free time in december as I am changing my employer. So
> >> I am planning to work a little on some architectural improvements for
> >> camel 3.0.0. As these things are very critical to the stability of
> >> camel I would like to get feedback before I start any substantial work.
> >>
> >> As you surely know currently camel-core is quite tangled. So it is quite
> >> difficult where to start. Some time ago I proposed some improvements to
> >> simply reduce those dependency cycles. As I now know a lot more about
> >> camel I think that this simple aproach will not really work. So this
> >> time I want to do this a little more structured. So I start with two
> >> simple goals:
> >>
> >> "The camel components should know as little as possible about camel
> >> core"
> >>
> >> "The classes needed to setup camel should be separate from the things
> >> needed at run time"
> >>
> >> So why should this be important? Currently components depend on
> >> camel-core as a whole and there are no further rules which classes the
> >> components should use and which classes should be private to core. Even
> >> classes from the impl package are needed. So this means that any
> >> refactoring we do in camel core could affect all components. As camel
> >> is growing steadily this can become quite problematic.
> >>
> >> So my idea would be to split camel-core into three parts:
> >>
> >> api, builder, impl
> >>
> >> These should be structured in a way that these big building blocks do
> >> not have cyclic dependencies. Any other cycles can be ignored in this
> >> step.
> >>
> >> As allowed depdencies I propose ( "->" means may use, depends on):
> >>
> >> * -> api
> >> end user config -> builder
> >> builder -> impl
> >>
> >> I think the first thing we should do is to discuss and reach a consensus
> >> about a basic architecure and rules like above. Then the next step is
> >> to assign each package of core to one of the basic parts. Then the next
> >> step is to resolve cycles between the parts.
> >>
> >> What do you think about these ideas?
> >>
> >> Thanks
> >>
> >> Christian
> >>
> >> Christian Schneider
> >> Informationsverarbeitung
> >> Business Solutions
> >> Handel und Dispatching
> >>
> >> Tel : +49-(0)721-63-15482
> >>
> >> EnBW Systeme Infrastruktur Support GmbH
> >> Sitz der Gesellschaft: Karlsruhe
> >> Handelsregister: Amtsgericht Mannheim - HRB 108550
> >> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
> >> Geschäftsführer: Jochen Adenau, Hans-Günther Meier

--
Daniel Kulp
[hidden email]
http://dankulp.com/blog
Reply | Threaded
Open this post in threaded view
|

AW: [DISCUSS] Some thoughts about the architecture of camel

Schneider Christian
In reply to this post by Claus Ibsen-2
If you consider spring and jdk a (somewhat) breaking change then a 3.0 version is the right thing of course. I don´t care too much if this version is called 2.6 or 3.0 ;-)

In this case I would like to have the more advanced stuff in the 4.0 version. Do you think that is possible?

Best Regards

Christian


Christian Schneider
Informationsverarbeitung
Business Solutions
Handel und Dispatching

Tel : +49-(0)721-63-15482

EnBW Systeme Infrastruktur Support GmbH
Sitz der Gesellschaft: Karlsruhe
Handelsregister: Amtsgericht Mannheim ­ HRB 108550
Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
Geschäftsführer: Jochen Adenau, Hans-Günther Meier


-----Ursprüngliche Nachricht-----
Von: Claus Ibsen [mailto:[hidden email]]
Gesendet: Dienstag, 19. Oktober 2010 14:22
An: [hidden email]
Betreff: Re: [DISCUSS] Some thoughts about the architecture of camel

On Tue, Oct 19, 2010 at 1:47 PM, Daniel Kulp <[hidden email]> wrote:

> On Tuesday 19 October 2010 12:23:27 am Claus Ibsen wrote:
>> Hi
>>
>> I think the idea is really great, but I think the timing for this is
>> *not* the right spot.
>
>
> Why not?
>
>>
>> And by saying that I mean the goal of Camel 3.0 is to have a short
>> development cycle (not like 2.0 which took a long time).
>> And as a minimum (IMHO):
>> - To upgrade to JDK 1.6+,
>> - Spring 3.0+,
>> - To optimize the router internally,
>> - And to switch to slf4j logger (*)
>> - Keeping backwards compatibility as much as possible with 2.x is paramount
>
> Umm..  Everything listed there could go into a 2.6 release.   I don't see any
> reason for that to be what defines a 3.0 release.
>

Sorry but those changes are major. Changing the minimum supported JDK
platform is.
eg Spring 2.5 still supports JDK 1.4. Only Spring 3.0 is JDK 1.5+ only.

We can't just change that mid stream.

And we dont du usually do so many minor releases as you do with CXF,
eg 2.2.1 -> 2.2.11
At Camel we do 2.0, 2.1, 2.2, 2.3 etc.


> If we are going to have a 3.0, lets get the work done that would need to be
> done to provide a stable platform for the next year or so and provides the
> API's and feature that are requried for the various new and exciting ideas
> people are proposing.
>

Sorry but I think the community want an easy upgrade path for JDK 1.6
/ Spring 3.0 based on the current 2.x architecture.

It works really well, easy to write custom components. Hence why we
got so many now.
And many 3rd party projects have integrated out of the box with Camel.
And we would lose some if we keep changing the architecture.

We should not do a "Tapestry" with the Camel project, and change the
core at each major release.


> Dan
>
>
>> Switching to slf4j instead of commons logging, allows us to use the
>> MDC logging feature.
>> This allows us to push information to the logs such as message id,
>> transaction id etc. which can more easily correlate logs, not only
>> with Camel alone, but also with other projects such as ActiveMQ, SMX
>> etc.
>>
>>
>> On top of that we now have many 3rd party projects which integrate out
>> of the box with Camel, so changing the package structure in camel-core
>> will break their integration. Which means they may not take up the
>> effort to support both Camel 2.x/3.x.
>>
>> However I do think we should take the effort and pick up the low
>> hanging fruits. I am sure there could be a couple of tangles which can
>> be identified and fixed in Camel 3.0, without breaking backwards
>> compatibility.
>>
>> I think doing this is something for Camel 4 or 5 or 6 (or whatever
>> future version it may be) when or if we change to use Scala and use
>> some other framework as foundation. There are cool stuff being
>> developed for ActiveMQ 6 which are potential as a backbone for route
>> messages. And it has a much better threading model which Camel can
>> benefit as well.
>>
>> Anyway practical works beats theory, so setting up a branch in the
>> sandbox to do experiments is a great idea.
>>
>> But its very important that we keep backwards compatibility with Camel
>> 3.0. There are so many people started using Camel 2.x now so we should
>> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
>> 2.x but now on JDK 1.6 and with X other internal upgrades.
>>
>> Okay my first cup of coffee is ready, so beware this mail was written
>> before I got "my first fix".
>>
>> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
>> > I changed the thread name to [discuss].
>> >
>> > I like that idea and it's something we contemplated in the past. This
>> > will bring back the idea of getting the dsl out of core as well.
>> >
>> > What I'd propose Christian is to add your proposal to the roadmap [1]. I
>> > will do the same for the dsl idea. There at least 2 ideas for dsl's
>> > built on top of the camel dsl (scheduling and debugging) that make me
>> > even more interested in coming up with a better solution.
>> >
>> > Once we get some traction on the main refactoring ideas I'd suggest
>> > starting one (or more) branches and start hacking, because there's not a
>> > whole lot of time left if we want to meet our target.
>> >
>> > Cheers,
>> > Hadrian
>> >
>> > [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>> >
>> > On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>> >> Hi all,
>> >>
>> >> I will have some free time in december as I am changing my employer. So
>> >> I am planning to work a little on some architectural improvements for
>> >> camel 3.0.0. As these things are very critical to the stability of
>> >> camel I would like to get feedback before I start any substantial work.
>> >>
>> >> As you surely know currently camel-core is quite tangled. So it is quite
>> >> difficult where to start. Some time ago I proposed some improvements to
>> >> simply reduce those dependency cycles. As I now know a lot more about
>> >> camel I think that this simple aproach will not really work. So this
>> >> time I want to do this a little more structured. So I start with two
>> >> simple goals:
>> >>
>> >> "The camel components should know as little as possible about camel
>> >> core"
>> >>
>> >> "The classes needed to setup camel should be separate from the things
>> >> needed at run time"
>> >>
>> >> So why should this be important? Currently components depend on
>> >> camel-core as a whole and there are no further rules which classes the
>> >> components should use and which classes should be private to core. Even
>> >> classes from the impl package are needed. So this means that any
>> >> refactoring we do in camel core could affect all components. As camel
>> >> is growing steadily this can become quite problematic.
>> >>
>> >> So my idea would be to split camel-core into three parts:
>> >>
>> >> api, builder, impl
>> >>
>> >> These should be structured in a way that these big building blocks do
>> >> not have cyclic dependencies. Any other cycles can be ignored in this
>> >> step.
>> >>
>> >> As allowed depdencies I propose ( "->" means may use, depends on):
>> >>
>> >> * -> api
>> >> end user config -> builder
>> >> builder -> impl
>> >>
>> >> I think the first thing we should do is to discuss and reach a consensus
>> >> about a basic architecure and rules like above. Then the next step is
>> >> to assign each package of core to one of the basic parts. Then the next
>> >> step is to resolve cycles between the parts.
>> >>
>> >> What do you think about these ideas?
>> >>
>> >> Thanks
>> >>
>> >> Christian
>> >>
>> >> Christian Schneider
>> >> Informationsverarbeitung
>> >> Business Solutions
>> >> Handel und Dispatching
>> >>
>> >> Tel : +49-(0)721-63-15482
>> >>
>> >> EnBW Systeme Infrastruktur Support GmbH
>> >> Sitz der Gesellschaft: Karlsruhe
>> >> Handelsregister: Amtsgericht Mannheim - HRB 108550
>> >> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>> >> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>
> --
> Daniel Kulp
> [hidden email]
> http://dankulp.com/blog
>



--
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus
Reply | Threaded
Open this post in threaded view
|

Re: Some thoughts about the architecture of camel

jstrachan
In reply to this post by Schneider Christian
On 18 October 2010 10:43, Schneider Christian
<[hidden email]> wrote:

> Hi all,
>
> I will have some free time in december as I am changing my employer. So I am planning to work a little on some architectural improvements for camel 3.0.0. As these things are very critical to the stability of camel I would like to get feedback before I start any substantial work.
>
> As you surely know currently camel-core is quite tangled. So it is quite difficult where to start. Some time ago I proposed some improvements to simply reduce those dependency cycles. As I now know a lot more about camel I think that this simple aproach will not really work. So this time I want to do this a little more structured. So I start with two simple goals:
>
> "The camel components should know as little as possible about camel core"
>
> "The classes needed to setup camel should be separate from the things needed at run time"
>
> So why should this be important? Currently components depend on camel-core as a whole and there are no further rules which classes the components should use and which classes should be private to core. Even classes from the impl package are needed. So this means that any refactoring we do in camel core could affect all components. As camel is growing steadily this can become quite problematic.
>
> So my idea would be to split camel-core into three parts:
>
> api, builder, impl

What benefits do you see for end users and component developers having
to depend on at least 3 jars rather than one?

One of the reasons I like camel-core as it is; its nice and simple and
just works. You can build & run routes with some simple components
using the single camel-core jar. (Just add camel-test to do testing).

Sure there's some cyclic package dependencies. Given the large number
of use cases in Camel (route design, testing, JAXB model, Java DSL and
base set of components) its kinda hard to totally avoid those while
having convention over configuration, decent defaults etc.

I value ease of use for end users & component developers and backwards
compatibility over cyclic dependency metrics any day :)

--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

jstrachan
In reply to this post by hadrian
On 18 October 2010 18:28, Hadrian Zbarcea <[hidden email]> wrote:
> I changed the thread name to [discuss].
>
> I like that idea and it's something we contemplated in the past. This will bring back the idea of getting the dsl out of core as well.

What benefits does that have BTW? IMHO more dependencies for no actual
gain has negative value.

FWIW early camel had Java DSL separate from the JAXB model and it was
a maintenance nightmare which is why we unified it into the same
model.

--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/
Reply | Threaded
Open this post in threaded view
|

Re: AW: [DISCUSS] Some thoughts about the architecture of camel

dkulp@apache.org
In reply to this post by Schneider Christian
On Tuesday 19 October 2010 8:22:11 am Schneider Christian wrote:
> +1 For having a 2.6 release like Dan proposed. As Claus said the new
> release should be as backwards compatible as possbile so I think it is not
> necessary to increase the major release. Q1 2011 could be a good date for
> this release.

I was actually thinking Dec for this if possible.   I'm strongly considering
making 2.2.12 the last 2.2.x release for CXF that I do which would be
December.   Need to present that to the CXF community, probably later today,
but in any case, getting folks off of 2.2.x sooner is quite important.


> +1 For doing the more advanced stuff like architecture of camel-core like I
> proposed in the 3.0 release. As the 2.6 release would have the short term
> stuff the release date could even be after Q1 2011 in my opinion.

Late Q1 or early Q2 or so.   That's 5-6 months which would be a reasonable dev
effort for 3.0.  IMO.

Dan



>
> Best Regards
>
> Christian
>
>
>
>
>
> Christian Schneider
> Informationsverarbeitung
> Business Solutions
> Handel und Dispatching
>
> Tel : +49-(0)721-63-15482
>
> EnBW Systeme Infrastruktur Support GmbH
> Sitz der Gesellschaft: Karlsruhe
> Handelsregister: Amtsgericht Mannheim ­ HRB 108550
> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>
>
> -----Ursprüngliche Nachricht-----
> Von: Daniel Kulp [mailto:[hidden email]]
> Gesendet: Dienstag, 19. Oktober 2010 13:47
> An: [hidden email]
> Cc: Claus Ibsen
> Betreff: Re: [DISCUSS] Some thoughts about the architecture of camel
>
> On Tuesday 19 October 2010 12:23:27 am Claus Ibsen wrote:
> > Hi
> >
> > I think the idea is really great, but I think the timing for this is
> > *not* the right spot.
>
> Why not?
>
> > And by saying that I mean the goal of Camel 3.0 is to have a short
> > development cycle (not like 2.0 which took a long time).
> > And as a minimum (IMHO):
> > - To upgrade to JDK 1.6+,
> > - Spring 3.0+,
> > - To optimize the router internally,
> > - And to switch to slf4j logger (*)
> > - Keeping backwards compatibility as much as possible with 2.x is
> > paramount
>
> Umm..  Everything listed there could go into a 2.6 release.   I don't see
> any reason for that to be what defines a 3.0 release.
>
> If we are going to have a 3.0, lets get the work done that would need to be
> done to provide a stable platform for the next year or so and provides the
> API's and feature that are requried for the various new and exciting ideas
> people are proposing.
>
> Dan
>
> > Switching to slf4j instead of commons logging, allows us to use the
> > MDC logging feature.
> > This allows us to push information to the logs such as message id,
> > transaction id etc. which can more easily correlate logs, not only
> > with Camel alone, but also with other projects such as ActiveMQ, SMX
> > etc.
> >
> >
> > On top of that we now have many 3rd party projects which integrate out
> > of the box with Camel, so changing the package structure in camel-core
> > will break their integration. Which means they may not take up the
> > effort to support both Camel 2.x/3.x.
> >
> > However I do think we should take the effort and pick up the low
> > hanging fruits. I am sure there could be a couple of tangles which can
> > be identified and fixed in Camel 3.0, without breaking backwards
> > compatibility.
> >
> > I think doing this is something for Camel 4 or 5 or 6 (or whatever
> > future version it may be) when or if we change to use Scala and use
> > some other framework as foundation. There are cool stuff being
> > developed for ActiveMQ 6 which are potential as a backbone for route
> > messages. And it has a much better threading model which Camel can
> > benefit as well.
> >
> > Anyway practical works beats theory, so setting up a branch in the
> > sandbox to do experiments is a great idea.
> >
> > But its very important that we keep backwards compatibility with Camel
> > 3.0. There are so many people started using Camel 2.x now so we should
> > keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
> > 2.x but now on JDK 1.6 and with X other internal upgrades.
> >
> > Okay my first cup of coffee is ready, so beware this mail was written
> > before I got "my first fix".
> >
> > On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]>
wrote:

> > > I changed the thread name to [discuss].
> > >
> > > I like that idea and it's something we contemplated in the past. This
> > > will bring back the idea of getting the dsl out of core as well.
> > >
> > > What I'd propose Christian is to add your proposal to the roadmap [1].
> > > I will do the same for the dsl idea. There at least 2 ideas for dsl's
> > > built on top of the camel dsl (scheduling and debugging) that make me
> > > even more interested in coming up with a better solution.
> > >
> > > Once we get some traction on the main refactoring ideas I'd suggest
> > > starting one (or more) branches and start hacking, because there's not
> > > a whole lot of time left if we want to meet our target.
> > >
> > > Cheers,
> > > Hadrian
> > >
> > > [1]
> > > https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
> > >
> > > On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
> > >> Hi all,
> > >>
> > >> I will have some free time in december as I am changing my employer.
> > >> So I am planning to work a little on some architectural improvements
> > >> for camel 3.0.0. As these things are very critical to the stability
> > >> of camel I would like to get feedback before I start any substantial
> > >> work.
> > >>
> > >> As you surely know currently camel-core is quite tangled. So it is
> > >> quite difficult where to start. Some time ago I proposed some
> > >> improvements to simply reduce those dependency cycles. As I now know
> > >> a lot more about camel I think that this simple aproach will not
> > >> really work. So this time I want to do this a little more structured.
> > >> So I start with two simple goals:
> > >>
> > >> "The camel components should know as little as possible about camel
> > >> core"
> > >>
> > >> "The classes needed to setup camel should be separate from the things
> > >> needed at run time"
> > >>
> > >> So why should this be important? Currently components depend on
> > >> camel-core as a whole and there are no further rules which classes the
> > >> components should use and which classes should be private to core.
> > >> Even classes from the impl package are needed. So this means that any
> > >> refactoring we do in camel core could affect all components. As camel
> > >> is growing steadily this can become quite problematic.
> > >>
> > >> So my idea would be to split camel-core into three parts:
> > >>
> > >> api, builder, impl
> > >>
> > >> These should be structured in a way that these big building blocks do
> > >> not have cyclic dependencies. Any other cycles can be ignored in this
> > >> step.
> > >>
> > >> As allowed depdencies I propose ( "->" means may use, depends on):
> > >>
> > >> * -> api
> > >> end user config -> builder
> > >> builder -> impl
> > >>
> > >> I think the first thing we should do is to discuss and reach a
> > >> consensus about a basic architecure and rules like above. Then the
> > >> next step is to assign each package of core to one of the basic
> > >> parts. Then the next step is to resolve cycles between the parts.
> > >>
> > >> What do you think about these ideas?
> > >>
> > >> Thanks
> > >>
> > >> Christian
> > >>
> > >> Christian Schneider
> > >> Informationsverarbeitung
> > >> Business Solutions
> > >> Handel und Dispatching
> > >>
> > >> Tel : +49-(0)721-63-15482
> > >>
> > >> EnBW Systeme Infrastruktur Support GmbH
> > >> Sitz der Gesellschaft: Karlsruhe
> > >> Handelsregister: Amtsgericht Mannheim - HRB 108550
> > >> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
> > >> Geschäftsführer: Jochen Adenau, Hans-Günther Meier

--
Daniel Kulp
[hidden email]
http://dankulp.com/blog
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Some thoughts about the architecture of camel

jstrachan
In reply to this post by Schneider Christian
Changing Camel to 3.x matches the Spring naming conventions sounds a
good enough idea in itself; but adding in the JDK 5->6 change too,
then 3.0 sounds like a no brainer to me.

Then Camel 2.x works with Spring 2.x and Camel 3.x works with Spring
3.x; nice and easy to remember.

If folks want to experiment with a non-backwards compatible 4.x
version of Camel we can always have parallel experimental branches
where folks can play; as progress is made we can weigh up the
strengths and weaknesses of the different refactorings folks
experiment with.


On 19 October 2010 13:26, Schneider Christian
<[hidden email]> wrote:

> If you consider spring and jdk a (somewhat) breaking change then a 3.0 version is the right thing of course. I don´t care too much if this version is called 2.6 or 3.0 ;-)
>
> In this case I would like to have the more advanced stuff in the 4.0 version. Do you think that is possible?
>
> Best Regards
>
> Christian
>
>
> Christian Schneider
> Informationsverarbeitung
> Business Solutions
> Handel und Dispatching
>
> Tel : +49-(0)721-63-15482
>
> EnBW Systeme Infrastruktur Support GmbH
> Sitz der Gesellschaft: Karlsruhe
> Handelsregister: Amtsgericht Mannheim ­ HRB 108550
> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>
>
> -----Ursprüngliche Nachricht-----
> Von: Claus Ibsen [mailto:[hidden email]]
> Gesendet: Dienstag, 19. Oktober 2010 14:22
> An: [hidden email]
> Betreff: Re: [DISCUSS] Some thoughts about the architecture of camel
>
> On Tue, Oct 19, 2010 at 1:47 PM, Daniel Kulp <[hidden email]> wrote:
>> On Tuesday 19 October 2010 12:23:27 am Claus Ibsen wrote:
>>> Hi
>>>
>>> I think the idea is really great, but I think the timing for this is
>>> *not* the right spot.
>>
>>
>> Why not?
>>
>>>
>>> And by saying that I mean the goal of Camel 3.0 is to have a short
>>> development cycle (not like 2.0 which took a long time).
>>> And as a minimum (IMHO):
>>> - To upgrade to JDK 1.6+,
>>> - Spring 3.0+,
>>> - To optimize the router internally,
>>> - And to switch to slf4j logger (*)
>>> - Keeping backwards compatibility as much as possible with 2.x is paramount
>>
>> Umm..  Everything listed there could go into a 2.6 release.   I don't see any
>> reason for that to be what defines a 3.0 release.
>>
>
> Sorry but those changes are major. Changing the minimum supported JDK
> platform is.
> eg Spring 2.5 still supports JDK 1.4. Only Spring 3.0 is JDK 1.5+ only.
>
> We can't just change that mid stream.
>
> And we dont du usually do so many minor releases as you do with CXF,
> eg 2.2.1 -> 2.2.11
> At Camel we do 2.0, 2.1, 2.2, 2.3 etc.
>
>
>> If we are going to have a 3.0, lets get the work done that would need to be
>> done to provide a stable platform for the next year or so and provides the
>> API's and feature that are requried for the various new and exciting ideas
>> people are proposing.
>>
>
> Sorry but I think the community want an easy upgrade path for JDK 1.6
> / Spring 3.0 based on the current 2.x architecture.
>
> It works really well, easy to write custom components. Hence why we
> got so many now.
> And many 3rd party projects have integrated out of the box with Camel.
> And we would lose some if we keep changing the architecture.
>
> We should not do a "Tapestry" with the Camel project, and change the
> core at each major release.
>
>
>> Dan
>>
>>
>>> Switching to slf4j instead of commons logging, allows us to use the
>>> MDC logging feature.
>>> This allows us to push information to the logs such as message id,
>>> transaction id etc. which can more easily correlate logs, not only
>>> with Camel alone, but also with other projects such as ActiveMQ, SMX
>>> etc.
>>>
>>>
>>> On top of that we now have many 3rd party projects which integrate out
>>> of the box with Camel, so changing the package structure in camel-core
>>> will break their integration. Which means they may not take up the
>>> effort to support both Camel 2.x/3.x.
>>>
>>> However I do think we should take the effort and pick up the low
>>> hanging fruits. I am sure there could be a couple of tangles which can
>>> be identified and fixed in Camel 3.0, without breaking backwards
>>> compatibility.
>>>
>>> I think doing this is something for Camel 4 or 5 or 6 (or whatever
>>> future version it may be) when or if we change to use Scala and use
>>> some other framework as foundation. There are cool stuff being
>>> developed for ActiveMQ 6 which are potential as a backbone for route
>>> messages. And it has a much better threading model which Camel can
>>> benefit as well.
>>>
>>> Anyway practical works beats theory, so setting up a branch in the
>>> sandbox to do experiments is a great idea.
>>>
>>> But its very important that we keep backwards compatibility with Camel
>>> 3.0. There are so many people started using Camel 2.x now so we should
>>> keep them happy with an easy upgrade path. Eg Camel 3.0 is just like
>>> 2.x but now on JDK 1.6 and with X other internal upgrades.
>>>
>>> Okay my first cup of coffee is ready, so beware this mail was written
>>> before I got "my first fix".
>>>
>>> On Mon, Oct 18, 2010 at 7:28 PM, Hadrian Zbarcea <[hidden email]> wrote:
>>> > I changed the thread name to [discuss].
>>> >
>>> > I like that idea and it's something we contemplated in the past. This
>>> > will bring back the idea of getting the dsl out of core as well.
>>> >
>>> > What I'd propose Christian is to add your proposal to the roadmap [1]. I
>>> > will do the same for the dsl idea. There at least 2 ideas for dsl's
>>> > built on top of the camel dsl (scheduling and debugging) that make me
>>> > even more interested in coming up with a better solution.
>>> >
>>> > Once we get some traction on the main refactoring ideas I'd suggest
>>> > starting one (or more) branches and start hacking, because there's not a
>>> > whole lot of time left if we want to meet our target.
>>> >
>>> > Cheers,
>>> > Hadrian
>>> >
>>> > [1] https://cwiki.apache.org/confluence/display/CAMEL/Camel+3.0+-+Roadmap
>>> >
>>> > On Oct 18, 2010, at 5:43 AM, Schneider Christian wrote:
>>> >> Hi all,
>>> >>
>>> >> I will have some free time in december as I am changing my employer. So
>>> >> I am planning to work a little on some architectural improvements for
>>> >> camel 3.0.0. As these things are very critical to the stability of
>>> >> camel I would like to get feedback before I start any substantial work.
>>> >>
>>> >> As you surely know currently camel-core is quite tangled. So it is quite
>>> >> difficult where to start. Some time ago I proposed some improvements to
>>> >> simply reduce those dependency cycles. As I now know a lot more about
>>> >> camel I think that this simple aproach will not really work. So this
>>> >> time I want to do this a little more structured. So I start with two
>>> >> simple goals:
>>> >>
>>> >> "The camel components should know as little as possible about camel
>>> >> core"
>>> >>
>>> >> "The classes needed to setup camel should be separate from the things
>>> >> needed at run time"
>>> >>
>>> >> So why should this be important? Currently components depend on
>>> >> camel-core as a whole and there are no further rules which classes the
>>> >> components should use and which classes should be private to core. Even
>>> >> classes from the impl package are needed. So this means that any
>>> >> refactoring we do in camel core could affect all components. As camel
>>> >> is growing steadily this can become quite problematic.
>>> >>
>>> >> So my idea would be to split camel-core into three parts:
>>> >>
>>> >> api, builder, impl
>>> >>
>>> >> These should be structured in a way that these big building blocks do
>>> >> not have cyclic dependencies. Any other cycles can be ignored in this
>>> >> step.
>>> >>
>>> >> As allowed depdencies I propose ( "->" means may use, depends on):
>>> >>
>>> >> * -> api
>>> >> end user config -> builder
>>> >> builder -> impl
>>> >>
>>> >> I think the first thing we should do is to discuss and reach a consensus
>>> >> about a basic architecure and rules like above. Then the next step is
>>> >> to assign each package of core to one of the basic parts. Then the next
>>> >> step is to resolve cycles between the parts.
>>> >>
>>> >> What do you think about these ideas?
>>> >>
>>> >> Thanks
>>> >>
>>> >> Christian
>>> >>
>>> >> Christian Schneider
>>> >> Informationsverarbeitung
>>> >> Business Solutions
>>> >> Handel und Dispatching
>>> >>
>>> >> Tel : +49-(0)721-63-15482
>>> >>
>>> >> EnBW Systeme Infrastruktur Support GmbH
>>> >> Sitz der Gesellschaft: Karlsruhe
>>> >> Handelsregister: Amtsgericht Mannheim - HRB 108550
>>> >> Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
>>> >> Geschäftsführer: Jochen Adenau, Hans-Günther Meier
>>
>> --
>> Daniel Kulp
>> [hidden email]
>> http://dankulp.com/blog
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>



--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/
Reply | Threaded
Open this post in threaded view
|

AW: Some thoughts about the architecture of camel

Schneider Christian
In reply to this post by jstrachan
Hi James,

it is not absolutely necessary to split the jar into three jars. More important is to have rules that say that a component developer should only depdend on the API part and to check that the internal dependencies do not have cycles between the three logical modules. The only disadvantage of not breaking up camel core into three modules is that maven will not help you in avoiding cycles which would be the case with separate modules. As the rules can be checked with tools like structure 101 this is not too bad though.

I don´t think the cyclic depdencies are only a "metric". They are a real problem when the code grows as you can not understand or change anything isolated. To have three clearly defined parts in camel core that should not have cycles between them is quite reasonable imho.

Especially I think the builders should be separated as they are not needed at runtime. The builder pattern creates many cycles and it confuses people who try to understand the runtime behaviour. Of course I do not speak against builders and the dsl. They are extremely convenient and clear to use for end users and make a good part of the appeal of camel.

So to sum it up I think breaking up camel-core logically is very important. At the same time I understand that ease of use is a value that is probably as important as a clear architecture. The good thing is that I am quite sure that we can achieve both.

Best Regards

Christian


Christian Schneider
Informationsverarbeitung
Business Solutions
Handel und Dispatching

Tel : +49-(0)721-63-15482

EnBW Systeme Infrastruktur Support GmbH
Sitz der Gesellschaft: Karlsruhe
Handelsregister: Amtsgericht Mannheim ­ HRB 108550
Vorsitzender des Aufsichtsrats: Dr. Bernhard Beck
Geschäftsführer: Jochen Adenau, Hans-Günther Meier


-----Ursprüngliche Nachricht-----
Von: James Strachan [mailto:[hidden email]]
Gesendet: Dienstag, 19. Oktober 2010 14:31
An: [hidden email]
Betreff: Re: Some thoughts about the architecture of camel


>
> So my idea would be to split camel-core into three parts:
>
> api, builder, impl

What benefits do you see for end users and component developers having
to depend on at least 3 jars rather than one?

One of the reasons I like camel-core as it is; its nice and simple and
just works. You can build & run routes with some simple components
using the single camel-core jar. (Just add camel-test to do testing).

Sure there's some cyclic package dependencies. Given the large number
of use cases in Camel (route design, testing, JAXB model, Java DSL and
base set of components) its kinda hard to totally avoid those while
having convention over configuration, decent defaults etc.

I value ease of use for end users & component developers and backwards
compatibility over cyclic dependency metrics any day :)

--
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://fusesource.com/
12