[DISCUSS] - Thoughts on Apache Camel 2.18 and towards 3.0

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
45 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] - Thoughts on Apache Camel 2.18 and towards 3.0

Matt Pavlovich
Johan-

I'm a big fan of long winded statements as well, but not following this
one =)

Are you suggesting the current core architecture is problematic or that
a conversion to something like reactive would result in "strange" bugs?

Thanks,
Matt

On 4/4/16 10:52 AM, Johan Edstrom wrote:

> I think the core should focus on speed, consistency, reliability
> and last extensibility; Camel has a nice traction but we don’t
> want to see “strange” bugs from new features if that makes
> sense as a long windy sentence…..
>
>
>
>> On Apr 4, 2016, at 9:36 AM, Matt Pavlovich <[hidden email]> wrote:
>>
>>
>>
>> On 3/24/16 3:27 PM, Raul Kripalani wrote:
>>> On Thu, Mar 24, 2016 at 5:24 PM, Krzysztof Sobkowiak <
>>> [hidden email]> wrote:
>>>
>>>> I think, the way to Camel 3 should also include the renovation of the Core
>>>> (if really necessary) or even rewriting and
>>>> making it more asynchronous, e.g. using rx.java (the later can be
>>>> eventually part of Camel 4 roadmap if too dangerous
>>>> for Camel 3).
>>>>
>>> This also relates to the concurrency model. Camel revolves around the idea
>>> that more threads == more performance. Our processing actually
>>> hijacks/piggy-backs the threads of the 3rd party library of the component
>>> (Netty, Jetty, Spring JMS, etc. threads), and this is not good. One just
>>> needs to look around: reactive programming, coroutines, event loops,
>>> generators, goroutines (in Go), etc. to realise we're kinda obsolete by now.
>> "Obsolete" by what standard?
>>> The async routing engine was a step in the right direction, but most
>>> components don't even support it.
>>>
>>> Now that we know the reactive programming model is here to stay and it
>>> wasn't a fad, I feel we should definitely go in that direction. We should
>>> look at Akka, RxJava and the Reactive Streams [3] spec.
>> I expect there will be long term valuable pieces pulled from reactive programming, just as there are from virtually every new "paradigm" that pops up every few years-- but there will also be a balance of "doesn't meet the hype". Anyone remember AOP (Btw-- which now has 3-4 sub-derivatives)?  I suggest letting the dust settle to see what the benefits-tradeoffs matrix looks like before committing to overhauling Camel core.
>>
>> -Matt

Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] - Thoughts on Apache Camel 2.18 and towards 3.0

Claus Ibsen-2
In reply to this post by Matt Pavlovich
Hi

The migration of the docs is already started with the components. We
are migrating them one by one which takes a rather long time to get
all done, but we do have a good start already.

Thee is docs in the src/doc folder of the components that has been
migrated, such as
https://github.com/apache/camel/blob/master/components/camel-ahc/src/main/docs/ahc.adoc

And a start of a website that put together all that docs at
https://github.com/apache/camel/tree/master/docs/user-manual/en

The latter is just very rough and certainly not set in stone.


The first goal is to get all the component docs migrated. Where it
grabs parts of the docs from the source code and auto maintain in the
.adoc files. So we ensure all the options are always 100% up to date.
We can then do the same for the EIPs as we can do for the docs. And as
well for data formats and languages. So all of them are always up to
date.

The actual look and feel of the website is not started yet. And yes I
do think the Apache Karaf team did a good job of a website make over,
at least on the front page. When you dive down then the docs is the
old style.






On Mon, Apr 4, 2016 at 6:57 PM, Matt Pavlovich <[hidden email]> wrote:

>
>
> On 4/4/16 11:12 AM, Raul Kripalani wrote:
>>
>> On Mon, Apr 4, 2016 at 4:44 PM, Matt Pavlovich <[hidden email]> wrote:
>>
>>> The current website looks the same as it did when it was created:
>>>>
>>>>
>>>> https://web.archive.org/web/20070701184530/http://activemq.apache.org/camel/
>>>>
>>> I thought the Karaf guys did a nice job on the website re-work. Is there
>>> a
>>> new base framework for all Apache projects, or are we at ground zero on
>>> this?
>>
>>
>> I dig the Karaf redesign - clean, modern and simple. I'd love to aim for
>> something similar in Camel.
>>
>> What do we do about our content editing system? My proposal is to move
>> away
>> from Confluence and adopt Markdown, AsciiDoc, or the like.
>>
>> Pros:
>>
>> 1. Docs can be versioned alongside code
>> 2. The HTML is purer and with less artifacts => accurate styling =>
>> cleaner
>> visuals.
>> 3. A breeze to edit, fix typos, add release notes, etc.
>>
>> Cons:
>>
>> 1. Large migration effort.
>
>
> I agree, a markdown-based CMS would be handy. Do we know what Karaf is
> using?
>
> Re Cons #1: I think the docs need a massive overhaul anyway. Probably a good
> time to just do it.
>
> The component config should be separated by version and not a hodge-podge of
> carve-outs like we have today.
>
> Rough pass at a TOC:
> (Suggest short and sweet)
> Index
> Documentation
> Download
> Support
>
> Then the sub-sections fall under the top level categories.
>
> Documentation
>    + Core
>    + Components
>    + Data Formats
>    + etc..
>
> Download
>     + Grab the bits
>     + Grab the source
>     + Links to svn / git etc
>
> Contributor Documentation
>     + Camel API
>     + etc..
>
> Support
>      + Community  ( IRC, mailing lists, LinkedIn Group, etc)
>      + Commercial
>
>
>
>>
>> Cheers,
>>
>> *Raúl Kripalani*
>> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
>> Messaging Engineer
>> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
>> Blog: raul.io
>> <http://raul.io/?utm_source=email&utm_medium=email&utm_campaign=apache> |
>> twitter: @raulvk <https://twitter.com/raulvk>
>>
>



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] - Thoughts on Apache Camel 2.18 and towards 3.0

Claus Ibsen-2
In reply to this post by Gnanaguru S
On Thu, Mar 24, 2016 at 1:12 PM, Gnanaguru S <[hidden email]> wrote:

>
> Guys,
>
> These are some of the feature, I wish to see in upcoming camel versions. It
> would be useful to have.
>
> 1. Timeout option in a internal synchronous endpoint - we very much use
> camel for orchestration layer. In a use case where many downstream
> routes/endpoints are called synchronously, I would like to have a direct:
> endpoint which has a timeout. So that I can timeout a call to some
> particular downstream route, handle the exception and proceed to the next
> step. In fact this option is already there with seda: endpoint, but I wont
> prefer to use seda: for synchronous flow. Also I can use internal JMS
> endpoints between routes to achieve this, but sometimes using JMS for
> internal transaction is heavy.
>
> 2. Improved/Sophisticated xmljson component.
>
> 3. Improved/even Simpler exception handling.
>
> 4. Native Camel transformation component, I am bit obsessed with XSLT. So I
> was thinking, why dont camel   incorporate something like xslt: internally.
> like Camel transformer component ? Something better than XSLT component,
> something very camelish. ;)
>
> 5. Better CXF configurations, for example configuring timeout for a cxf
> consumer and producer endpoint is not that straight forward like jetty /
> http(4) components.
>
> 6. Better/Improved unit testing, Camel test support is great, like
> AdviceWith & Mock endpoints. But some of their methods have issues, I am
> seeing latest camel versions have lot of fixes, but I think some more
> features can be introduced. For example, if I want send a sample XML as a
> request, I dont want to use a IOUtil or a File reader to load the file and
> then to a string and then pass it to a producer template, instead it will be
> easier if there is a something called as mockRequest. May be there are
> better options already, please ignore my ignorance here ;)
>

You can send a java.io.File as the message body in the producer template.




> 7.  JSON validator. XML XSD validation is nice and straight forward, but it
> will be great if we have something similar for JSON as well. like
> to:json-validator:classpath/response.json
>
> 8. Ability to load a route context directly to a unit test case
>
> These are some aspects I could think of at the moment, I will try to add
> more. But please ignore if some of these ideas sounds silly ;)
>
> Happy long weekend !
>
> Cheers
> Guru
> http://gnan.io
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/DISCUSS-Thoughts-on-Apache-Camel-2-18-and-towards-3-0-tp5779549p5779641.html
> Sent from the Camel Development mailing list archive at Nabble.com.



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] - Thoughts on Apache Camel 2.18 and towards 3.0

Claus Ibsen-2
In reply to this post by Claus Ibsen-2
Hi

Just an update on some of these tasks lined

On Wed, Mar 23, 2016 at 11:07 AM, Claus Ibsen <[hidden email]> wrote:

> Hi
>
> So Camel 2.17 was the last release supporting Java 1.7.
> The next Camel 2.18 is requiring Java 1.8.
>
> Here is some thoughts of mine about this release up for discussion.
>
>
>
> a)
> I see the overall goal of Camel 2.18 as a stepping stone towards Java
> 1.8 and Camel 3.0.
>
> By that I mean the release should be a way of moving our existing
> users from Java 1.7 and the current Camel APIs and the likes gradually
> towards Java 1.8 and eventually Camel 3.0.
>
> In other words we should not get carried away to change/break APIs and
> whatnot just because Java 1.8 lambdas and functions.
>
> There are too many current users that rely on the current Camel API
> and we cannot go around change processor / expression / predicate /
> aggregation strategy and other interfaces to be java 8 functional if
> that means current code cannot compile. And certainly not adding
> Optional<X> as return types all over.
>
> The following releases (Camel 2.19 or 3.0) can pick up that torch and
> be more Java 1.8 aggressive. For example Camel 3.0 can expect API
> changes that are Java 8 lambda / functional based. And as well changes
> in the DSL to go with that.
>
> There are some minor code changes needed to make the source compile as
> source 1.8 to go in this Camel 2.18 let alone.
>
>
> b)
> Drop components that do not support and run on Java 1.8
> And potentially remove some deprecated components
>

Not started yet.


>
> c)
> Drop karaf 2.x.
> And move to karaf 4.x for all our testing.
>

The missing part is the tests/camel-itest-osgi which is in a sad state
already. The tests dont run either well today. I wonder if we should
zap it, and recreate a new itest that are up to date. We should then
reuse the work on the camel-test-karaf module that is in the works.


>
> d)
> Drop Jetty 8.x.
>
> This also requires to upgrade at least two components that currently
> rely on Jetty 8 to use Jetty 9.
>

Not started yet

>
> e)
> Upgrade to latest Jetty 9.
> Jetty 9.3 (or is it 9.4) requires Java 1.8
>

Not started yet

>
> f)
> Drop support for older versions of Spring. We have a number of
> camel-test-spring3 etc modules that can be dropped. And maybe even
> spring 4.0. as its also EOL.
>

Done

>
> g)
> Potentially move spring-dm out of camel-spring into a camel-spring-dm
> module. So camel-spring can use latest version of Spring safely. This
> also makes it easier to deprecated spring-dm and remove it eventually.
> The Karaf team is working on a sping -> blueprint layer so you can use
> spring xml files but Karaf will "convert" that under the hood to
> blueprint and run it as blueprint. When that is ready we no longer
> need spring-dm.
>

Done

>
> h)
> Continue adding components docs in the source, eg src/main/doc files.
> So we eventually have as many/all of them. This is an ongoing effort,
> as we need to do this for the EIPs and the other parts of the docs.
>
> However I see this as a great step for a new documentation and
> website, that IMHO is a big goal for Camel 3.0. To make the project
> website fresh and modern. And make the documentation easier for end
> users to use and view.
>
>
> i)
> Add camel-hysterix component and integrate camel's circuit breaker
> into turbine/hysterix so you can see metrics from camel in the
> dashboard. Eg to integrate with the popular Netflix OSS stack.
>

Component added. The metrics is still pending.

>
> j)
> Split camel-cxf into modules so we can separate WS and RS and also
> spring vs blueprint. Today its big ball of dependencies that is a bit
> hard to slice and dice. Specially for MSA style with REST and you dont
> want to add in a bunch of extra not needed JARs.
>

Not started yet.

>
> k)
> Continue as usual by adding new components, data formats, fix bugs, and so on.
>
>
> l)
> Timeline. This release do not need to have 6-8 months timeframe. We
> could try to get this "stepping stone" release done sooner, so it can
> be released during/shortly after summer.
>
> There is plenty of "first work" that we must do with the java 8
> upgrade and dropping older techs etc, that we have our hands full for
> a while.
>
> Doing a release with these changes allows our end users to migrate
> along in a easy way, than a big bang - breaking apis - release would
> do. And the latter would be more appropriate to be released as Camel
> 3.0.
>
> Then towards the end of this year, we can see where we are and plan
> for a Camel 3.0 with a new website and documentation that such a
> release deserve. For example if we release Camel 3.0 in start of 2017
> then its also Camel's 10 year birthday year.
>
> And doing such a release with a rewamped website with fresh looking
> documentation and content, is what helps the project a lot.
>
> The current website looks the same as it did when it was created:
> https://web.archive.org/web/20070701184530/http://activemq.apache.org/camel/
>
> PS: We surely also need a better "what is Camel" story on the front
> page. Its still that very first one with all the tech jumble that was
> initially created.
>
> PPS: I would also love to see a new Camel logo. The current one is a
> bit dull and boring.
>
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] - Thoughts on Apache Camel 2.18 and towards 3.0

Claus Ibsen-2
In reply to this post by Claus Ibsen-2
Hi

Just though I wanted to do a status update on those initial tasks we
set out to do.

On Wed, Mar 23, 2016 at 11:07 AM, Claus Ibsen <[hidden email]> wrote:

> Hi
>
> So Camel 2.17 was the last release supporting Java 1.7.
> The next Camel 2.18 is requiring Java 1.8.
>
> Here is some thoughts of mine about this release up for discussion.
>
>
>
> a)
> I see the overall goal of Camel 2.18 as a stepping stone towards Java
> 1.8 and Camel 3.0.
>
> By that I mean the release should be a way of moving our existing
> users from Java 1.7 and the current Camel APIs and the likes gradually
> towards Java 1.8 and eventually Camel 3.0.
>
> In other words we should not get carried away to change/break APIs and
> whatnot just because Java 1.8 lambdas and functions.
>
> There are too many current users that rely on the current Camel API
> and we cannot go around change processor / expression / predicate /
> aggregation strategy and other interfaces to be java 8 functional if
> that means current code cannot compile. And certainly not adding
> Optional<X> as return types all over.
>
> The following releases (Camel 2.19 or 3.0) can pick up that torch and
> be more Java 1.8 aggressive. For example Camel 3.0 can expect API
> changes that are Java 8 lambda / functional based. And as well changes
> in the DSL to go with that.
>
> There are some minor code changes needed to make the source compile as
> source 1.8 to go in this Camel 2.18 let alone.
>
>
> b)
> Drop components that do not support and run on Java 1.8
> And potentially remove some deprecated components
>

Outstanding.

>
> c)
> Drop karaf 2.x.
> And move to karaf 4.x for all our testing.
>

Done

>
> d)
> Drop Jetty 8.x.
>
> This also requires to upgrade at least two components that currently
> rely on Jetty 8 to use Jetty 9.
>

Done

>
> e)
> Upgrade to latest Jetty 9.
> Jetty 9.3 (or is it 9.4) requires Java 1.8
>

Outstanding. There is some API breakings in jetty 9.2 to 9.3 (surprise
surprise).



>
> f)
> Drop support for older versions of Spring. We have a number of
> camel-test-spring3 etc modules that can be dropped. And maybe even
> spring 4.0. as its also EOL.
>

Done

>
> g)
> Potentially move spring-dm out of camel-spring into a camel-spring-dm
> module. So camel-spring can use latest version of Spring safely. This
> also makes it easier to deprecated spring-dm and remove it eventually.
> The Karaf team is working on a sping -> blueprint layer so you can use
> spring xml files but Karaf will "convert" that under the hood to
> blueprint and run it as blueprint. When that is ready we no longer
> need spring-dm.
>

Done

>
> h)
> Continue adding components docs in the source, eg src/main/doc files.
> So we eventually have as many/all of them. This is an ongoing effort,
> as we need to do this for the EIPs and the other parts of the docs.
>
> However I see this as a great step for a new documentation and
> website, that IMHO is a big goal for Camel 3.0. To make the project
> website fresh and modern. And make the documentation easier for end
> users to use and view.
>

A lot has been done, thanks a lot to Andrea who have done most of the migration.

What is outstanding is to generate/update the EIP documentation for
all their options in a table, just as we do for components.
This should not be hard to do, as we just do similar eg add some code
in the the mvn plugin, and add some markers in the .adoc file where it
should insert/update the table.



>
> i)
> Add camel-hysterix component and integrate camel's circuit breaker
> into turbine/hysterix so you can see metrics from camel in the
> dashboard. Eg to integrate with the popular Netflix OSS stack.
>

Done

>
> j)
> Split camel-cxf into modules so we can separate WS and RS and also
> spring vs blueprint. Today its big ball of dependencies that is a bit
> hard to slice and dice. Specially for MSA style with REST and you dont
> want to add in a bunch of extra not needed JARs.
>

Outstanding. I think this is somewhat big work but would be good if
some of the CXF guys would pickup this task.
The CXF with RS is "hurt" when its dragging in a old WS dependencies.



>
> k)
> Continue as usual by adding new components, data formats, fix bugs, and so on.
>

Yep this is ongoing and we got 8 new components already.


>
> l)
> Timeline. This release do not need to have 6-8 months timeframe. We
> could try to get this "stepping stone" release done sooner, so it can
> be released during/shortly after summer.
>

I think after the summer vacation would be good time. We usually do a
release around september/early october.
So lets try to aim for that.


> There is plenty of "first work" that we must do with the java 8
> upgrade and dropping older techs etc, that we have our hands full for
> a while.
>
> Doing a release with these changes allows our end users to migrate
> along in a easy way, than a big bang - breaking apis - release would
> do. And the latter would be more appropriate to be released as Camel
> 3.0.
>
> Then towards the end of this year, we can see where we are and plan
> for a Camel 3.0 with a new website and documentation that such a
> release deserve. For example if we release Camel 3.0 in start of 2017
> then its also Camel's 10 year birthday year.
>
> And doing such a release with a rewamped website with fresh looking
> documentation and content, is what helps the project a lot.
>
> The current website looks the same as it did when it was created:
> https://web.archive.org/web/20070701184530/http://activemq.apache.org/camel/
>
> PS: We surely also need a better "what is Camel" story on the front
> page. Its still that very first one with all the tech jumble that was
> initially created.
>
> PPS: I would also love to see a new Camel logo. The current one is a
> bit dull and boring.
>
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
123