[DISCUSS] Extract spring-boot support

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSS] Extract spring-boot support

Guillaume Nodet-2
Hi everyone,

Since Camel 3.0.0 has been released, and given we now have different
subprojects, I'd like to discuss the possibility of moving the spring boot
support into a different git repository.
I see several benefits:
  * being able to support different versions of spring boot
  * better decoupling
  * improved build speed
I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226 that
could be used for the new repo.  I haven't tackled the PR for the main
camel repo yet, but it should be easier I think.
The internal tooling had to be adapted to cope with the new setup.  The
main difference is that each starter uses a maven plugin which generates:
the starter pom, the spring boot configs and updates the catalog.

Feedback welcomed !

Cheers,
Guillaume Nodet
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Claus Ibsen-2
Hi

+1
This is great stuff.

So just think how would the developer do when eg you add a new
component in the main Camel repo, lets call it components/camel-foobar
?

Then you go to camel-spring-boot and want to generate a
camel-foobar-starter because it should support SB too (Mind that not
all components should have SB -starters).
So it sounds like there is a new maven plugin you then use to generate
all of that?

Something a like:

cd components-starter
mvn camel-spring-boot-generator:prepare-spring-boot-starter -Dname=foobar

Or how?

We should have the steps documented in the readme file






On Mon, Dec 9, 2019 at 10:47 PM Guillaume Nodet <[hidden email]> wrote:

>
> Hi everyone,
>
> Since Camel 3.0.0 has been released, and given we now have different
> subprojects, I'd like to discuss the possibility of moving the spring boot
> support into a different git repository.
> I see several benefits:
>   * being able to support different versions of spring boot
>   * better decoupling
>   * improved build speed
> I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226 that
> could be used for the new repo.  I haven't tackled the PR for the main
> camel repo yet, but it should be easier I think.
> The internal tooling had to be adapted to cope with the new setup.  The
> main difference is that each starter uses a maven plugin which generates:
> the starter pom, the spring boot configs and updates the catalog.
>
> Feedback welcomed !
>
> Cheers,
> Guillaume Nodet



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Andrea Cosentino-2
In reply to this post by Guillaume Nodet-2
Really good work!

I think this is just the first step, we can use a similar approach for the others flavours.

But we need to document everything really well and we need to rethink about the release process, I guess.

Thanks!

--
Andrea Cosentino 
----------------------------------
Apache Camel PMC Chair
Apache Karaf Committer
Apache Servicemix PMC Member
Email: [hidden email]
Twitter: @oscerd2
Github: oscerd






On Monday, December 9, 2019, 10:47:55 PM GMT+1, Guillaume Nodet <[hidden email]> wrote:





Hi everyone,

Since Camel 3.0.0 has been released, and given we now have different
subprojects, I'd like to discuss the possibility of moving the spring boot
support into a different git repository.
I see several benefits:
  * being able to support different versions of spring boot
  * better decoupling
  * improved build speed
I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226 that
could be used for the new repo.  I haven't tackled the PR for the main
camel repo yet, but it should be easier I think.
The internal tooling had to be adapted to cope with the new setup.  The
main difference is that each starter uses a maven plugin which generates:
the starter pom, the spring boot configs and updates the catalog.

Feedback welcomed !

Cheers,
Guillaume Nodet
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Zoran Regvart-2
In reply to this post by Guillaume Nodet-2
Hi Guillaume,
+1 for the effort, I very much appreciate any effort to
simplify/speedup the build and separate the concerns.

I would also like to know how the workflow for keeping the starters in
sync would look like, would this be done by individually: on changes
to the Camel repository there needs to be a corresponding change in
the Camel Spring Boot repository? Could we invest in some automation
here?

zoran

On Mon, Dec 9, 2019 at 10:47 PM Guillaume Nodet <[hidden email]> wrote:

>
> Hi everyone,
>
> Since Camel 3.0.0 has been released, and given we now have different
> subprojects, I'd like to discuss the possibility of moving the spring boot
> support into a different git repository.
> I see several benefits:
>   * being able to support different versions of spring boot
>   * better decoupling
>   * improved build speed
> I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226 that
> could be used for the new repo.  I haven't tackled the PR for the main
> camel repo yet, but it should be easier I think.
> The internal tooling had to be adapted to cope with the new setup.  The
> main difference is that each starter uses a maven plugin which generates:
> the starter pom, the spring boot configs and updates the catalog.
>
> Feedback welcomed !
>
> Cheers,
> Guillaume Nodet



--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Guillaume Nodet-2
The steps to perform when adding / removing a new camel component are still
to be defined.
Currently, what is working is the update of existing starters.
I think creating two goals to add / remove a starter can be easily done, so
that i could be done in an easy step.  I'll work on that.
I've also spotted that the BOM generation isn't fully updated with the new
proposed source layout, so I'll fix it first.

Guillaume

Le mar. 10 déc. 2019 à 10:19, Zoran Regvart <[hidden email]> a écrit :

> Hi Guillaume,
> +1 for the effort, I very much appreciate any effort to
> simplify/speedup the build and separate the concerns.
>
> I would also like to know how the workflow for keeping the starters in
> sync would look like, would this be done by individually: on changes
> to the Camel repository there needs to be a corresponding change in
> the Camel Spring Boot repository? Could we invest in some automation
> here?
>
> zoran
>
> On Mon, Dec 9, 2019 at 10:47 PM Guillaume Nodet <[hidden email]> wrote:
> >
> > Hi everyone,
> >
> > Since Camel 3.0.0 has been released, and given we now have different
> > subprojects, I'd like to discuss the possibility of moving the spring
> boot
> > support into a different git repository.
> > I see several benefits:
> >   * being able to support different versions of spring boot
> >   * better decoupling
> >   * improved build speed
> > I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226
> that
> > could be used for the new repo.  I haven't tackled the PR for the main
> > camel repo yet, but it should be easier I think.
> > The internal tooling had to be adapted to cope with the new setup.  The
> > main difference is that each starter uses a maven plugin which generates:
> > the starter pom, the spring boot configs and updates the catalog.
> >
> > Feedback welcomed !
> >
> > Cheers,
> > Guillaume Nodet
>
>
>
> --
> Zoran Regvart
>


--
------------------------
Guillaume Nodet
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Claus Ibsen-2
In reply to this post by Zoran Regvart-2
On Tue, Dec 10, 2019 at 10:19 AM Zoran Regvart <[hidden email]> wrote:

>
> Hi Guillaume,
> +1 for the effort, I very much appreciate any effort to
> simplify/speedup the build and separate the concerns.
>
> I would also like to know how the workflow for keeping the starters in
> sync would look like, would this be done by individually: on changes
> to the Camel repository there needs to be a corresponding change in
> the Camel Spring Boot repository? Could we invest in some automation
> here?

I actually dont mind its not fully automated, as for example today we
end up mistakenly generating -starter JARs
from some stuff that does not work with SB or is not intended at all
(eg another runtime like MP or OSGi).
And then we need to go in an add code into the maven plugin to skip
those which are hardcoded in the source.

So I think its fine if its a step where you then go to the
camel-spring-boot repo and then run a maven plugin that generate the
-starter.

With the Camel catalog we can/should generate a new page on the
website that lists all the artifacts and has [x] for which runtimes
they support.
Then you can quickly see where there is gaps etc
https://issues.apache.org/jira/browse/CAMEL-13804


>
> zoran
>
> On Mon, Dec 9, 2019 at 10:47 PM Guillaume Nodet <[hidden email]> wrote:
> >
> > Hi everyone,
> >
> > Since Camel 3.0.0 has been released, and given we now have different
> > subprojects, I'd like to discuss the possibility of moving the spring boot
> > support into a different git repository.
> > I see several benefits:
> >   * being able to support different versions of spring boot
> >   * better decoupling
> >   * improved build speed
> > I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226 that
> > could be used for the new repo.  I haven't tackled the PR for the main
> > camel repo yet, but it should be easier I think.
> > The internal tooling had to be adapted to cope with the new setup.  The
> > main difference is that each starter uses a maven plugin which generates:
> > the starter pom, the spring boot configs and updates the catalog.
> >
> > Feedback welcomed !
> >
> > Cheers,
> > Guillaume Nodet
>
>
>
> --
> Zoran Regvart



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Guillaume Nodet-2
Le mar. 10 déc. 2019 à 10:35, Claus Ibsen <[hidden email]> a écrit :

> On Tue, Dec 10, 2019 at 10:19 AM Zoran Regvart <[hidden email]> wrote:
> >
> > Hi Guillaume,
> > +1 for the effort, I very much appreciate any effort to
> > simplify/speedup the build and separate the concerns.
> >
> > I would also like to know how the workflow for keeping the starters in
> > sync would look like, would this be done by individually: on changes
> > to the Camel repository there needs to be a corresponding change in
> > the Camel Spring Boot repository? Could we invest in some automation
> > here?
>
> I actually dont mind its not fully automated, as for example today we
> end up mistakenly generating -starter JARs
> from some stuff that does not work with SB or is not intended at all
> (eg another runtime like MP or OSGi).
> And then we need to go in an add code into the maven plugin to skip
> those which are hardcoded in the source.
>
> So I think its fine if its a step where you then go to the
> camel-spring-boot repo and then run a maven plugin that generate the
> -starter.
>

Right, that's what I did.
The commands are available at
https://github.com/gnodet/camel/tree/CAMEL-14226#creating--deleting-starters
and
do use a maven plugin underneath.


>
> With the Camel catalog we can/should generate a new page on the
> website that lists all the artifacts and has [x] for which runtimes
> they support.
> Then you can quickly see where there is gaps etc
> https://issues.apache.org/jira/browse/CAMEL-13804
>
>
Agreed, I'll see if something can be generated.

Afaik, I've fixed all the generation and tests, so the whole build works
fine now.
I'll work on the main camel repo now.

Guillaume


> >
> > zoran
> >
> > On Mon, Dec 9, 2019 at 10:47 PM Guillaume Nodet <[hidden email]>
> wrote:
> > >
> > > Hi everyone,
> > >
> > > Since Camel 3.0.0 has been released, and given we now have different
> > > subprojects, I'd like to discuss the possibility of moving the spring
> boot
> > > support into a different git repository.
> > > I see several benefits:
> > >   * being able to support different versions of spring boot
> > >   * better decoupling
> > >   * improved build speed
> > > I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226
> that
> > > could be used for the new repo.  I haven't tackled the PR for the main
> > > camel repo yet, but it should be easier I think.
> > > The internal tooling had to be adapted to cope with the new setup.  The
> > > main difference is that each starter uses a maven plugin which
> generates:
> > > the starter pom, the spring boot configs and updates the catalog.
> > >
> > > Feedback welcomed !
> > >
> > > Cheers,
> > > Guillaume Nodet
> >
> >
> >
> > --
> > Zoran Regvart
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>


--
------------------------
Guillaume Nodet
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Guillaume Nodet-2
In reply to this post by Guillaume Nodet-2
Things have progressed nicely:
  * the branch https://github.com/gnodet/camel/tree/CAMEL-14226 is the
spring boot support
  * the branch https://github.com/gnodet/camel/tree/CAMEL-14226-core is the
camel repo where spring-boot is deleted
All components relying on spring-boot (the camel-spring-boot and
camel-spring-cloud-xxx) along with examples that were using spring-boot
have been "moved" to the spring-boot support branch.
This branch contains 2 shell commands to add / remove a spring-boot starter.

The next steps would be to:
  * eventually start a vote if people think it's needed (feedback ?)
  * create the repo and merge the PRs
  * (re-)write generators to provide content on the web site

On the web site side of things, I haven't paid a close attention to how the
web site is published.  The camel source tree has some content (
https://github.com/apache/camel/tree/master/docs), but how is that pushed
to the web site ? Same question for blogs actually ;-)

Guillaume

Le lun. 9 déc. 2019 à 22:47, Guillaume Nodet <[hidden email]> a écrit :

> Hi everyone,
>
> Since Camel 3.0.0 has been released, and given we now have different
> subprojects, I'd like to discuss the possibility of moving the spring boot
> support into a different git repository.
> I see several benefits:
>   * being able to support different versions of spring boot
>   * better decoupling
>   * improved build speed
> I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226 that
> could be used for the new repo.  I haven't tackled the PR for the main
> camel repo yet, but it should be easier I think.
> The internal tooling had to be adapted to cope with the new setup.  The
> main difference is that each starter uses a maven plugin which generates:
> the starter pom, the spring boot configs and updates the catalog.
>
> Feedback welcomed !
>
> Cheers,
> Guillaume Nodet
>
>
>
>

--
------------------------
Guillaume Nodet
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Zoran Regvart-2
Hi Guillaume,

On Thu, Dec 12, 2019 at 4:27 PM Guillaume Nodet <[hidden email]> wrote:
> On the web site side of things, I haven't paid a close attention to how the
> web site is published.  The camel source tree has some content (
> https://github.com/apache/camel/tree/master/docs), but how is that pushed
> to the web site ? Same question for blogs actually ;-)

Website build is described in the README[1] of the camel-website
repository. We have a Jenkins job[2] that takes care of building it
and publishing it to the `asf-site` branch from which it's pulled by
INFRA to the web server.

Parts of it are in Markdown (e.g blog) and published by Hugo,
reference documentation is pulled from camel, camel-k and
camel-quarkus repositories and built by Antora.

Hope this helps, if anyone else misses this information I can add more
on the build/publish process to the README,

zoran

[1] https://github.com/apache/camel-website/blob/master/README.md
[2] https://builds.apache.org/job/Camel.website/
--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Claus Ibsen-2
In reply to this post by Guillaume Nodet-2
Hi

The -P fastinstall does not yet work, it still runs unit tests.

On Thu, Dec 12, 2019 at 4:27 PM Guillaume Nodet <[hidden email]> wrote:

>
> Things have progressed nicely:
>   * the branch https://github.com/gnodet/camel/tree/CAMEL-14226 is the
> spring boot support
>   * the branch https://github.com/gnodet/camel/tree/CAMEL-14226-core is the
> camel repo where spring-boot is deleted
> All components relying on spring-boot (the camel-spring-boot and
> camel-spring-cloud-xxx) along with examples that were using spring-boot
> have been "moved" to the spring-boot support branch.
> This branch contains 2 shell commands to add / remove a spring-boot starter.
>
> The next steps would be to:
>   * eventually start a vote if people think it's needed (feedback ?)
>   * create the repo and merge the PRs
>   * (re-)write generators to provide content on the web site
>
> On the web site side of things, I haven't paid a close attention to how the
> web site is published.  The camel source tree has some content (
> https://github.com/apache/camel/tree/master/docs), but how is that pushed
> to the web site ? Same question for blogs actually ;-)
>
> Guillaume
>
> Le lun. 9 déc. 2019 à 22:47, Guillaume Nodet <[hidden email]> a écrit :
>
> > Hi everyone,
> >
> > Since Camel 3.0.0 has been released, and given we now have different
> > subprojects, I'd like to discuss the possibility of moving the spring boot
> > support into a different git repository.
> > I see several benefits:
> >   * being able to support different versions of spring boot
> >   * better decoupling
> >   * improved build speed
> > I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226 that
> > could be used for the new repo.  I haven't tackled the PR for the main
> > camel repo yet, but it should be easier I think.
> > The internal tooling had to be adapted to cope with the new setup.  The
> > main difference is that each starter uses a maven plugin which generates:
> > the starter pom, the spring boot configs and updates the catalog.
> >
> > Feedback welcomed !
> >
> > Cheers,
> > Guillaume Nodet
> >
> >
> >
> >
>
> --
> ------------------------
> Guillaume Nodet



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Claus Ibsen-2
In reply to this post by Zoran Regvart-2
Hi

For the docs/website, then I think camel-spring-boot should be similar
to camel-quarkus, where Spring Boot would be listed under "projects"
on the website.

And for camel-quarkus we have a tool that generates the list of
components, and its documentation.
https://github.com/apache/camel-quarkus/tree/master/tooling/package-maven-plugin
https://github.com/apache/camel-quarkus/blob/master/docs/modules/ROOT/pages/list-of-camel-quarkus-extensions.adoc

We could have that too for camel-spring-boot. And for the component
documentation with the extra information about spring-boot
auto-configuration, then we could maybe

- keep it as-is (and find a way for camel-spring-boot to update on
camel main project) maybe via some script or java/maven tool

or

- copy over the existing documentation from the camel releases (you
can find the adoc files in the camel-catalog JAR).
- remove the spring-boot auto-configuration from the camel documentation
- add the spring boot auto-configuration section in the camel-spring-boot tool

or

- remove all the auto-configuration documentation all together, they
are almost identical to the component level options which is already
listed
- the options are generated in the -starter JARs and there are spring
tools that can do code assistances in editors




On Thu, Dec 12, 2019 at 7:39 PM Zoran Regvart <[hidden email]> wrote:

>
> Hi Guillaume,
>
> On Thu, Dec 12, 2019 at 4:27 PM Guillaume Nodet <[hidden email]> wrote:
> > On the web site side of things, I haven't paid a close attention to how the
> > web site is published.  The camel source tree has some content (
> > https://github.com/apache/camel/tree/master/docs), but how is that pushed
> > to the web site ? Same question for blogs actually ;-)
>
> Website build is described in the README[1] of the camel-website
> repository. We have a Jenkins job[2] that takes care of building it
> and publishing it to the `asf-site` branch from which it's pulled by
> INFRA to the web server.
>
> Parts of it are in Markdown (e.g blog) and published by Hugo,
> reference documentation is pulled from camel, camel-k and
> camel-quarkus repositories and built by Antora.
>
> Hope this helps, if anyone else misses this information I can add more
> on the build/publish process to the README,
>
> zoran
>
> [1] https://github.com/apache/camel-website/blob/master/README.md
> [2] https://builds.apache.org/job/Camel.website/
> --
> Zoran Regvart



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Gregor Zurowski-2
In reply to this post by Andrea Cosentino-2
Hi Everyone:

I wanted to follow up on Andrea's question regarding the release
process.  As Guillaume already pointed out, one of the benefits of
splitting Spring Boot support into a separate repository is to be able
to support multiple Spring Boot versions.  This would mean that the
Spring Boot support artifacts will have a separate release cycle?  If
this is the case, I believe it's extremely important to provide an
overview that shows which Camel version works with which Spring Boot
support artifacts.

Thanks,
Gregor

On Tue, Dec 10, 2019 at 8:11 AM Andrea Cosentino
<[hidden email]> wrote:

>
> Really good work!
>
> I think this is just the first step, we can use a similar approach for the others flavours.
>
> But we need to document everything really well and we need to rethink about the release process, I guess.
>
> Thanks!
>
> --
> Andrea Cosentino
> ----------------------------------
> Apache Camel PMC Chair
> Apache Karaf Committer
> Apache Servicemix PMC Member
> Email: [hidden email]
> Twitter: @oscerd2
> Github: oscerd
>
>
>
>
>
>
> On Monday, December 9, 2019, 10:47:55 PM GMT+1, Guillaume Nodet <[hidden email]> wrote:
>
>
>
>
>
> Hi everyone,
>
> Since Camel 3.0.0 has been released, and given we now have different
> subprojects, I'd like to discuss the possibility of moving the spring boot
> support into a different git repository.
> I see several benefits:
>   * being able to support different versions of spring boot
>   * better decoupling
>   * improved build speed
> I've created a PR at https://github.com/gnodet/camel/tree/CAMEL-14226 that
> could be used for the new repo.  I haven't tackled the PR for the main
> camel repo yet, but it should be easier I think.
> The internal tooling had to be adapted to cope with the new setup.  The
> main difference is that each starter uses a maven plugin which generates:
> the starter pom, the spring boot configs and updates the catalog.
>
> Feedback welcomed !
>
> Cheers,
> Guillaume Nodet
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Zoran Regvart-2
Hi Cameleers,

On Fri, Dec 20, 2019 at 9:33 AM Gregor Zurowski
<[hidden email]> wrote:
> I wanted to follow up on Andrea's question regarding the release
> process.  As Guillaume already pointed out, one of the benefits of
> splitting Spring Boot support into a separate repository is to be able
> to support multiple Spring Boot versions.  This would mean that the
> Spring Boot support artifacts will have a separate release cycle?  If
> this is the case, I believe it's extremely important to provide an
> overview that shows which Camel version works with which Spring Boot
> support artifacts.

I assumed that we would release a Camel version and, perhaps shortly
after, the Camel Spring Boot starters. I assumed that the Camel Spring
Boot starters would have the same version as the Camel release.

A version of Camel depends on a Spring version, which would in some
way dictates the Spring Boot version, so we need to make those in
sync.

The argument for supporting multiple Spring Boot versions, say 2.x and
3.x (of Spring Boot), we could release from two branches depending on
the same Camel version. For that we would need to come up with a
versioning scheme that allows the users to pick the correct version.

zoran
--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSS] Extract spring-boot support

Gregor Zurowski-2
> I assumed that we would release a Camel version and, perhaps shortly
> after, the Camel Spring Boot starters. I assumed that the Camel Spring
> Boot starters would have the same version as the Camel release.

I agree, the version parity between Camel and Camel Spring Boot (which
is implicitly assured in the current single repo setup) makes it very
easy to identify which Camel works with which Camel Spring Boot
artifact. I would definitely support to release both with the same
version and within the same short timespan.