A "Kamel" crazy idea

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

A "Kamel" crazy idea

Nicola Ferraro
Hi Cameleers,
it's now passed some time since I started thinking about a new project that
we can begin here at Apache Camel, and I'd like to have your opinion.

We've already been targeting cloud-native applications with Camel,
especially on top of Kubernetes, that is becoming "the standard" cloud
platform. But writing a Camel integration and running it on Kubernetes
requires some effort: choosing the base platform (spring-boot, karaf,
simple main?), adding health checks (actuator?), packaging a docker image
and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
publishing the image on a docker registry, then finally deploying the
resources on a Kubernetes cluster.

The resulting integration container is then far from being optimal from a
resource consumption point of view: it is likely that a Camel Spring-Boot
application will require at least 200MB of RAM and also some CPU shares
because of polling threads used by many components.

In case people use a CI/CD pipeline, it will take also a long time to get
from a code update to having a Kubernetes POD up and running.
Apart from compilation and image push/pull time, also startup time is often
~10 seconds for Camel + Spring-Boot in a container with standard limits on
resources, making it difficult to propose this combination for "serverless
integration" (this term is becoming increasingly more popular).

So, my proposal is to start to investigate a "more cloud-native" approach
to integration: *making Camel integrations first-class citizens in
Kubernetes, and making them super fast and lightweight.*

We can base the project on Kubernetes Custom Resource Definitions (CRD)
<https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>,
for example a Integration CRD and have a Kubernetes "operator"
<https://coreos.com/operators/> taking care of:
- Optimizing the integration that we want to run
- Packaging in a container
- Running it on Kubernetes
- Managing its entire lifecycle

A Kubernetes-native integration may look like:

-------------------
kind: "Integration"
apiVersion: "camel.apache.org/v1alpha1"
metadata:
 name: "example"
spec:
 replicas: 1
 routes:
  - id: timer
    route:
     - type: endpoint
       uri: timer:tick
     - type: endpoint
       uri: log:info
-------------------

For those who are not familiar with Kubernetes resources, this kind of
YAML/JSON resource definitions are really common.
The example route is embedded in the Kubernetes resource declaration and
follows a basic "flow DSL". We may start from a basic one and evolve it as
new requirements arrive from the community.

I've made a very simple (but working) POC here:
https://github.com/nicolaferraro/integration-operator.

This idea of a "Cloud-Native Camel" on Kubernetes (project codename can be "
*Kamel*", if you like it :D), will be an enabler for a lot of nice features.

For example, we can propose "Kamel" as "ideal" platform for "serverless
integration" (I see many people reinventing the wheel out there): the
operator can reduce resource consumption of a single integration by
optimizing the runtime and also pause/resume integrations when they are not
used, that is the basic idea behind "serverless" (e.g. think to
HTTP-triggered integrations, but not only).
Focusing on serverless will bring more emphasis on push-based notifications
(webhooks, cloud events <https://cloudevents.io/>), that are rarely used in
Camel components, that prefer a poll based approach being it simpler to use
in classic deployments, but not so good in the cloud, where more resources
become higher direct costs for the users.

The presence of the simplified DSL enables also experimenting on "*reduced*
subsets of Camel" implemented in languages other than Java, for example one
language that has a reactive approach on thread scheduling and a really low
memory footprint, like Go.

But apart from this kind of experiments (that are valid IMO), the "Kamel"
optimizer will have free room to choose the right platform for the
integration that the user wants to run, including, in the future, doing AOT
compilation using Graal/VM (less memory, faster startup) if the features
(components) used in the integration are supporting it (maybe we can add
AOT compilation in the roadmap for Camel 3).
A silly optimization: integrations starting from "timer:..." may be
scheduled directly with Kubernetes CronJobs, so they will consume resources
only when actually running.

Being the final integrations lightweight and being the DSL
language-independent, we may see a increased adoption of Camel also as
agile integration layer for not-only-java applications (both "cloud" and
"serverless" applications).

I'm the first one that would like to work on a project ilke this. I've
worked on many Kubernetes/Openshift based applications and frameworks in
the past years, also on operators and CRDs, and I think this way of
redesigning integrations has a lot of potential.

Integrations will not be necessarily limited to the simplified DSL, but we
can add extension points for scripting and even custom libraries (although
limiting the freedom of the optimizer).

The most important thing: it may become a great project, since it's driven
by a great community.

So, what do you think? Is it crazy enough?

Nicola
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Johan Edstrom-2
I find it super +1 - If not only for your awesome narrative.



> On Jul 12, 2018, at 5:30 PM, Nicola Ferraro <[hidden email]> wrote:
>
> Hi Cameleers,
> it's now passed some time since I started thinking about a new project that
> we can begin here at Apache Camel, and I'd like to have your opinion.
>
> We've already been targeting cloud-native applications with Camel,
> especially on top of Kubernetes, that is becoming "the standard" cloud
> platform. But writing a Camel integration and running it on Kubernetes
> requires some effort: choosing the base platform (spring-boot, karaf,
> simple main?), adding health checks (actuator?), packaging a docker image
> and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> publishing the image on a docker registry, then finally deploying the
> resources on a Kubernetes cluster.
>
> The resulting integration container is then far from being optimal from a
> resource consumption point of view: it is likely that a Camel Spring-Boot
> application will require at least 200MB of RAM and also some CPU shares
> because of polling threads used by many components.
>
> In case people use a CI/CD pipeline, it will take also a long time to get
> from a code update to having a Kubernetes POD up and running.
> Apart from compilation and image push/pull time, also startup time is often
> ~10 seconds for Camel + Spring-Boot in a container with standard limits on
> resources, making it difficult to propose this combination for "serverless
> integration" (this term is becoming increasingly more popular).
>
> So, my proposal is to start to investigate a "more cloud-native" approach
> to integration: *making Camel integrations first-class citizens in
> Kubernetes, and making them super fast and lightweight.*
>
> We can base the project on Kubernetes Custom Resource Definitions (CRD)
> <https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>,
> for example a Integration CRD and have a Kubernetes "operator"
> <https://coreos.com/operators/> taking care of:
> - Optimizing the integration that we want to run
> - Packaging in a container
> - Running it on Kubernetes
> - Managing its entire lifecycle
>
> A Kubernetes-native integration may look like:
>
> -------------------
> kind: "Integration"
> apiVersion: "camel.apache.org/v1alpha1"
> metadata:
> name: "example"
> spec:
> replicas: 1
> routes:
>  - id: timer
>    route:
>     - type: endpoint
>       uri: timer:tick
>     - type: endpoint
>       uri: log:info
> -------------------
>
> For those who are not familiar with Kubernetes resources, this kind of
> YAML/JSON resource definitions are really common.
> The example route is embedded in the Kubernetes resource declaration and
> follows a basic "flow DSL". We may start from a basic one and evolve it as
> new requirements arrive from the community.
>
> I've made a very simple (but working) POC here:
> https://github.com/nicolaferraro/integration-operator.
>
> This idea of a "Cloud-Native Camel" on Kubernetes (project codename can be "
> *Kamel*", if you like it :D), will be an enabler for a lot of nice features.
>
> For example, we can propose "Kamel" as "ideal" platform for "serverless
> integration" (I see many people reinventing the wheel out there): the
> operator can reduce resource consumption of a single integration by
> optimizing the runtime and also pause/resume integrations when they are not
> used, that is the basic idea behind "serverless" (e.g. think to
> HTTP-triggered integrations, but not only).
> Focusing on serverless will bring more emphasis on push-based notifications
> (webhooks, cloud events <https://cloudevents.io/>), that are rarely used in
> Camel components, that prefer a poll based approach being it simpler to use
> in classic deployments, but not so good in the cloud, where more resources
> become higher direct costs for the users.
>
> The presence of the simplified DSL enables also experimenting on "*reduced*
> subsets of Camel" implemented in languages other than Java, for example one
> language that has a reactive approach on thread scheduling and a really low
> memory footprint, like Go.
>
> But apart from this kind of experiments (that are valid IMO), the "Kamel"
> optimizer will have free room to choose the right platform for the
> integration that the user wants to run, including, in the future, doing AOT
> compilation using Graal/VM (less memory, faster startup) if the features
> (components) used in the integration are supporting it (maybe we can add
> AOT compilation in the roadmap for Camel 3).
> A silly optimization: integrations starting from "timer:..." may be
> scheduled directly with Kubernetes CronJobs, so they will consume resources
> only when actually running.
>
> Being the final integrations lightweight and being the DSL
> language-independent, we may see a increased adoption of Camel also as
> agile integration layer for not-only-java applications (both "cloud" and
> "serverless" applications).
>
> I'm the first one that would like to work on a project ilke this. I've
> worked on many Kubernetes/Openshift based applications and frameworks in
> the past years, also on operators and CRDs, and I think this way of
> redesigning integrations has a lot of potential.
>
> Integrations will not be necessarily limited to the simplified DSL, but we
> can add extension points for scripting and even custom libraries (although
> limiting the freedom of the optimizer).
>
> The most important thing: it may become a great project, since it's driven
> by a great community.
>
> So, what do you think? Is it crazy enough?
>
> Nicola

Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Johan Edstrom-2
In reply to this post by Nicola Ferraro
Jeff Genender, I and James Carman long ago were tossing
around the idea of Ibex, it would have been a Scala based AKKA
eco system for putting in routes. You’d just say run in this namespace, conform
to these Actor roles and we compose the tree for you.

What you propose is a bit easier and more tangible to attract space
for Camel as an engine to well, keep open source in “I can’t even write a main class”

:)

> On Jul 12, 2018, at 5:30 PM, Nicola Ferraro <[hidden email]> wrote:
>
> Hi Cameleers,
> it's now passed some time since I started thinking about a new project that
> we can begin here at Apache Camel, and I'd like to have your opinion.
>
> We've already been targeting cloud-native applications with Camel,
> especially on top of Kubernetes, that is becoming "the standard" cloud
> platform. But writing a Camel integration and running it on Kubernetes
> requires some effort: choosing the base platform (spring-boot, karaf,
> simple main?), adding health checks (actuator?), packaging a docker image
> and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> publishing the image on a docker registry, then finally deploying the
> resources on a Kubernetes cluster.
>
> The resulting integration container is then far from being optimal from a
> resource consumption point of view: it is likely that a Camel Spring-Boot
> application will require at least 200MB of RAM and also some CPU shares
> because of polling threads used by many components.
>
> In case people use a CI/CD pipeline, it will take also a long time to get
> from a code update to having a Kubernetes POD up and running.
> Apart from compilation and image push/pull time, also startup time is often
> ~10 seconds for Camel + Spring-Boot in a container with standard limits on
> resources, making it difficult to propose this combination for "serverless
> integration" (this term is becoming increasingly more popular).
>
> So, my proposal is to start to investigate a "more cloud-native" approach
> to integration: *making Camel integrations first-class citizens in
> Kubernetes, and making them super fast and lightweight.*
>
> We can base the project on Kubernetes Custom Resource Definitions (CRD)
> <https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>,
> for example a Integration CRD and have a Kubernetes "operator"
> <https://coreos.com/operators/> taking care of:
> - Optimizing the integration that we want to run
> - Packaging in a container
> - Running it on Kubernetes
> - Managing its entire lifecycle
>
> A Kubernetes-native integration may look like:
>
> -------------------
> kind: "Integration"
> apiVersion: "camel.apache.org/v1alpha1"
> metadata:
> name: "example"
> spec:
> replicas: 1
> routes:
>  - id: timer
>    route:
>     - type: endpoint
>       uri: timer:tick
>     - type: endpoint
>       uri: log:info
> -------------------
>
> For those who are not familiar with Kubernetes resources, this kind of
> YAML/JSON resource definitions are really common.
> The example route is embedded in the Kubernetes resource declaration and
> follows a basic "flow DSL". We may start from a basic one and evolve it as
> new requirements arrive from the community.
>
> I've made a very simple (but working) POC here:
> https://github.com/nicolaferraro/integration-operator.
>
> This idea of a "Cloud-Native Camel" on Kubernetes (project codename can be "
> *Kamel*", if you like it :D), will be an enabler for a lot of nice features.
>
> For example, we can propose "Kamel" as "ideal" platform for "serverless
> integration" (I see many people reinventing the wheel out there): the
> operator can reduce resource consumption of a single integration by
> optimizing the runtime and also pause/resume integrations when they are not
> used, that is the basic idea behind "serverless" (e.g. think to
> HTTP-triggered integrations, but not only).
> Focusing on serverless will bring more emphasis on push-based notifications
> (webhooks, cloud events <https://cloudevents.io/>), that are rarely used in
> Camel components, that prefer a poll based approach being it simpler to use
> in classic deployments, but not so good in the cloud, where more resources
> become higher direct costs for the users.
>
> The presence of the simplified DSL enables also experimenting on "*reduced*
> subsets of Camel" implemented in languages other than Java, for example one
> language that has a reactive approach on thread scheduling and a really low
> memory footprint, like Go.
>
> But apart from this kind of experiments (that are valid IMO), the "Kamel"
> optimizer will have free room to choose the right platform for the
> integration that the user wants to run, including, in the future, doing AOT
> compilation using Graal/VM (less memory, faster startup) if the features
> (components) used in the integration are supporting it (maybe we can add
> AOT compilation in the roadmap for Camel 3).
> A silly optimization: integrations starting from "timer:..." may be
> scheduled directly with Kubernetes CronJobs, so they will consume resources
> only when actually running.
>
> Being the final integrations lightweight and being the DSL
> language-independent, we may see a increased adoption of Camel also as
> agile integration layer for not-only-java applications (both "cloud" and
> "serverless" applications).
>
> I'm the first one that would like to work on a project ilke this. I've
> worked on many Kubernetes/Openshift based applications and frameworks in
> the past years, also on operators and CRDs, and I think this way of
> redesigning integrations has a lot of potential.
>
> Integrations will not be necessarily limited to the simplified DSL, but we
> can add extension points for scripting and even custom libraries (although
> limiting the freedom of the optimizer).
>
> The most important thing: it may become a great project, since it's driven
> by a great community.
>
> So, what do you think? Is it crazy enough?
>
> Nicola

Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Willem.Jiang
Administrator
Yeah, it's a really good idea to combine the K8S with Camel.
In my mind if we want to host a camel application on the cloud,  it could
be first step that we can run the camel engine on demand in K8S.

So I really like the project idea  *making Camel integrations first-class
citizens in Kubernetes, and making them super fast and lightweight.*

I think the Apache Openwhisk is doing the same thing here[1].

[1]https://github.com/apache/incubator-openwhisk-deploy-kube




Willem Jiang

Twitter: willemjiang
Weibo: 姜宁willem

On Fri, Jul 13, 2018 at 8:31 AM, Johan Edstrom <[hidden email]> wrote:

> Jeff Genender, I and James Carman long ago were tossing
> around the idea of Ibex, it would have been a Scala based AKKA
> eco system for putting in routes. You’d just say run in this namespace,
> conform
> to these Actor roles and we compose the tree for you.
>
> What you propose is a bit easier and more tangible to attract space
> for Camel as an engine to well, keep open source in “I can’t even write a
> main class”
>
> :)
>
> > On Jul 12, 2018, at 5:30 PM, Nicola Ferraro <[hidden email]>
> wrote:
> >
> > Hi Cameleers,
> > it's now passed some time since I started thinking about a new project
> that
> > we can begin here at Apache Camel, and I'd like to have your opinion.
> >
> > We've already been targeting cloud-native applications with Camel,
> > especially on top of Kubernetes, that is becoming "the standard" cloud
> > platform. But writing a Camel integration and running it on Kubernetes
> > requires some effort: choosing the base platform (spring-boot, karaf,
> > simple main?), adding health checks (actuator?), packaging a docker image
> > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > publishing the image on a docker registry, then finally deploying the
> > resources on a Kubernetes cluster.
> >
> > The resulting integration container is then far from being optimal from a
> > resource consumption point of view: it is likely that a Camel Spring-Boot
> > application will require at least 200MB of RAM and also some CPU shares
> > because of polling threads used by many components.
> >
> > In case people use a CI/CD pipeline, it will take also a long time to get
> > from a code update to having a Kubernetes POD up and running.
> > Apart from compilation and image push/pull time, also startup time is
> often
> > ~10 seconds for Camel + Spring-Boot in a container with standard limits
> on
> > resources, making it difficult to propose this combination for
> "serverless
> > integration" (this term is becoming increasingly more popular).
> >
> > So, my proposal is to start to investigate a "more cloud-native" approach
> > to integration: *making Camel integrations first-class citizens in
> > Kubernetes, and making them super fast and lightweight.*
> >
> > We can base the project on Kubernetes Custom Resource Definitions (CRD)
> > <https://kubernetes.io/docs/concepts/extend-kubernetes/
> api-extension/custom-resources/>,
> > for example a Integration CRD and have a Kubernetes "operator"
> > <https://coreos.com/operators/> taking care of:
> > - Optimizing the integration that we want to run
> > - Packaging in a container
> > - Running it on Kubernetes
> > - Managing its entire lifecycle
> >
> > A Kubernetes-native integration may look like:
> >
> > -------------------
> > kind: "Integration"
> > apiVersion: "camel.apache.org/v1alpha1"
> > metadata:
> > name: "example"
> > spec:
> > replicas: 1
> > routes:
> >  - id: timer
> >    route:
> >     - type: endpoint
> >       uri: timer:tick
> >     - type: endpoint
> >       uri: log:info
> > -------------------
> >
> > For those who are not familiar with Kubernetes resources, this kind of
> > YAML/JSON resource definitions are really common.
> > The example route is embedded in the Kubernetes resource declaration and
> > follows a basic "flow DSL". We may start from a basic one and evolve it
> as
> > new requirements arrive from the community.
> >
> > I've made a very simple (but working) POC here:
> > https://github.com/nicolaferraro/integration-operator.
> >
> > This idea of a "Cloud-Native Camel" on Kubernetes (project codename can
> be "
> > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> features.
> >
> > For example, we can propose "Kamel" as "ideal" platform for "serverless
> > integration" (I see many people reinventing the wheel out there): the
> > operator can reduce resource consumption of a single integration by
> > optimizing the runtime and also pause/resume integrations when they are
> not
> > used, that is the basic idea behind "serverless" (e.g. think to
> > HTTP-triggered integrations, but not only).
> > Focusing on serverless will bring more emphasis on push-based
> notifications
> > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> used in
> > Camel components, that prefer a poll based approach being it simpler to
> use
> > in classic deployments, but not so good in the cloud, where more
> resources
> > become higher direct costs for the users.
> >
> > The presence of the simplified DSL enables also experimenting on
> "*reduced*
> > subsets of Camel" implemented in languages other than Java, for example
> one
> > language that has a reactive approach on thread scheduling and a really
> low
> > memory footprint, like Go.
> >
> > But apart from this kind of experiments (that are valid IMO), the "Kamel"
> > optimizer will have free room to choose the right platform for the
> > integration that the user wants to run, including, in the future, doing
> AOT
> > compilation using Graal/VM (less memory, faster startup) if the features
> > (components) used in the integration are supporting it (maybe we can add
> > AOT compilation in the roadmap for Camel 3).
> > A silly optimization: integrations starting from "timer:..." may be
> > scheduled directly with Kubernetes CronJobs, so they will consume
> resources
> > only when actually running.
> >
> > Being the final integrations lightweight and being the DSL
> > language-independent, we may see a increased adoption of Camel also as
> > agile integration layer for not-only-java applications (both "cloud" and
> > "serverless" applications).
> >
> > I'm the first one that would like to work on a project ilke this. I've
> > worked on many Kubernetes/Openshift based applications and frameworks in
> > the past years, also on operators and CRDs, and I think this way of
> > redesigning integrations has a lot of potential.
> >
> > Integrations will not be necessarily limited to the simplified DSL, but
> we
> > can add extension points for scripting and even custom libraries
> (although
> > limiting the freedom of the optimizer).
> >
> > The most important thing: it may become a great project, since it's
> driven
> > by a great community.
> >
> > So, what do you think? Is it crazy enough?
> >
> > Nicola
>
>
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Onder SEZGIN
Hi,

This can make Kamel language, dsl agnostic and will give more room to
integrate faster.
Definitely +1.



On Fri, Jul 13, 2018 at 3:54 AM Willem Jiang <[hidden email]> wrote:

> Yeah, it's a really good idea to combine the K8S with Camel.
> In my mind if we want to host a camel application on the cloud,  it could
> be first step that we can run the camel engine on demand in K8S.
>
> So I really like the project idea  *making Camel integrations first-class
> citizens in Kubernetes, and making them super fast and lightweight.*
>
> I think the Apache Openwhisk is doing the same thing here[1].
>
> [1]https://github.com/apache/incubator-openwhisk-deploy-kube
>
>
>
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Fri, Jul 13, 2018 at 8:31 AM, Johan Edstrom <[hidden email]> wrote:
>
> > Jeff Genender, I and James Carman long ago were tossing
> > around the idea of Ibex, it would have been a Scala based AKKA
> > eco system for putting in routes. You’d just say run in this namespace,
> > conform
> > to these Actor roles and we compose the tree for you.
> >
> > What you propose is a bit easier and more tangible to attract space
> > for Camel as an engine to well, keep open source in “I can’t even write a
> > main class”
> >
> > :)
> >
> > > On Jul 12, 2018, at 5:30 PM, Nicola Ferraro <[hidden email]>
> > wrote:
> > >
> > > Hi Cameleers,
> > > it's now passed some time since I started thinking about a new project
> > that
> > > we can begin here at Apache Camel, and I'd like to have your opinion.
> > >
> > > We've already been targeting cloud-native applications with Camel,
> > > especially on top of Kubernetes, that is becoming "the standard" cloud
> > > platform. But writing a Camel integration and running it on Kubernetes
> > > requires some effort: choosing the base platform (spring-boot, karaf,
> > > simple main?), adding health checks (actuator?), packaging a docker
> image
> > > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > > publishing the image on a docker registry, then finally deploying the
> > > resources on a Kubernetes cluster.
> > >
> > > The resulting integration container is then far from being optimal
> from a
> > > resource consumption point of view: it is likely that a Camel
> Spring-Boot
> > > application will require at least 200MB of RAM and also some CPU shares
> > > because of polling threads used by many components.
> > >
> > > In case people use a CI/CD pipeline, it will take also a long time to
> get
> > > from a code update to having a Kubernetes POD up and running.
> > > Apart from compilation and image push/pull time, also startup time is
> > often
> > > ~10 seconds for Camel + Spring-Boot in a container with standard limits
> > on
> > > resources, making it difficult to propose this combination for
> > "serverless
> > > integration" (this term is becoming increasingly more popular).
> > >
> > > So, my proposal is to start to investigate a "more cloud-native"
> approach
> > > to integration: *making Camel integrations first-class citizens in
> > > Kubernetes, and making them super fast and lightweight.*
> > >
> > > We can base the project on Kubernetes Custom Resource Definitions (CRD)
> > > <https://kubernetes.io/docs/concepts/extend-kubernetes/
> > api-extension/custom-resources/>,
> > > for example a Integration CRD and have a Kubernetes "operator"
> > > <https://coreos.com/operators/> taking care of:
> > > - Optimizing the integration that we want to run
> > > - Packaging in a container
> > > - Running it on Kubernetes
> > > - Managing its entire lifecycle
> > >
> > > A Kubernetes-native integration may look like:
> > >
> > > -------------------
> > > kind: "Integration"
> > > apiVersion: "camel.apache.org/v1alpha1"
> > > metadata:
> > > name: "example"
> > > spec:
> > > replicas: 1
> > > routes:
> > >  - id: timer
> > >    route:
> > >     - type: endpoint
> > >       uri: timer:tick
> > >     - type: endpoint
> > >       uri: log:info
> > > -------------------
> > >
> > > For those who are not familiar with Kubernetes resources, this kind of
> > > YAML/JSON resource definitions are really common.
> > > The example route is embedded in the Kubernetes resource declaration
> and
> > > follows a basic "flow DSL". We may start from a basic one and evolve it
> > as
> > > new requirements arrive from the community.
> > >
> > > I've made a very simple (but working) POC here:
> > > https://github.com/nicolaferraro/integration-operator.
> > >
> > > This idea of a "Cloud-Native Camel" on Kubernetes (project codename can
> > be "
> > > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> > features.
> > >
> > > For example, we can propose "Kamel" as "ideal" platform for "serverless
> > > integration" (I see many people reinventing the wheel out there): the
> > > operator can reduce resource consumption of a single integration by
> > > optimizing the runtime and also pause/resume integrations when they are
> > not
> > > used, that is the basic idea behind "serverless" (e.g. think to
> > > HTTP-triggered integrations, but not only).
> > > Focusing on serverless will bring more emphasis on push-based
> > notifications
> > > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> > used in
> > > Camel components, that prefer a poll based approach being it simpler to
> > use
> > > in classic deployments, but not so good in the cloud, where more
> > resources
> > > become higher direct costs for the users.
> > >
> > > The presence of the simplified DSL enables also experimenting on
> > "*reduced*
> > > subsets of Camel" implemented in languages other than Java, for example
> > one
> > > language that has a reactive approach on thread scheduling and a really
> > low
> > > memory footprint, like Go.
> > >
> > > But apart from this kind of experiments (that are valid IMO), the
> "Kamel"
> > > optimizer will have free room to choose the right platform for the
> > > integration that the user wants to run, including, in the future, doing
> > AOT
> > > compilation using Graal/VM (less memory, faster startup) if the
> features
> > > (components) used in the integration are supporting it (maybe we can
> add
> > > AOT compilation in the roadmap for Camel 3).
> > > A silly optimization: integrations starting from "timer:..." may be
> > > scheduled directly with Kubernetes CronJobs, so they will consume
> > resources
> > > only when actually running.
> > >
> > > Being the final integrations lightweight and being the DSL
> > > language-independent, we may see a increased adoption of Camel also as
> > > agile integration layer for not-only-java applications (both "cloud"
> and
> > > "serverless" applications).
> > >
> > > I'm the first one that would like to work on a project ilke this. I've
> > > worked on many Kubernetes/Openshift based applications and frameworks
> in
> > > the past years, also on operators and CRDs, and I think this way of
> > > redesigning integrations has a lot of potential.
> > >
> > > Integrations will not be necessarily limited to the simplified DSL, but
> > we
> > > can add extension points for scripting and even custom libraries
> > (although
> > > limiting the freedom of the optimizer).
> > >
> > > The most important thing: it may become a great project, since it's
> > driven
> > > by a great community.
> > >
> > > So, what do you think? Is it crazy enough?
> > >
> > > Nicola
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Claus Ibsen-2
In reply to this post by Nicola Ferraro
+1

Great idea and love the Kamel name.



On Fri, Jul 13, 2018 at 1:30 AM, Nicola Ferraro <[hidden email]> wrote:

> Hi Cameleers,
> it's now passed some time since I started thinking about a new project that
> we can begin here at Apache Camel, and I'd like to have your opinion.
>
> We've already been targeting cloud-native applications with Camel,
> especially on top of Kubernetes, that is becoming "the standard" cloud
> platform. But writing a Camel integration and running it on Kubernetes
> requires some effort: choosing the base platform (spring-boot, karaf,
> simple main?), adding health checks (actuator?), packaging a docker image
> and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> publishing the image on a docker registry, then finally deploying the
> resources on a Kubernetes cluster.
>
> The resulting integration container is then far from being optimal from a
> resource consumption point of view: it is likely that a Camel Spring-Boot
> application will require at least 200MB of RAM and also some CPU shares
> because of polling threads used by many components.
>
> In case people use a CI/CD pipeline, it will take also a long time to get
> from a code update to having a Kubernetes POD up and running.
> Apart from compilation and image push/pull time, also startup time is often
> ~10 seconds for Camel + Spring-Boot in a container with standard limits on
> resources, making it difficult to propose this combination for "serverless
> integration" (this term is becoming increasingly more popular).
>
> So, my proposal is to start to investigate a "more cloud-native" approach
> to integration: *making Camel integrations first-class citizens in
> Kubernetes, and making them super fast and lightweight.*
>
> We can base the project on Kubernetes Custom Resource Definitions (CRD)
> <https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>,
> for example a Integration CRD and have a Kubernetes "operator"
> <https://coreos.com/operators/> taking care of:
> - Optimizing the integration that we want to run
> - Packaging in a container
> - Running it on Kubernetes
> - Managing its entire lifecycle
>
> A Kubernetes-native integration may look like:
>
> -------------------
> kind: "Integration"
> apiVersion: "camel.apache.org/v1alpha1"
> metadata:
>  name: "example"
> spec:
>  replicas: 1
>  routes:
>   - id: timer
>     route:
>      - type: endpoint
>        uri: timer:tick
>      - type: endpoint
>        uri: log:info
> -------------------
>
> For those who are not familiar with Kubernetes resources, this kind of
> YAML/JSON resource definitions are really common.
> The example route is embedded in the Kubernetes resource declaration and
> follows a basic "flow DSL". We may start from a basic one and evolve it as
> new requirements arrive from the community.
>
> I've made a very simple (but working) POC here:
> https://github.com/nicolaferraro/integration-operator.
>
> This idea of a "Cloud-Native Camel" on Kubernetes (project codename can be "
> *Kamel*", if you like it :D), will be an enabler for a lot of nice features.
>
> For example, we can propose "Kamel" as "ideal" platform for "serverless
> integration" (I see many people reinventing the wheel out there): the
> operator can reduce resource consumption of a single integration by
> optimizing the runtime and also pause/resume integrations when they are not
> used, that is the basic idea behind "serverless" (e.g. think to
> HTTP-triggered integrations, but not only).
> Focusing on serverless will bring more emphasis on push-based notifications
> (webhooks, cloud events <https://cloudevents.io/>), that are rarely used in
> Camel components, that prefer a poll based approach being it simpler to use
> in classic deployments, but not so good in the cloud, where more resources
> become higher direct costs for the users.
>
> The presence of the simplified DSL enables also experimenting on "*reduced*
> subsets of Camel" implemented in languages other than Java, for example one
> language that has a reactive approach on thread scheduling and a really low
> memory footprint, like Go.
>
> But apart from this kind of experiments (that are valid IMO), the "Kamel"
> optimizer will have free room to choose the right platform for the
> integration that the user wants to run, including, in the future, doing AOT
> compilation using Graal/VM (less memory, faster startup) if the features
> (components) used in the integration are supporting it (maybe we can add
> AOT compilation in the roadmap for Camel 3).
> A silly optimization: integrations starting from "timer:..." may be
> scheduled directly with Kubernetes CronJobs, so they will consume resources
> only when actually running.
>
> Being the final integrations lightweight and being the DSL
> language-independent, we may see a increased adoption of Camel also as
> agile integration layer for not-only-java applications (both "cloud" and
> "serverless" applications).
>
> I'm the first one that would like to work on a project ilke this. I've
> worked on many Kubernetes/Openshift based applications and frameworks in
> the past years, also on operators and CRDs, and I think this way of
> redesigning integrations has a lot of potential.
>
> Integrations will not be necessarily limited to the simplified DSL, but we
> can add extension points for scripting and even custom libraries (although
> limiting the freedom of the optimizer).
>
> The most important thing: it may become a great project, since it's driven
> by a great community.
>
> So, what do you think? Is it crazy enough?
>
> Nicola



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

R: Re: A "Kamel" crazy idea

Andrea Cosentino-2
+1
Great idea! 
 
  Il ven, 13 lug, 2018 alle 7:53, Claus Ibsen<[hidden email]> ha scritto:   +1

Great idea and love the Kamel name.



On Fri, Jul 13, 2018 at 1:30 AM, Nicola Ferraro <[hidden email]> wrote:

> Hi Cameleers,
> it's now passed some time since I started thinking about a new project that
> we can begin here at Apache Camel, and I'd like to have your opinion.
>
> We've already been targeting cloud-native applications with Camel,
> especially on top of Kubernetes, that is becoming "the standard" cloud
> platform. But writing a Camel integration and running it on Kubernetes
> requires some effort: choosing the base platform (spring-boot, karaf,
> simple main?), adding health checks (actuator?), packaging a docker image
> and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> publishing the image on a docker registry, then finally deploying the
> resources on a Kubernetes cluster.
>
> The resulting integration container is then far from being optimal from a
> resource consumption point of view: it is likely that a Camel Spring-Boot
> application will require at least 200MB of RAM and also some CPU shares
> because of polling threads used by many components.
>
> In case people use a CI/CD pipeline, it will take also a long time to get
> from a code update to having a Kubernetes POD up and running.
> Apart from compilation and image push/pull time, also startup time is often
> ~10 seconds for Camel + Spring-Boot in a container with standard limits on
> resources, making it difficult to propose this combination for "serverless
> integration" (this term is becoming increasingly more popular).
>
> So, my proposal is to start to investigate a "more cloud-native" approach
> to integration: *making Camel integrations first-class citizens in
> Kubernetes, and making them super fast and lightweight.*
>
> We can base the project on Kubernetes Custom Resource Definitions (CRD)
> <https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>,
> for example a Integration CRD and have a Kubernetes "operator"
> <https://coreos.com/operators/> taking care of:
> - Optimizing the integration that we want to run
> - Packaging in a container
> - Running it on Kubernetes
> - Managing its entire lifecycle
>
> A Kubernetes-native integration may look like:
>
> -------------------
> kind: "Integration"
> apiVersion: "camel.apache.org/v1alpha1"
> metadata:
>  name: "example"
> spec:
>  replicas: 1
>  routes:
>  - id: timer
>    route:
>      - type: endpoint
>        uri: timer:tick
>      - type: endpoint
>        uri: log:info
> -------------------
>
> For those who are not familiar with Kubernetes resources, this kind of
> YAML/JSON resource definitions are really common.
> The example route is embedded in the Kubernetes resource declaration and
> follows a basic "flow DSL". We may start from a basic one and evolve it as
> new requirements arrive from the community.
>
> I've made a very simple (but working) POC here:
> https://github.com/nicolaferraro/integration-operator.
>
> This idea of a "Cloud-Native Camel" on Kubernetes (project codename can be "
> *Kamel*", if you like it :D), will be an enabler for a lot of nice features.
>
> For example, we can propose "Kamel" as "ideal" platform for "serverless
> integration" (I see many people reinventing the wheel out there): the
> operator can reduce resource consumption of a single integration by
> optimizing the runtime and also pause/resume integrations when they are not
> used, that is the basic idea behind "serverless" (e.g. think to
> HTTP-triggered integrations, but not only).
> Focusing on serverless will bring more emphasis on push-based notifications
> (webhooks, cloud events <https://cloudevents.io/>), that are rarely used in
> Camel components, that prefer a poll based approach being it simpler to use
> in classic deployments, but not so good in the cloud, where more resources
> become higher direct costs for the users.
>
> The presence of the simplified DSL enables also experimenting on "*reduced*
> subsets of Camel" implemented in languages other than Java, for example one
> language that has a reactive approach on thread scheduling and a really low
> memory footprint, like Go.
>
> But apart from this kind of experiments (that are valid IMO), the "Kamel"
> optimizer will have free room to choose the right platform for the
> integration that the user wants to run, including, in the future, doing AOT
> compilation using Graal/VM (less memory, faster startup) if the features
> (components) used in the integration are supporting it (maybe we can add
> AOT compilation in the roadmap for Camel 3).
> A silly optimization: integrations starting from "timer:..." may be
> scheduled directly with Kubernetes CronJobs, so they will consume resources
> only when actually running.
>
> Being the final integrations lightweight and being the DSL
> language-independent, we may see a increased adoption of Camel also as
> agile integration layer for not-only-java applications (both "cloud" and
> "serverless" applications).
>
> I'm the first one that would like to work on a project ilke this. I've
> worked on many Kubernetes/Openshift based applications and frameworks in
> the past years, also on operators and CRDs, and I think this way of
> redesigning integrations has a lot of potential.
>
> Integrations will not be necessarily limited to the simplified DSL, but we
> can add extension points for scripting and even custom libraries (although
> limiting the freedom of the optimizer).
>
> The most important thing: it may become a great project, since it's driven
> by a great community.
>
> So, what do you think? Is it crazy enough?
>
> Nicola



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
 
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Sascha Dirbach
In reply to this post by Nicola Ferraro
Hi Nicola,

+1

Great idea, do you already have a concept/idea/sketch on how to deal
with complex configurations? i.e. SSL certs for outgoing calls, XSLT
transformations, etc.

When I think about it, you could probably use configmaps/secrets to
mount these config in the container an then reference them in the route.

Best regards,

Sascha

Am 13.07.2018 um 01:30 schrieb Nicola Ferraro:

> Hi Cameleers,
> it's now passed some time since I started thinking about a new project that
> we can begin here at Apache Camel, and I'd like to have your opinion.
>
> We've already been targeting cloud-native applications with Camel,
> especially on top of Kubernetes, that is becoming "the standard" cloud
> platform. But writing a Camel integration and running it on Kubernetes
> requires some effort: choosing the base platform (spring-boot, karaf,
> simple main?), adding health checks (actuator?), packaging a docker image
> and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> publishing the image on a docker registry, then finally deploying the
> resources on a Kubernetes cluster.
>
> The resulting integration container is then far from being optimal from a
> resource consumption point of view: it is likely that a Camel Spring-Boot
> application will require at least 200MB of RAM and also some CPU shares
> because of polling threads used by many components.
>
> In case people use a CI/CD pipeline, it will take also a long time to get
> from a code update to having a Kubernetes POD up and running.
> Apart from compilation and image push/pull time, also startup time is often
> ~10 seconds for Camel + Spring-Boot in a container with standard limits on
> resources, making it difficult to propose this combination for "serverless
> integration" (this term is becoming increasingly more popular).
>
> So, my proposal is to start to investigate a "more cloud-native" approach
> to integration: *making Camel integrations first-class citizens in
> Kubernetes, and making them super fast and lightweight.*
>
> We can base the project on Kubernetes Custom Resource Definitions (CRD)
> <https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>,
> for example a Integration CRD and have a Kubernetes "operator"
> <https://coreos.com/operators/> taking care of:
> - Optimizing the integration that we want to run
> - Packaging in a container
> - Running it on Kubernetes
> - Managing its entire lifecycle
>
> A Kubernetes-native integration may look like:
>
> -------------------
> kind: "Integration"
> apiVersion: "camel.apache.org/v1alpha1"
> metadata:
>  name: "example"
> spec:
>  replicas: 1
>  routes:
>   - id: timer
>     route:
>      - type: endpoint
>        uri: timer:tick
>      - type: endpoint
>        uri: log:info
> -------------------
>
> For those who are not familiar with Kubernetes resources, this kind of
> YAML/JSON resource definitions are really common.
> The example route is embedded in the Kubernetes resource declaration and
> follows a basic "flow DSL". We may start from a basic one and evolve it as
> new requirements arrive from the community.
>
> I've made a very simple (but working) POC here:
> https://github.com/nicolaferraro/integration-operator.
>
> This idea of a "Cloud-Native Camel" on Kubernetes (project codename can be "
> *Kamel*", if you like it :D), will be an enabler for a lot of nice features.
>
> For example, we can propose "Kamel" as "ideal" platform for "serverless
> integration" (I see many people reinventing the wheel out there): the
> operator can reduce resource consumption of a single integration by
> optimizing the runtime and also pause/resume integrations when they are not
> used, that is the basic idea behind "serverless" (e.g. think to
> HTTP-triggered integrations, but not only).
> Focusing on serverless will bring more emphasis on push-based notifications
> (webhooks, cloud events <https://cloudevents.io/>), that are rarely used in
> Camel components, that prefer a poll based approach being it simpler to use
> in classic deployments, but not so good in the cloud, where more resources
> become higher direct costs for the users.
>
> The presence of the simplified DSL enables also experimenting on "*reduced*
> subsets of Camel" implemented in languages other than Java, for example one
> language that has a reactive approach on thread scheduling and a really low
> memory footprint, like Go.
>
> But apart from this kind of experiments (that are valid IMO), the "Kamel"
> optimizer will have free room to choose the right platform for the
> integration that the user wants to run, including, in the future, doing AOT
> compilation using Graal/VM (less memory, faster startup) if the features
> (components) used in the integration are supporting it (maybe we can add
> AOT compilation in the roadmap for Camel 3).
> A silly optimization: integrations starting from "timer:..." may be
> scheduled directly with Kubernetes CronJobs, so they will consume resources
> only when actually running.
>
> Being the final integrations lightweight and being the DSL
> language-independent, we may see a increased adoption of Camel also as
> agile integration layer for not-only-java applications (both "cloud" and
> "serverless" applications).
>
> I'm the first one that would like to work on a project ilke this. I've
> worked on many Kubernetes/Openshift based applications and frameworks in
> the past years, also on operators and CRDs, and I think this way of
> redesigning integrations has a lot of potential.
>
> Integrations will not be necessarily limited to the simplified DSL, but we
> can add extension points for scripting and even custom libraries (although
> limiting the freedom of the optimizer).
>
> The most important thing: it may become a great project, since it's driven
> by a great community.
>
> So, what do you think? Is it crazy enough?
>
> Nicola
>
--
Sascha Dirbach

Inhaber

endless webservices
Marco Paetschke & Sascha Dirbach GbR

Kirchweg 113
28201 Bremen

Mobil: +49 (0)160-94182103
Mail: [hidden email]
Web: www.endless-webservices.de

USt-IdNr.: DE310969215



signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

lburgazzoli
Love it so ++1

Btw, I did start some (early) experiments around alternative canel runtime
in my spare time:

- https://github.com/lburgazzoli/camel-go
- https://github.com/lburgazzoli/camel-go-examples/tree/master/example-yaml

Happy to make it part of the kamel initiative if you find it interesting.

On Fri, 13 Jul 2018 at 08:14, Sascha Dirbach <
[hidden email]> wrote:

> Hi Nicola,
>
> +1
>
> Great idea, do you already have a concept/idea/sketch on how to deal
> with complex configurations? i.e. SSL certs for outgoing calls, XSLT
> transformations, etc.
>
> When I think about it, you could probably use configmaps/secrets to
> mount these config in the container an then reference them in the route.
>
> Best regards,
>
> Sascha
>
> Am 13.07.2018 um 01:30 schrieb Nicola Ferraro:
> > Hi Cameleers,
> > it's now passed some time since I started thinking about a new project
> that
> > we can begin here at Apache Camel, and I'd like to have your opinion.
> >
> > We've already been targeting cloud-native applications with Camel,
> > especially on top of Kubernetes, that is becoming "the standard" cloud
> > platform. But writing a Camel integration and running it on Kubernetes
> > requires some effort: choosing the base platform (spring-boot, karaf,
> > simple main?), adding health checks (actuator?), packaging a docker image
> > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > publishing the image on a docker registry, then finally deploying the
> > resources on a Kubernetes cluster.
> >
> > The resulting integration container is then far from being optimal from a
> > resource consumption point of view: it is likely that a Camel Spring-Boot
> > application will require at least 200MB of RAM and also some CPU shares
> > because of polling threads used by many components.
> >
> > In case people use a CI/CD pipeline, it will take also a long time to get
> > from a code update to having a Kubernetes POD up and running.
> > Apart from compilation and image push/pull time, also startup time is
> often
> > ~10 seconds for Camel + Spring-Boot in a container with standard limits
> on
> > resources, making it difficult to propose this combination for
> "serverless
> > integration" (this term is becoming increasingly more popular).
> >
> > So, my proposal is to start to investigate a "more cloud-native" approach
> > to integration: *making Camel integrations first-class citizens in
> > Kubernetes, and making them super fast and lightweight.*
> >
> > We can base the project on Kubernetes Custom Resource Definitions (CRD)
> > <
> https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
> >,
> > for example a Integration CRD and have a Kubernetes "operator"
> > <https://coreos.com/operators/> taking care of:
> > - Optimizing the integration that we want to run
> > - Packaging in a container
> > - Running it on Kubernetes
> > - Managing its entire lifecycle
> >
> > A Kubernetes-native integration may look like:
> >
> > -------------------
> > kind: "Integration"
> > apiVersion: "camel.apache.org/v1alpha1"
> > metadata:
> >  name: "example"
> > spec:
> >  replicas: 1
> >  routes:
> >   - id: timer
> >     route:
> >      - type: endpoint
> >        uri: timer:tick
> >      - type: endpoint
> >        uri: log:info
> > -------------------
> >
> > For those who are not familiar with Kubernetes resources, this kind of
> > YAML/JSON resource definitions are really common.
> > The example route is embedded in the Kubernetes resource declaration and
> > follows a basic "flow DSL". We may start from a basic one and evolve it
> as
> > new requirements arrive from the community.
> >
> > I've made a very simple (but working) POC here:
> > https://github.com/nicolaferraro/integration-operator.
> >
> > This idea of a "Cloud-Native Camel" on Kubernetes (project codename can
> be "
> > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> features.
> >
> > For example, we can propose "Kamel" as "ideal" platform for "serverless
> > integration" (I see many people reinventing the wheel out there): the
> > operator can reduce resource consumption of a single integration by
> > optimizing the runtime and also pause/resume integrations when they are
> not
> > used, that is the basic idea behind "serverless" (e.g. think to
> > HTTP-triggered integrations, but not only).
> > Focusing on serverless will bring more emphasis on push-based
> notifications
> > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> used in
> > Camel components, that prefer a poll based approach being it simpler to
> use
> > in classic deployments, but not so good in the cloud, where more
> resources
> > become higher direct costs for the users.
> >
> > The presence of the simplified DSL enables also experimenting on
> "*reduced*
> > subsets of Camel" implemented in languages other than Java, for example
> one
> > language that has a reactive approach on thread scheduling and a really
> low
> > memory footprint, like Go.
> >
> > But apart from this kind of experiments (that are valid IMO), the "Kamel"
> > optimizer will have free room to choose the right platform for the
> > integration that the user wants to run, including, in the future, doing
> AOT
> > compilation using Graal/VM (less memory, faster startup) if the features
> > (components) used in the integration are supporting it (maybe we can add
> > AOT compilation in the roadmap for Camel 3).
> > A silly optimization: integrations starting from "timer:..." may be
> > scheduled directly with Kubernetes CronJobs, so they will consume
> resources
> > only when actually running.
> >
> > Being the final integrations lightweight and being the DSL
> > language-independent, we may see a increased adoption of Camel also as
> > agile integration layer for not-only-java applications (both "cloud" and
> > "serverless" applications).
> >
> > I'm the first one that would like to work on a project ilke this. I've
> > worked on many Kubernetes/Openshift based applications and frameworks in
> > the past years, also on operators and CRDs, and I think this way of
> > redesigning integrations has a lot of potential.
> >
> > Integrations will not be necessarily limited to the simplified DSL, but
> we
> > can add extension points for scripting and even custom libraries
> (although
> > limiting the freedom of the optimizer).
> >
> > The most important thing: it may become a great project, since it's
> driven
> > by a great community.
> >
> > So, what do you think? Is it crazy enough?
> >
> > Nicola
> >
>
> --
> Sascha Dirbach
>
> Inhaber
>
> endless webservices
> Marco Paetschke & Sascha Dirbach GbR
>
> Kirchweg 113
> 28201 Bremen
>
> Mobil: +49 (0)160-94182103
> Mail: [hidden email]
> Web: www.endless-webservices.de
>
> USt-IdNr.: DE310969215
>
>
> --
--
Luca Burgazzoli
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Willem.Jiang
Administrator
Hi Luca,

You are building Go version of Camel by implement the DSL in Go.
Now we can start the camel context by using camel-go to load the route,
but here are some missing points that I want fill such as  how to reuse the
Camel components?





Willem Jiang

Twitter: willemjiang
Weibo: 姜宁willem

On Fri, Jul 13, 2018 at 2:29 PM, Luca Burgazzoli <[hidden email]>
wrote:

> Love it so ++1
>
> Btw, I did start some (early) experiments around alternative canel runtime
> in my spare time:
>
> - https://github.com/lburgazzoli/camel-go
> - https://github.com/lburgazzoli/camel-go-examples/
> tree/master/example-yaml
>
> Happy to make it part of the kamel initiative if you find it interesting.
>
> On Fri, 13 Jul 2018 at 08:14, Sascha Dirbach <
> [hidden email]> wrote:
>
> > Hi Nicola,
> >
> > +1
> >
> > Great idea, do you already have a concept/idea/sketch on how to deal
> > with complex configurations? i.e. SSL certs for outgoing calls, XSLT
> > transformations, etc.
> >
> > When I think about it, you could probably use configmaps/secrets to
> > mount these config in the container an then reference them in the route.
> >
> > Best regards,
> >
> > Sascha
> >
> > Am 13.07.2018 um 01:30 schrieb Nicola Ferraro:
> > > Hi Cameleers,
> > > it's now passed some time since I started thinking about a new project
> > that
> > > we can begin here at Apache Camel, and I'd like to have your opinion.
> > >
> > > We've already been targeting cloud-native applications with Camel,
> > > especially on top of Kubernetes, that is becoming "the standard" cloud
> > > platform. But writing a Camel integration and running it on Kubernetes
> > > requires some effort: choosing the base platform (spring-boot, karaf,
> > > simple main?), adding health checks (actuator?), packaging a docker
> image
> > > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > > publishing the image on a docker registry, then finally deploying the
> > > resources on a Kubernetes cluster.
> > >
> > > The resulting integration container is then far from being optimal
> from a
> > > resource consumption point of view: it is likely that a Camel
> Spring-Boot
> > > application will require at least 200MB of RAM and also some CPU shares
> > > because of polling threads used by many components.
> > >
> > > In case people use a CI/CD pipeline, it will take also a long time to
> get
> > > from a code update to having a Kubernetes POD up and running.
> > > Apart from compilation and image push/pull time, also startup time is
> > often
> > > ~10 seconds for Camel + Spring-Boot in a container with standard limits
> > on
> > > resources, making it difficult to propose this combination for
> > "serverless
> > > integration" (this term is becoming increasingly more popular).
> > >
> > > So, my proposal is to start to investigate a "more cloud-native"
> approach
> > > to integration: *making Camel integrations first-class citizens in
> > > Kubernetes, and making them super fast and lightweight.*
> > >
> > > We can base the project on Kubernetes Custom Resource Definitions (CRD)
> > > <
> > https://kubernetes.io/docs/concepts/extend-kubernetes/
> api-extension/custom-resources/
> > >,
> > > for example a Integration CRD and have a Kubernetes "operator"
> > > <https://coreos.com/operators/> taking care of:
> > > - Optimizing the integration that we want to run
> > > - Packaging in a container
> > > - Running it on Kubernetes
> > > - Managing its entire lifecycle
> > >
> > > A Kubernetes-native integration may look like:
> > >
> > > -------------------
> > > kind: "Integration"
> > > apiVersion: "camel.apache.org/v1alpha1"
> > > metadata:
> > >  name: "example"
> > > spec:
> > >  replicas: 1
> > >  routes:
> > >   - id: timer
> > >     route:
> > >      - type: endpoint
> > >        uri: timer:tick
> > >      - type: endpoint
> > >        uri: log:info
> > > -------------------
> > >
> > > For those who are not familiar with Kubernetes resources, this kind of
> > > YAML/JSON resource definitions are really common.
> > > The example route is embedded in the Kubernetes resource declaration
> and
> > > follows a basic "flow DSL". We may start from a basic one and evolve it
> > as
> > > new requirements arrive from the community.
> > >
> > > I've made a very simple (but working) POC here:
> > > https://github.com/nicolaferraro/integration-operator.
> > >
> > > This idea of a "Cloud-Native Camel" on Kubernetes (project codename can
> > be "
> > > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> > features.
> > >
> > > For example, we can propose "Kamel" as "ideal" platform for "serverless
> > > integration" (I see many people reinventing the wheel out there): the
> > > operator can reduce resource consumption of a single integration by
> > > optimizing the runtime and also pause/resume integrations when they are
> > not
> > > used, that is the basic idea behind "serverless" (e.g. think to
> > > HTTP-triggered integrations, but not only).
> > > Focusing on serverless will bring more emphasis on push-based
> > notifications
> > > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> > used in
> > > Camel components, that prefer a poll based approach being it simpler to
> > use
> > > in classic deployments, but not so good in the cloud, where more
> > resources
> > > become higher direct costs for the users.
> > >
> > > The presence of the simplified DSL enables also experimenting on
> > "*reduced*
> > > subsets of Camel" implemented in languages other than Java, for example
> > one
> > > language that has a reactive approach on thread scheduling and a really
> > low
> > > memory footprint, like Go.
> > >
> > > But apart from this kind of experiments (that are valid IMO), the
> "Kamel"
> > > optimizer will have free room to choose the right platform for the
> > > integration that the user wants to run, including, in the future, doing
> > AOT
> > > compilation using Graal/VM (less memory, faster startup) if the
> features
> > > (components) used in the integration are supporting it (maybe we can
> add
> > > AOT compilation in the roadmap for Camel 3).
> > > A silly optimization: integrations starting from "timer:..." may be
> > > scheduled directly with Kubernetes CronJobs, so they will consume
> > resources
> > > only when actually running.
> > >
> > > Being the final integrations lightweight and being the DSL
> > > language-independent, we may see a increased adoption of Camel also as
> > > agile integration layer for not-only-java applications (both "cloud"
> and
> > > "serverless" applications).
> > >
> > > I'm the first one that would like to work on a project ilke this. I've
> > > worked on many Kubernetes/Openshift based applications and frameworks
> in
> > > the past years, also on operators and CRDs, and I think this way of
> > > redesigning integrations has a lot of potential.
> > >
> > > Integrations will not be necessarily limited to the simplified DSL, but
> > we
> > > can add extension points for scripting and even custom libraries
> > (although
> > > limiting the freedom of the optimizer).
> > >
> > > The most important thing: it may become a great project, since it's
> > driven
> > > by a great community.
> > >
> > > So, what do you think? Is it crazy enough?
> > >
> > > Nicola
> > >
> >
> > --
> > Sascha Dirbach
> >
> > Inhaber
> >
> > endless webservices
> > Marco Paetschke & Sascha Dirbach GbR
> >
> > Kirchweg 113
> > 28201 Bremen
> >
> > Mobil: +49 (0)160-94182103
> > Mail: [hidden email]
> > Web: www.endless-webservices.de
> >
> > USt-IdNr.: DE310969215
> >
> >
> > --
> --
> Luca Burgazzoli
>
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

lburgazzoli
Of course you need to port them to go lang, the project is not intended to
be a drop in replacement for camel but a companion for integration that do
not require all the power and flexibility the jvm implementation provide

On Fri, 13 Jul 2018 at 08:55, Willem Jiang <[hidden email]> wrote:

> Hi Luca,
>
> You are building Go version of Camel by implement the DSL in Go.
> Now we can start the camel context by using camel-go to load the route,
> but here are some missing points that I want fill such as  how to reuse the
> Camel components?
>
>
>
>
>
> Willem Jiang
>
> Twitter: willemjiang
> Weibo: 姜宁willem
>
> On Fri, Jul 13, 2018 at 2:29 PM, Luca Burgazzoli <[hidden email]>
> wrote:
>
> > Love it so ++1
> >
> > Btw, I did start some (early) experiments around alternative canel
> runtime
> > in my spare time:
> >
> > - https://github.com/lburgazzoli/camel-go
> > - https://github.com/lburgazzoli/camel-go-examples/
> > tree/master/example-yaml
> >
> > Happy to make it part of the kamel initiative if you find it interesting.
> >
> > On Fri, 13 Jul 2018 at 08:14, Sascha Dirbach <
> > [hidden email]> wrote:
> >
> > > Hi Nicola,
> > >
> > > +1
> > >
> > > Great idea, do you already have a concept/idea/sketch on how to deal
> > > with complex configurations? i.e. SSL certs for outgoing calls, XSLT
> > > transformations, etc.
> > >
> > > When I think about it, you could probably use configmaps/secrets to
> > > mount these config in the container an then reference them in the
> route.
> > >
> > > Best regards,
> > >
> > > Sascha
> > >
> > > Am 13.07.2018 um 01:30 schrieb Nicola Ferraro:
> > > > Hi Cameleers,
> > > > it's now passed some time since I started thinking about a new
> project
> > > that
> > > > we can begin here at Apache Camel, and I'd like to have your opinion.
> > > >
> > > > We've already been targeting cloud-native applications with Camel,
> > > > especially on top of Kubernetes, that is becoming "the standard"
> cloud
> > > > platform. But writing a Camel integration and running it on
> Kubernetes
> > > > requires some effort: choosing the base platform (spring-boot, karaf,
> > > > simple main?), adding health checks (actuator?), packaging a docker
> > image
> > > > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > > > publishing the image on a docker registry, then finally deploying the
> > > > resources on a Kubernetes cluster.
> > > >
> > > > The resulting integration container is then far from being optimal
> > from a
> > > > resource consumption point of view: it is likely that a Camel
> > Spring-Boot
> > > > application will require at least 200MB of RAM and also some CPU
> shares
> > > > because of polling threads used by many components.
> > > >
> > > > In case people use a CI/CD pipeline, it will take also a long time to
> > get
> > > > from a code update to having a Kubernetes POD up and running.
> > > > Apart from compilation and image push/pull time, also startup time is
> > > often
> > > > ~10 seconds for Camel + Spring-Boot in a container with standard
> limits
> > > on
> > > > resources, making it difficult to propose this combination for
> > > "serverless
> > > > integration" (this term is becoming increasingly more popular).
> > > >
> > > > So, my proposal is to start to investigate a "more cloud-native"
> > approach
> > > > to integration: *making Camel integrations first-class citizens in
> > > > Kubernetes, and making them super fast and lightweight.*
> > > >
> > > > We can base the project on Kubernetes Custom Resource Definitions
> (CRD)
> > > > <
> > > https://kubernetes.io/docs/concepts/extend-kubernetes/
> > api-extension/custom-resources/
> > > >,
> > > > for example a Integration CRD and have a Kubernetes "operator"
> > > > <https://coreos.com/operators/> taking care of:
> > > > - Optimizing the integration that we want to run
> > > > - Packaging in a container
> > > > - Running it on Kubernetes
> > > > - Managing its entire lifecycle
> > > >
> > > > A Kubernetes-native integration may look like:
> > > >
> > > > -------------------
> > > > kind: "Integration"
> > > > apiVersion: "camel.apache.org/v1alpha1"
> > > > metadata:
> > > >  name: "example"
> > > > spec:
> > > >  replicas: 1
> > > >  routes:
> > > >   - id: timer
> > > >     route:
> > > >      - type: endpoint
> > > >        uri: timer:tick
> > > >      - type: endpoint
> > > >        uri: log:info
> > > > -------------------
> > > >
> > > > For those who are not familiar with Kubernetes resources, this kind
> of
> > > > YAML/JSON resource definitions are really common.
> > > > The example route is embedded in the Kubernetes resource declaration
> > and
> > > > follows a basic "flow DSL". We may start from a basic one and evolve
> it
> > > as
> > > > new requirements arrive from the community.
> > > >
> > > > I've made a very simple (but working) POC here:
> > > > https://github.com/nicolaferraro/integration-operator.
> > > >
> > > > This idea of a "Cloud-Native Camel" on Kubernetes (project codename
> can
> > > be "
> > > > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> > > features.
> > > >
> > > > For example, we can propose "Kamel" as "ideal" platform for
> "serverless
> > > > integration" (I see many people reinventing the wheel out there): the
> > > > operator can reduce resource consumption of a single integration by
> > > > optimizing the runtime and also pause/resume integrations when they
> are
> > > not
> > > > used, that is the basic idea behind "serverless" (e.g. think to
> > > > HTTP-triggered integrations, but not only).
> > > > Focusing on serverless will bring more emphasis on push-based
> > > notifications
> > > > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> > > used in
> > > > Camel components, that prefer a poll based approach being it simpler
> to
> > > use
> > > > in classic deployments, but not so good in the cloud, where more
> > > resources
> > > > become higher direct costs for the users.
> > > >
> > > > The presence of the simplified DSL enables also experimenting on
> > > "*reduced*
> > > > subsets of Camel" implemented in languages other than Java, for
> example
> > > one
> > > > language that has a reactive approach on thread scheduling and a
> really
> > > low
> > > > memory footprint, like Go.
> > > >
> > > > But apart from this kind of experiments (that are valid IMO), the
> > "Kamel"
> > > > optimizer will have free room to choose the right platform for the
> > > > integration that the user wants to run, including, in the future,
> doing
> > > AOT
> > > > compilation using Graal/VM (less memory, faster startup) if the
> > features
> > > > (components) used in the integration are supporting it (maybe we can
> > add
> > > > AOT compilation in the roadmap for Camel 3).
> > > > A silly optimization: integrations starting from "timer:..." may be
> > > > scheduled directly with Kubernetes CronJobs, so they will consume
> > > resources
> > > > only when actually running.
> > > >
> > > > Being the final integrations lightweight and being the DSL
> > > > language-independent, we may see a increased adoption of Camel also
> as
> > > > agile integration layer for not-only-java applications (both "cloud"
> > and
> > > > "serverless" applications).
> > > >
> > > > I'm the first one that would like to work on a project ilke this.
> I've
> > > > worked on many Kubernetes/Openshift based applications and frameworks
> > in
> > > > the past years, also on operators and CRDs, and I think this way of
> > > > redesigning integrations has a lot of potential.
> > > >
> > > > Integrations will not be necessarily limited to the simplified DSL,
> but
> > > we
> > > > can add extension points for scripting and even custom libraries
> > > (although
> > > > limiting the freedom of the optimizer).
> > > >
> > > > The most important thing: it may become a great project, since it's
> > > driven
> > > > by a great community.
> > > >
> > > > So, what do you think? Is it crazy enough?
> > > >
> > > > Nicola
> > > >
> > >
> > > --
> > > Sascha Dirbach
> > >
> > > Inhaber
> > >
> > > endless webservices
> > > Marco Paetschke & Sascha Dirbach GbR
> > >
> > > Kirchweg 113
> > > 28201 Bremen
> > >
> > > Mobil: +49 (0)160-94182103
> > > Mail: [hidden email]
> > > Web: www.endless-webservices.de
> > >
> > > USt-IdNr.: DE310969215
> > >
> > >
> > > --
> > --
> > Luca Burgazzoli
> >
>
--
--
Luca Burgazzoli
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Zoran Regvart-2
In reply to this post by Nicola Ferraro
Hi Nicola,
+1 on the idea and the great write-up, seems to me like something that
we should definitely explore further.

I also like the "Kamel" name :)

zoran

On Fri, Jul 13, 2018 at 1:30 AM, Nicola Ferraro <[hidden email]> wrote:

> Hi Cameleers,
> it's now passed some time since I started thinking about a new project that
> we can begin here at Apache Camel, and I'd like to have your opinion.
>
> We've already been targeting cloud-native applications with Camel,
> especially on top of Kubernetes, that is becoming "the standard" cloud
> platform. But writing a Camel integration and running it on Kubernetes
> requires some effort: choosing the base platform (spring-boot, karaf,
> simple main?), adding health checks (actuator?), packaging a docker image
> and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> publishing the image on a docker registry, then finally deploying the
> resources on a Kubernetes cluster.
>
> The resulting integration container is then far from being optimal from a
> resource consumption point of view: it is likely that a Camel Spring-Boot
> application will require at least 200MB of RAM and also some CPU shares
> because of polling threads used by many components.
>
> In case people use a CI/CD pipeline, it will take also a long time to get
> from a code update to having a Kubernetes POD up and running.
> Apart from compilation and image push/pull time, also startup time is often
> ~10 seconds for Camel + Spring-Boot in a container with standard limits on
> resources, making it difficult to propose this combination for "serverless
> integration" (this term is becoming increasingly more popular).
>
> So, my proposal is to start to investigate a "more cloud-native" approach
> to integration: *making Camel integrations first-class citizens in
> Kubernetes, and making them super fast and lightweight.*
>
> We can base the project on Kubernetes Custom Resource Definitions (CRD)
> <https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>,
> for example a Integration CRD and have a Kubernetes "operator"
> <https://coreos.com/operators/> taking care of:
> - Optimizing the integration that we want to run
> - Packaging in a container
> - Running it on Kubernetes
> - Managing its entire lifecycle
>
> A Kubernetes-native integration may look like:
>
> -------------------
> kind: "Integration"
> apiVersion: "camel.apache.org/v1alpha1"
> metadata:
>  name: "example"
> spec:
>  replicas: 1
>  routes:
>   - id: timer
>     route:
>      - type: endpoint
>        uri: timer:tick
>      - type: endpoint
>        uri: log:info
> -------------------
>
> For those who are not familiar with Kubernetes resources, this kind of
> YAML/JSON resource definitions are really common.
> The example route is embedded in the Kubernetes resource declaration and
> follows a basic "flow DSL". We may start from a basic one and evolve it as
> new requirements arrive from the community.
>
> I've made a very simple (but working) POC here:
> https://github.com/nicolaferraro/integration-operator.
>
> This idea of a "Cloud-Native Camel" on Kubernetes (project codename can be "
> *Kamel*", if you like it :D), will be an enabler for a lot of nice features.
>
> For example, we can propose "Kamel" as "ideal" platform for "serverless
> integration" (I see many people reinventing the wheel out there): the
> operator can reduce resource consumption of a single integration by
> optimizing the runtime and also pause/resume integrations when they are not
> used, that is the basic idea behind "serverless" (e.g. think to
> HTTP-triggered integrations, but not only).
> Focusing on serverless will bring more emphasis on push-based notifications
> (webhooks, cloud events <https://cloudevents.io/>), that are rarely used in
> Camel components, that prefer a poll based approach being it simpler to use
> in classic deployments, but not so good in the cloud, where more resources
> become higher direct costs for the users.
>
> The presence of the simplified DSL enables also experimenting on "*reduced*
> subsets of Camel" implemented in languages other than Java, for example one
> language that has a reactive approach on thread scheduling and a really low
> memory footprint, like Go.
>
> But apart from this kind of experiments (that are valid IMO), the "Kamel"
> optimizer will have free room to choose the right platform for the
> integration that the user wants to run, including, in the future, doing AOT
> compilation using Graal/VM (less memory, faster startup) if the features
> (components) used in the integration are supporting it (maybe we can add
> AOT compilation in the roadmap for Camel 3).
> A silly optimization: integrations starting from "timer:..." may be
> scheduled directly with Kubernetes CronJobs, so they will consume resources
> only when actually running.
>
> Being the final integrations lightweight and being the DSL
> language-independent, we may see a increased adoption of Camel also as
> agile integration layer for not-only-java applications (both "cloud" and
> "serverless" applications).
>
> I'm the first one that would like to work on a project ilke this. I've
> worked on many Kubernetes/Openshift based applications and frameworks in
> the past years, also on operators and CRDs, and I think this way of
> redesigning integrations has a lot of potential.
>
> Integrations will not be necessarily limited to the simplified DSL, but we
> can add extension points for scripting and even custom libraries (although
> limiting the freedom of the optimizer).
>
> The most important thing: it may become a great project, since it's driven
> by a great community.
>
> So, what do you think? Is it crazy enough?
>
> Nicola



--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

astefanutti
In reply to this post by Nicola Ferraro
Hi Nicola,

I love the idea.

I just wonder whether YAML/JSON is an expressive enough format in the long term. But as you’ve mentioned, starting simple would enable experimenting some very interesting / promising optimisations. So it seems worth taking that path, instead of trying to embed a complex DSL or the existing XML DSL into the CRD.

Definitely +1

> On 13 Jul 2018, at 01:30, Nicola Ferraro <[hidden email]> wrote:
>
> Hi Cameleers,
> it's now passed some time since I started thinking about a new project that
> we can begin here at Apache Camel, and I'd like to have your opinion.
>
> We've already been targeting cloud-native applications with Camel,
> especially on top of Kubernetes, that is becoming "the standard" cloud
> platform. But writing a Camel integration and running it on Kubernetes
> requires some effort: choosing the base platform (spring-boot, karaf,
> simple main?), adding health checks (actuator?), packaging a docker image
> and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> publishing the image on a docker registry, then finally deploying the
> resources on a Kubernetes cluster.
>
> The resulting integration container is then far from being optimal from a
> resource consumption point of view: it is likely that a Camel Spring-Boot
> application will require at least 200MB of RAM and also some CPU shares
> because of polling threads used by many components.
>
> In case people use a CI/CD pipeline, it will take also a long time to get
> from a code update to having a Kubernetes POD up and running.
> Apart from compilation and image push/pull time, also startup time is often
> ~10 seconds for Camel + Spring-Boot in a container with standard limits on
> resources, making it difficult to propose this combination for "serverless
> integration" (this term is becoming increasingly more popular).
>
> So, my proposal is to start to investigate a "more cloud-native" approach
> to integration: *making Camel integrations first-class citizens in
> Kubernetes, and making them super fast and lightweight.*
>
> We can base the project on Kubernetes Custom Resource Definitions (CRD)
> <https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/>,
> for example a Integration CRD and have a Kubernetes "operator"
> <https://coreos.com/operators/> taking care of:
> - Optimizing the integration that we want to run
> - Packaging in a container
> - Running it on Kubernetes
> - Managing its entire lifecycle
>
> A Kubernetes-native integration may look like:
>
> -------------------
> kind: "Integration"
> apiVersion: "camel.apache.org/v1alpha1"
> metadata:
> name: "example"
> spec:
> replicas: 1
> routes:
>  - id: timer
>    route:
>     - type: endpoint
>       uri: timer:tick
>     - type: endpoint
>       uri: log:info
> -------------------
>
> For those who are not familiar with Kubernetes resources, this kind of
> YAML/JSON resource definitions are really common.
> The example route is embedded in the Kubernetes resource declaration and
> follows a basic "flow DSL". We may start from a basic one and evolve it as
> new requirements arrive from the community.
>
> I've made a very simple (but working) POC here:
> https://github.com/nicolaferraro/integration-operator.
>
> This idea of a "Cloud-Native Camel" on Kubernetes (project codename can be "
> *Kamel*", if you like it :D), will be an enabler for a lot of nice features.
>
> For example, we can propose "Kamel" as "ideal" platform for "serverless
> integration" (I see many people reinventing the wheel out there): the
> operator can reduce resource consumption of a single integration by
> optimizing the runtime and also pause/resume integrations when they are not
> used, that is the basic idea behind "serverless" (e.g. think to
> HTTP-triggered integrations, but not only).
> Focusing on serverless will bring more emphasis on push-based notifications
> (webhooks, cloud events <https://cloudevents.io/>), that are rarely used in
> Camel components, that prefer a poll based approach being it simpler to use
> in classic deployments, but not so good in the cloud, where more resources
> become higher direct costs for the users.
>
> The presence of the simplified DSL enables also experimenting on "*reduced*
> subsets of Camel" implemented in languages other than Java, for example one
> language that has a reactive approach on thread scheduling and a really low
> memory footprint, like Go.
>
> But apart from this kind of experiments (that are valid IMO), the "Kamel"
> optimizer will have free room to choose the right platform for the
> integration that the user wants to run, including, in the future, doing AOT
> compilation using Graal/VM (less memory, faster startup) if the features
> (components) used in the integration are supporting it (maybe we can add
> AOT compilation in the roadmap for Camel 3).
> A silly optimization: integrations starting from "timer:..." may be
> scheduled directly with Kubernetes CronJobs, so they will consume resources
> only when actually running.
>
> Being the final integrations lightweight and being the DSL
> language-independent, we may see a increased adoption of Camel also as
> agile integration layer for not-only-java applications (both "cloud" and
> "serverless" applications).
>
> I'm the first one that would like to work on a project ilke this. I've
> worked on many Kubernetes/Openshift based applications and frameworks in
> the past years, also on operators and CRDs, and I think this way of
> redesigning integrations has a lot of potential.
>
> Integrations will not be necessarily limited to the simplified DSL, but we
> can add extension points for scripting and even custom libraries (although
> limiting the freedom of the optimizer).
>
> The most important thing: it may become a great project, since it's driven
> by a great community.
>
> So, what do you think? Is it crazy enough?
>
> Nicola

Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Ricardo Zanini
Hi!

Awesome idea. Seeing this being viable in a platform like Apache OpenWhisk
would be great too. I keep hearing customers asking about lightweight
containers to run Camel context. :)

+++1



Zanini

On Fri, Jul 13, 2018 at 5:31 AM, Antonin Stefanutti <[hidden email]>
wrote:

> Hi Nicola,
>
> I love the idea.
>
> I just wonder whether YAML/JSON is an expressive enough format in the long
> term. But as you’ve mentioned, starting simple would enable experimenting
> some very interesting / promising optimisations. So it seems worth taking
> that path, instead of trying to embed a complex DSL or the existing XML DSL
> into the CRD.
>
> Definitely +1
>
> > On 13 Jul 2018, at 01:30, Nicola Ferraro <[hidden email]> wrote:
> >
> > Hi Cameleers,
> > it's now passed some time since I started thinking about a new project
> that
> > we can begin here at Apache Camel, and I'd like to have your opinion.
> >
> > We've already been targeting cloud-native applications with Camel,
> > especially on top of Kubernetes, that is becoming "the standard" cloud
> > platform. But writing a Camel integration and running it on Kubernetes
> > requires some effort: choosing the base platform (spring-boot, karaf,
> > simple main?), adding health checks (actuator?), packaging a docker image
> > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > publishing the image on a docker registry, then finally deploying the
> > resources on a Kubernetes cluster.
> >
> > The resulting integration container is then far from being optimal from a
> > resource consumption point of view: it is likely that a Camel Spring-Boot
> > application will require at least 200MB of RAM and also some CPU shares
> > because of polling threads used by many components.
> >
> > In case people use a CI/CD pipeline, it will take also a long time to get
> > from a code update to having a Kubernetes POD up and running.
> > Apart from compilation and image push/pull time, also startup time is
> often
> > ~10 seconds for Camel + Spring-Boot in a container with standard limits
> on
> > resources, making it difficult to propose this combination for
> "serverless
> > integration" (this term is becoming increasingly more popular).
> >
> > So, my proposal is to start to investigate a "more cloud-native" approach
> > to integration: *making Camel integrations first-class citizens in
> > Kubernetes, and making them super fast and lightweight.*
> >
> > We can base the project on Kubernetes Custom Resource Definitions (CRD)
> > <https://kubernetes.io/docs/concepts/extend-kubernetes/
> api-extension/custom-resources/>,
> > for example a Integration CRD and have a Kubernetes "operator"
> > <https://coreos.com/operators/> taking care of:
> > - Optimizing the integration that we want to run
> > - Packaging in a container
> > - Running it on Kubernetes
> > - Managing its entire lifecycle
> >
> > A Kubernetes-native integration may look like:
> >
> > -------------------
> > kind: "Integration"
> > apiVersion: "camel.apache.org/v1alpha1"
> > metadata:
> > name: "example"
> > spec:
> > replicas: 1
> > routes:
> >  - id: timer
> >    route:
> >     - type: endpoint
> >       uri: timer:tick
> >     - type: endpoint
> >       uri: log:info
> > -------------------
> >
> > For those who are not familiar with Kubernetes resources, this kind of
> > YAML/JSON resource definitions are really common.
> > The example route is embedded in the Kubernetes resource declaration and
> > follows a basic "flow DSL". We may start from a basic one and evolve it
> as
> > new requirements arrive from the community.
> >
> > I've made a very simple (but working) POC here:
> > https://github.com/nicolaferraro/integration-operator.
> >
> > This idea of a "Cloud-Native Camel" on Kubernetes (project codename can
> be "
> > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> features.
> >
> > For example, we can propose "Kamel" as "ideal" platform for "serverless
> > integration" (I see many people reinventing the wheel out there): the
> > operator can reduce resource consumption of a single integration by
> > optimizing the runtime and also pause/resume integrations when they are
> not
> > used, that is the basic idea behind "serverless" (e.g. think to
> > HTTP-triggered integrations, but not only).
> > Focusing on serverless will bring more emphasis on push-based
> notifications
> > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> used in
> > Camel components, that prefer a poll based approach being it simpler to
> use
> > in classic deployments, but not so good in the cloud, where more
> resources
> > become higher direct costs for the users.
> >
> > The presence of the simplified DSL enables also experimenting on
> "*reduced*
> > subsets of Camel" implemented in languages other than Java, for example
> one
> > language that has a reactive approach on thread scheduling and a really
> low
> > memory footprint, like Go.
> >
> > But apart from this kind of experiments (that are valid IMO), the "Kamel"
> > optimizer will have free room to choose the right platform for the
> > integration that the user wants to run, including, in the future, doing
> AOT
> > compilation using Graal/VM (less memory, faster startup) if the features
> > (components) used in the integration are supporting it (maybe we can add
> > AOT compilation in the roadmap for Camel 3).
> > A silly optimization: integrations starting from "timer:..." may be
> > scheduled directly with Kubernetes CronJobs, so they will consume
> resources
> > only when actually running.
> >
> > Being the final integrations lightweight and being the DSL
> > language-independent, we may see a increased adoption of Camel also as
> > agile integration layer for not-only-java applications (both "cloud" and
> > "serverless" applications).
> >
> > I'm the first one that would like to work on a project ilke this. I've
> > worked on many Kubernetes/Openshift based applications and frameworks in
> > the past years, also on operators and CRDs, and I think this way of
> > redesigning integrations has a lot of potential.
> >
> > Integrations will not be necessarily limited to the simplified DSL, but
> we
> > can add extension points for scripting and even custom libraries
> (although
> > limiting the freedom of the optimizer).
> >
> > The most important thing: it may become a great project, since it's
> driven
> > by a great community.
> >
> > So, what do you think? Is it crazy enough?
> >
> > Nicola
>
>
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Hiram Chirino
In reply to this post by astefanutti
Love the idea.

Personally, I'd keep using the existing Camel XML DSL if possible.  You can
still embed it in the CRD.  The operator that deploys it could inspect it
and figure whats the most optimized runtime that can support the DSL.
Perhaps if it's only using the restricted set of camel components supported
by camel-go (https://github.com/lburgazzoli/camel-go) then is uses that.
Otherwise it falls back to using camel spring boot.

For that to work I think we need the different runtime implementations to
provide a way to ask them: 'hey do you support running this camel route?'
Not a trivial thing to respond to, it might require a build step in there
for traditional Camel runtimes.


On Fri, Jul 13, 2018 at 4:32 AM Antonin Stefanutti <[hidden email]>
wrote:

> Hi Nicola,
>
> I love the idea.
>
> I just wonder whether YAML/JSON is an expressive enough format in the long
> term. But as you’ve mentioned, starting simple would enable experimenting
> some very interesting / promising optimisations. So it seems worth taking
> that path, instead of trying to embed a complex DSL or the existing XML DSL
> into the CRD.
>
> Definitely +1
>
> > On 13 Jul 2018, at 01:30, Nicola Ferraro <[hidden email]> wrote:
> >
> > Hi Cameleers,
> > it's now passed some time since I started thinking about a new project
> that
> > we can begin here at Apache Camel, and I'd like to have your opinion.
> >
> > We've already been targeting cloud-native applications with Camel,
> > especially on top of Kubernetes, that is becoming "the standard" cloud
> > platform. But writing a Camel integration and running it on Kubernetes
> > requires some effort: choosing the base platform (spring-boot, karaf,
> > simple main?), adding health checks (actuator?), packaging a docker image
> > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > publishing the image on a docker registry, then finally deploying the
> > resources on a Kubernetes cluster.
> >
> > The resulting integration container is then far from being optimal from a
> > resource consumption point of view: it is likely that a Camel Spring-Boot
> > application will require at least 200MB of RAM and also some CPU shares
> > because of polling threads used by many components.
> >
> > In case people use a CI/CD pipeline, it will take also a long time to get
> > from a code update to having a Kubernetes POD up and running.
> > Apart from compilation and image push/pull time, also startup time is
> often
> > ~10 seconds for Camel + Spring-Boot in a container with standard limits
> on
> > resources, making it difficult to propose this combination for
> "serverless
> > integration" (this term is becoming increasingly more popular).
> >
> > So, my proposal is to start to investigate a "more cloud-native" approach
> > to integration: *making Camel integrations first-class citizens in
> > Kubernetes, and making them super fast and lightweight.*
> >
> > We can base the project on Kubernetes Custom Resource Definitions (CRD)
> > <
> https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
> >,
> > for example a Integration CRD and have a Kubernetes "operator"
> > <https://coreos.com/operators/> taking care of:
> > - Optimizing the integration that we want to run
> > - Packaging in a container
> > - Running it on Kubernetes
> > - Managing its entire lifecycle
> >
> > A Kubernetes-native integration may look like:
> >
> > -------------------
> > kind: "Integration"
> > apiVersion: "camel.apache.org/v1alpha1"
> > metadata:
> > name: "example"
> > spec:
> > replicas: 1
> > routes:
> >  - id: timer
> >    route:
> >     - type: endpoint
> >       uri: timer:tick
> >     - type: endpoint
> >       uri: log:info
> > -------------------
> >
> > For those who are not familiar with Kubernetes resources, this kind of
> > YAML/JSON resource definitions are really common.
> > The example route is embedded in the Kubernetes resource declaration and
> > follows a basic "flow DSL". We may start from a basic one and evolve it
> as
> > new requirements arrive from the community.
> >
> > I've made a very simple (but working) POC here:
> > https://github.com/nicolaferraro/integration-operator.
> >
> > This idea of a "Cloud-Native Camel" on Kubernetes (project codename can
> be "
> > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> features.
> >
> > For example, we can propose "Kamel" as "ideal" platform for "serverless
> > integration" (I see many people reinventing the wheel out there): the
> > operator can reduce resource consumption of a single integration by
> > optimizing the runtime and also pause/resume integrations when they are
> not
> > used, that is the basic idea behind "serverless" (e.g. think to
> > HTTP-triggered integrations, but not only).
> > Focusing on serverless will bring more emphasis on push-based
> notifications
> > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> used in
> > Camel components, that prefer a poll based approach being it simpler to
> use
> > in classic deployments, but not so good in the cloud, where more
> resources
> > become higher direct costs for the users.
> >
> > The presence of the simplified DSL enables also experimenting on
> "*reduced*
> > subsets of Camel" implemented in languages other than Java, for example
> one
> > language that has a reactive approach on thread scheduling and a really
> low
> > memory footprint, like Go.
> >
> > But apart from this kind of experiments (that are valid IMO), the "Kamel"
> > optimizer will have free room to choose the right platform for the
> > integration that the user wants to run, including, in the future, doing
> AOT
> > compilation using Graal/VM (less memory, faster startup) if the features
> > (components) used in the integration are supporting it (maybe we can add
> > AOT compilation in the roadmap for Camel 3).
> > A silly optimization: integrations starting from "timer:..." may be
> > scheduled directly with Kubernetes CronJobs, so they will consume
> resources
> > only when actually running.
> >
> > Being the final integrations lightweight and being the DSL
> > language-independent, we may see a increased adoption of Camel also as
> > agile integration layer for not-only-java applications (both "cloud" and
> > "serverless" applications).
> >
> > I'm the first one that would like to work on a project ilke this. I've
> > worked on many Kubernetes/Openshift based applications and frameworks in
> > the past years, also on operators and CRDs, and I think this way of
> > redesigning integrations has a lot of potential.
> >
> > Integrations will not be necessarily limited to the simplified DSL, but
> we
> > can add extension points for scripting and even custom libraries
> (although
> > limiting the freedom of the optimizer).
> >
> > The most important thing: it may become a great project, since it's
> driven
> > by a great community.
> >
> > So, what do you think? Is it crazy enough?
> >
> > Nicola
>
>

--
Hiram Chirino
Engineering | Red Hat, Inc.
[hidden email] | fusesource.com | redhat.com
skype: hiramchirino | twitter: @hiramchirino
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

pontus.ullgren
I also like the idea but with some comments.

As Hiram Chirino I'm not sure YAML/JSON is the best language for this.
Perhaps a more fluent DSL like the Camel Java DSL or perhaps something like
Ballerina language would be better suited ?
Also in my experience even simple integrations, that is simple real world
integration and not just hello world demos, requires
you to add one or more Java class or scripting in addition to the core
components. So for it to be useful there must be some way to add custom
code in some way for aggregation/enrichment strategies.

If you go with a GO implementation I would also like some fallback for
using Java since I find it unlikely that the existing base of components
will be ported any time soon.

As for the name as other I like the "Kamel"  name however a being a newborn
kotlin fan I know of one pre-existing project with that name
that is close enough to allow for potential confusion.
https://github.com/topicusoverheid/kamel

Looking forward to see the result.

// Pontus






On Mon, 16 Jul 2018 at 15:15 Hiram Chirino <[hidden email]> wrote:

> Love the idea.
>
> Personally, I'd keep using the existing Camel XML DSL if possible.  You can
> still embed it in the CRD.  The operator that deploys it could inspect it
> and figure whats the most optimized runtime that can support the DSL.
> Perhaps if it's only using the restricted set of camel components supported
> by camel-go (https://github.com/lburgazzoli/camel-go) then is uses that.
> Otherwise it falls back to using camel spring boot.
>
> For that to work I think we need the different runtime implementations to
> provide a way to ask them: 'hey do you support running this camel route?'
> Not a trivial thing to respond to, it might require a build step in there
> for traditional Camel runtimes.
>
>
> On Fri, Jul 13, 2018 at 4:32 AM Antonin Stefanutti <[hidden email]>
> wrote:
>
> > Hi Nicola,
> >
> > I love the idea.
> >
> > I just wonder whether YAML/JSON is an expressive enough format in the
> long
> > term. But as you’ve mentioned, starting simple would enable experimenting
> > some very interesting / promising optimisations. So it seems worth taking
> > that path, instead of trying to embed a complex DSL or the existing XML
> DSL
> > into the CRD.
> >
> > Definitely +1
> >
> > > On 13 Jul 2018, at 01:30, Nicola Ferraro <[hidden email]> wrote:
> > >
> > > Hi Cameleers,
> > > it's now passed some time since I started thinking about a new project
> > that
> > > we can begin here at Apache Camel, and I'd like to have your opinion.
> > >
> > > We've already been targeting cloud-native applications with Camel,
> > > especially on top of Kubernetes, that is becoming "the standard" cloud
> > > platform. But writing a Camel integration and running it on Kubernetes
> > > requires some effort: choosing the base platform (spring-boot, karaf,
> > > simple main?), adding health checks (actuator?), packaging a docker
> image
> > > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > > publishing the image on a docker registry, then finally deploying the
> > > resources on a Kubernetes cluster.
> > >
> > > The resulting integration container is then far from being optimal
> from a
> > > resource consumption point of view: it is likely that a Camel
> Spring-Boot
> > > application will require at least 200MB of RAM and also some CPU shares
> > > because of polling threads used by many components.
> > >
> > > In case people use a CI/CD pipeline, it will take also a long time to
> get
> > > from a code update to having a Kubernetes POD up and running.
> > > Apart from compilation and image push/pull time, also startup time is
> > often
> > > ~10 seconds for Camel + Spring-Boot in a container with standard limits
> > on
> > > resources, making it difficult to propose this combination for
> > "serverless
> > > integration" (this term is becoming increasingly more popular).
> > >
> > > So, my proposal is to start to investigate a "more cloud-native"
> approach
> > > to integration: *making Camel integrations first-class citizens in
> > > Kubernetes, and making them super fast and lightweight.*
> > >
> > > We can base the project on Kubernetes Custom Resource Definitions (CRD)
> > > <
> >
> https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
> > >,
> > > for example a Integration CRD and have a Kubernetes "operator"
> > > <https://coreos.com/operators/> taking care of:
> > > - Optimizing the integration that we want to run
> > > - Packaging in a container
> > > - Running it on Kubernetes
> > > - Managing its entire lifecycle
> > >
> > > A Kubernetes-native integration may look like:
> > >
> > > -------------------
> > > kind: "Integration"
> > > apiVersion: "camel.apache.org/v1alpha1"
> > > metadata:
> > > name: "example"
> > > spec:
> > > replicas: 1
> > > routes:
> > >  - id: timer
> > >    route:
> > >     - type: endpoint
> > >       uri: timer:tick
> > >     - type: endpoint
> > >       uri: log:info
> > > -------------------
> > >
> > > For those who are not familiar with Kubernetes resources, this kind of
> > > YAML/JSON resource definitions are really common.
> > > The example route is embedded in the Kubernetes resource declaration
> and
> > > follows a basic "flow DSL". We may start from a basic one and evolve it
> > as
> > > new requirements arrive from the community.
> > >
> > > I've made a very simple (but working) POC here:
> > > https://github.com/nicolaferraro/integration-operator.
> > >
> > > This idea of a "Cloud-Native Camel" on Kubernetes (project codename can
> > be "
> > > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> > features.
> > >
> > > For example, we can propose "Kamel" as "ideal" platform for "serverless
> > > integration" (I see many people reinventing the wheel out there): the
> > > operator can reduce resource consumption of a single integration by
> > > optimizing the runtime and also pause/resume integrations when they are
> > not
> > > used, that is the basic idea behind "serverless" (e.g. think to
> > > HTTP-triggered integrations, but not only).
> > > Focusing on serverless will bring more emphasis on push-based
> > notifications
> > > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> > used in
> > > Camel components, that prefer a poll based approach being it simpler to
> > use
> > > in classic deployments, but not so good in the cloud, where more
> > resources
> > > become higher direct costs for the users.
> > >
> > > The presence of the simplified DSL enables also experimenting on
> > "*reduced*
> > > subsets of Camel" implemented in languages other than Java, for example
> > one
> > > language that has a reactive approach on thread scheduling and a really
> > low
> > > memory footprint, like Go.
> > >
> > > But apart from this kind of experiments (that are valid IMO), the
> "Kamel"
> > > optimizer will have free room to choose the right platform for the
> > > integration that the user wants to run, including, in the future, doing
> > AOT
> > > compilation using Graal/VM (less memory, faster startup) if the
> features
> > > (components) used in the integration are supporting it (maybe we can
> add
> > > AOT compilation in the roadmap for Camel 3).
> > > A silly optimization: integrations starting from "timer:..." may be
> > > scheduled directly with Kubernetes CronJobs, so they will consume
> > resources
> > > only when actually running.
> > >
> > > Being the final integrations lightweight and being the DSL
> > > language-independent, we may see a increased adoption of Camel also as
> > > agile integration layer for not-only-java applications (both "cloud"
> and
> > > "serverless" applications).
> > >
> > > I'm the first one that would like to work on a project ilke this. I've
> > > worked on many Kubernetes/Openshift based applications and frameworks
> in
> > > the past years, also on operators and CRDs, and I think this way of
> > > redesigning integrations has a lot of potential.
> > >
> > > Integrations will not be necessarily limited to the simplified DSL, but
> > we
> > > can add extension points for scripting and even custom libraries
> > (although
> > > limiting the freedom of the optimizer).
> > >
> > > The most important thing: it may become a great project, since it's
> > driven
> > > by a great community.
> > >
> > > So, what do you think? Is it crazy enough?
> > >
> > > Nicola
> >
> >
>
> --
> Hiram Chirino
> Engineering | Red Hat, Inc.
> [hidden email] | fusesource.com | redhat.com
> skype: hiramchirino | twitter: @hiramchirino
>
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Nicola Ferraro
It's clear to me that we need to add support for our existing XML DSL, that
is powerful. But there are multiple reasons why I'd like to also "add" a
limited yaml/json notation to "Kamel".

The first one (and simplest) is that json/yaml is the primary encoding for
all resources exchanged with the Kubernetes API server. I'm not saying that
for this reason the Integration resource cannot contain a XML
(/Java/Ballerina) section, but from a pure presentational point of view,
having the possibility to write simple use cases (even a "hello world") in
pure json/yaml is better than always requiring a mix of json and XML.

The second one is simplicity. Writing a optimizer (the module that
materializes the integration into running code, choosing a specific runtime
and configuration) for a turing-complete language like Java is not easy:
even if you manage to create a good parser, it may be able to do
optimizations only if you write routes in a particular way, without complex
constructs.
XML is ok from this point of view. The thing is that both json/yaml and XML
are just two different ways to serialize object trees, that can be then
statically analyzed.
The point is not XML vs json/yaml, it's more about tailoring a new
minimalistic DSL to the emerging use cases vs proposing "only" our classic
way of writing integrations. I think XML can be the "advanced" way. We can
experiment optimizations easily with the new DSL, and enable them also on
XML if it's worth.

Scripting should be part of the spec, but I'd try to use programming
languages only for processing/transformation, not for the route definition.

Third one is performance. Apart from the fact that json parsers are in
general said to be faster than XML parsers... Given the "fast startup"
target that we want to reach, we may think e.g. to translate the new DSL
into Java or Go code, then compiling it. This allows doing parsing at build
time in order to avoid it on startup. This kind of improvements are much
easier with a limited DSL but much more difficult with a existing
fully-fledged DSL..

Nicola



On Wed, Jul 18, 2018 at 11:51 PM Pontus Ullgren <[hidden email]> wrote:

> I also like the idea but with some comments.
>
> As Hiram Chirino I'm not sure YAML/JSON is the best language for this.
> Perhaps a more fluent DSL like the Camel Java DSL or perhaps something like
> Ballerina language would be better suited ?
> Also in my experience even simple integrations, that is simple real world
> integration and not just hello world demos, requires
> you to add one or more Java class or scripting in addition to the core
> components. So for it to be useful there must be some way to add custom
> code in some way for aggregation/enrichment strategies.
>
> If you go with a GO implementation I would also like some fallback for
> using Java since I find it unlikely that the existing base of components
> will be ported any time soon.
>
> As for the name as other I like the "Kamel"  name however a being a newborn
> kotlin fan I know of one pre-existing project with that name
> that is close enough to allow for potential confusion.
> https://github.com/topicusoverheid/kamel
>
> Looking forward to see the result.
>
> // Pontus
>
>
>
>
>
>
> On Mon, 16 Jul 2018 at 15:15 Hiram Chirino <[hidden email]> wrote:
>
> > Love the idea.
> >
> > Personally, I'd keep using the existing Camel XML DSL if possible.  You
> can
> > still embed it in the CRD.  The operator that deploys it could inspect it
> > and figure whats the most optimized runtime that can support the DSL.
> > Perhaps if it's only using the restricted set of camel components
> supported
> > by camel-go (https://github.com/lburgazzoli/camel-go) then is uses that.
> > Otherwise it falls back to using camel spring boot.
> >
> > For that to work I think we need the different runtime implementations to
> > provide a way to ask them: 'hey do you support running this camel route?'
> > Not a trivial thing to respond to, it might require a build step in there
> > for traditional Camel runtimes.
> >
> >
> > On Fri, Jul 13, 2018 at 4:32 AM Antonin Stefanutti <
> [hidden email]>
> > wrote:
> >
> > > Hi Nicola,
> > >
> > > I love the idea.
> > >
> > > I just wonder whether YAML/JSON is an expressive enough format in the
> > long
> > > term. But as you’ve mentioned, starting simple would enable
> experimenting
> > > some very interesting / promising optimisations. So it seems worth
> taking
> > > that path, instead of trying to embed a complex DSL or the existing XML
> > DSL
> > > into the CRD.
> > >
> > > Definitely +1
> > >
> > > > On 13 Jul 2018, at 01:30, Nicola Ferraro <[hidden email]>
> wrote:
> > > >
> > > > Hi Cameleers,
> > > > it's now passed some time since I started thinking about a new
> project
> > > that
> > > > we can begin here at Apache Camel, and I'd like to have your opinion.
> > > >
> > > > We've already been targeting cloud-native applications with Camel,
> > > > especially on top of Kubernetes, that is becoming "the standard"
> cloud
> > > > platform. But writing a Camel integration and running it on
> Kubernetes
> > > > requires some effort: choosing the base platform (spring-boot, karaf,
> > > > simple main?), adding health checks (actuator?), packaging a docker
> > image
> > > > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > > > publishing the image on a docker registry, then finally deploying the
> > > > resources on a Kubernetes cluster.
> > > >
> > > > The resulting integration container is then far from being optimal
> > from a
> > > > resource consumption point of view: it is likely that a Camel
> > Spring-Boot
> > > > application will require at least 200MB of RAM and also some CPU
> shares
> > > > because of polling threads used by many components.
> > > >
> > > > In case people use a CI/CD pipeline, it will take also a long time to
> > get
> > > > from a code update to having a Kubernetes POD up and running.
> > > > Apart from compilation and image push/pull time, also startup time is
> > > often
> > > > ~10 seconds for Camel + Spring-Boot in a container with standard
> limits
> > > on
> > > > resources, making it difficult to propose this combination for
> > > "serverless
> > > > integration" (this term is becoming increasingly more popular).
> > > >
> > > > So, my proposal is to start to investigate a "more cloud-native"
> > approach
> > > > to integration: *making Camel integrations first-class citizens in
> > > > Kubernetes, and making them super fast and lightweight.*
> > > >
> > > > We can base the project on Kubernetes Custom Resource Definitions
> (CRD)
> > > > <
> > >
> >
> https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
> > > >,
> > > > for example a Integration CRD and have a Kubernetes "operator"
> > > > <https://coreos.com/operators/> taking care of:
> > > > - Optimizing the integration that we want to run
> > > > - Packaging in a container
> > > > - Running it on Kubernetes
> > > > - Managing its entire lifecycle
> > > >
> > > > A Kubernetes-native integration may look like:
> > > >
> > > > -------------------
> > > > kind: "Integration"
> > > > apiVersion: "camel.apache.org/v1alpha1"
> > > > metadata:
> > > > name: "example"
> > > > spec:
> > > > replicas: 1
> > > > routes:
> > > >  - id: timer
> > > >    route:
> > > >     - type: endpoint
> > > >       uri: timer:tick
> > > >     - type: endpoint
> > > >       uri: log:info
> > > > -------------------
> > > >
> > > > For those who are not familiar with Kubernetes resources, this kind
> of
> > > > YAML/JSON resource definitions are really common.
> > > > The example route is embedded in the Kubernetes resource declaration
> > and
> > > > follows a basic "flow DSL". We may start from a basic one and evolve
> it
> > > as
> > > > new requirements arrive from the community.
> > > >
> > > > I've made a very simple (but working) POC here:
> > > > https://github.com/nicolaferraro/integration-operator.
> > > >
> > > > This idea of a "Cloud-Native Camel" on Kubernetes (project codename
> can
> > > be "
> > > > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> > > features.
> > > >
> > > > For example, we can propose "Kamel" as "ideal" platform for
> "serverless
> > > > integration" (I see many people reinventing the wheel out there): the
> > > > operator can reduce resource consumption of a single integration by
> > > > optimizing the runtime and also pause/resume integrations when they
> are
> > > not
> > > > used, that is the basic idea behind "serverless" (e.g. think to
> > > > HTTP-triggered integrations, but not only).
> > > > Focusing on serverless will bring more emphasis on push-based
> > > notifications
> > > > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> > > used in
> > > > Camel components, that prefer a poll based approach being it simpler
> to
> > > use
> > > > in classic deployments, but not so good in the cloud, where more
> > > resources
> > > > become higher direct costs for the users.
> > > >
> > > > The presence of the simplified DSL enables also experimenting on
> > > "*reduced*
> > > > subsets of Camel" implemented in languages other than Java, for
> example
> > > one
> > > > language that has a reactive approach on thread scheduling and a
> really
> > > low
> > > > memory footprint, like Go.
> > > >
> > > > But apart from this kind of experiments (that are valid IMO), the
> > "Kamel"
> > > > optimizer will have free room to choose the right platform for the
> > > > integration that the user wants to run, including, in the future,
> doing
> > > AOT
> > > > compilation using Graal/VM (less memory, faster startup) if the
> > features
> > > > (components) used in the integration are supporting it (maybe we can
> > add
> > > > AOT compilation in the roadmap for Camel 3).
> > > > A silly optimization: integrations starting from "timer:..." may be
> > > > scheduled directly with Kubernetes CronJobs, so they will consume
> > > resources
> > > > only when actually running.
> > > >
> > > > Being the final integrations lightweight and being the DSL
> > > > language-independent, we may see a increased adoption of Camel also
> as
> > > > agile integration layer for not-only-java applications (both "cloud"
> > and
> > > > "serverless" applications).
> > > >
> > > > I'm the first one that would like to work on a project ilke this.
> I've
> > > > worked on many Kubernetes/Openshift based applications and frameworks
> > in
> > > > the past years, also on operators and CRDs, and I think this way of
> > > > redesigning integrations has a lot of potential.
> > > >
> > > > Integrations will not be necessarily limited to the simplified DSL,
> but
> > > we
> > > > can add extension points for scripting and even custom libraries
> > > (although
> > > > limiting the freedom of the optimizer).
> > > >
> > > > The most important thing: it may become a great project, since it's
> > > driven
> > > > by a great community.
> > > >
> > > > So, what do you think? Is it crazy enough?
> > > >
> > > > Nicola
> > >
> > >
> >
> > --
> > Hiram Chirino
> > Engineering | Red Hat, Inc.
> > [hidden email] | fusesource.com | redhat.com
> > skype: hiramchirino | twitter: @hiramchirino
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Nicola Ferraro
Hi Cameleers,
it seems from the comments that this "Kamel" subproject is something we
want to start and I think that also the main camel core will benefit from
the new features it will bring.

I would like to donate the current "Kamel" code to Apache Camel, in order
to have a initial brick to start from.

From your reactions, the name "Kamel" seems cool and I'd like to keep it.
However if you talk to people about the "Kamel" project, they won't
understand if you're talking about "Kamel with the K" or "Camel with the C".
For this reasons I propose to keep "Kamel", but also use "Camel K" [ˈkæməl
keɪ] as friendly name when talking about it. This is in line with the
repository we should create for the subproject, that needs to start with
"camel-" (it will be "apache/camel-k" in that case).

These days I've been experimenting different ideas with Luca Burgazzoli. In
particular, there have been some concerns here on the power of a
declarative DSL (a new one or the existing XML one) and we've found a
strategy that will allow us to use also the Java DSL in Kamel integrations.
Without adding too many details here, at integration build time we can run
the user code in a build container and inspect the produced routes to get
metadata for the optimizer. This is one of the first things we should do
next.

This possibility of running code that produces integrations opened a lot of
unexpected paths, that I've written down in the project roadmap (
https://github.com/nicolaferraro/integration-operator) and summarized below.

One nice feature (kudos to Luca) is that we can simplify life for Kamel
users up to the point that they'll just need to write their routes on a
Java file and run them with e.g.:
"kamel run Routes.java"

Where "kamel" is a binary we release within the project. Kubernetes custom
resources will be used under the hood, but the "kamel" binary is a utility
that will provide a user experience comparable to that of current
serverless platforms (or even better).

But there's not just that.

Luca wrote this weekend a prototype for having a polyglot Camel (
https://github.com/lburgazzoli/camel-routes-loader).

With that, you will be able to e.g. write integrations in groovy and use
the same Kamel engine:
"kamel run routes.groovy"

But also JavaScript:
https://github.com/lburgazzoli/camel-routes-loader/blob/ca986541f7c422ee02c21727cdfe4293d64a364e/src/test/resources/ext/camel/routes.js#L2

This is a prototype right now, but a similar approach based on GraalVM has
a lot of potential, because users can not only use their preferred language
to write routes and processors, but also bind them to functionalities
available in their preferred libraries.

How much this will be feasible depends on the adoption of GraalVM, but I'm
seeing many frameworks adding metadata to make GraalVM work with
reflection. We've also done some work in Camel, there are some Jiras for it
and first tests made by Guillaume signal that it's something feasible, at
least for the Camel core and a subset of components. GraalVM would be
important also to reduce memory footprint and improve startup time, as
already said.

If the GraalVM approach works (it is working for other frameworks), instead
of rewriting a subset of Camel in Go (as the original proposal mentions),
we can just sanitize and recompile our existing codebase: this way we fully
leverage the strength of Apache Camel.

Another important thing in the roadmap is actually how we'll integrate with
current FaaS platforms. We will evaluate together if it's better to have a
tighter integration with some of them or to leverage knative for some
aspects of Kamel...

But, let's get started!

Il 19 lug 2018 10:45, "Nicola Ferraro" <[hidden email]> ha scritto:

It's clear to me that we need to add support for our existing XML DSL, that
is powerful. But there are multiple reasons why I'd like to also "add" a
limited yaml/json notation to "Kamel".

The first one (and simplest) is that json/yaml is the primary encoding for
all resources exchanged with the Kubernetes API server. I'm not saying that
for this reason the Integration resource cannot contain a XML
(/Java/Ballerina) section, but from a pure presentational point of view,
having the possibility to write simple use cases (even a "hello world") in
pure json/yaml is better than always requiring a mix of json and XML.

The second one is simplicity. Writing a optimizer (the module that
materializes the integration into running code, choosing a specific runtime
and configuration) for a turing-complete language like Java is not easy:
even if you manage to create a good parser, it may be able to do
optimizations only if you write routes in a particular way, without complex
constructs.
XML is ok from this point of view. The thing is that both json/yaml and XML
are just two different ways to serialize object trees, that can be then
statically analyzed.
The point is not XML vs json/yaml, it's more about tailoring a new
minimalistic DSL to the emerging use cases vs proposing "only" our classic
way of writing integrations. I think XML can be the "advanced" way. We can
experiment optimizations easily with the new DSL, and enable them also on
XML if it's worth.

Scripting should be part of the spec, but I'd try to use programming
languages only for processing/transformation, not for the route definition.

Third one is performance. Apart from the fact that json parsers are in
general said to be faster than XML parsers... Given the "fast startup"
target that we want to reach, we may think e.g. to translate the new DSL
into Java or Go code, then compiling it. This allows doing parsing at build
time in order to avoid it on startup. This kind of improvements are much
easier with a limited DSL but much more difficult with a existing
fully-fledged DSL..

Nicola



On Wed, Jul 18, 2018 at 11:51 PM Pontus Ullgren <[hidden email]> wrote:

> I also like the idea but with some comments.
>
> As Hiram Chirino I'm not sure YAML/JSON is the best language for this.
> Perhaps a more fluent DSL like the Camel Java DSL or perhaps something like
> Ballerina language would be better suited ?
> Also in my experience even simple integrations, that is simple real world
> integration and not just hello world demos, requires
> you to add one or more Java class or scripting in addition to the core
> components. So for it to be useful there must be some way to add custom
> code in some way for aggregation/enrichment strategies.
>
> If you go with a GO implementation I would also like some fallback for
> using Java since I find it unlikely that the existing base of components
> will be ported any time soon.
>
> As for the name as other I like the "Kamel"  name however a being a newborn
> kotlin fan I know of one pre-existing project with that name
> that is close enough to allow for potential confusion.
> https://github.com/topicusoverheid/kamel
>
> Looking forward to see the result.
>
> // Pontus
>
>
>
>
>
>
> On Mon, 16 Jul 2018 at 15:15 Hiram Chirino <[hidden email]> wrote:
>
> > Love the idea.
> >
> > Personally, I'd keep using the existing Camel XML DSL if possible.  You
> can
> > still embed it in the CRD.  The operator that deploys it could inspect it
> > and figure whats the most optimized runtime that can support the DSL.
> > Perhaps if it's only using the restricted set of camel components
> supported
> > by camel-go (https://github.com/lburgazzoli/camel-go) then is uses that.
> > Otherwise it falls back to using camel spring boot.
> >
> > For that to work I think we need the different runtime implementations to
> > provide a way to ask them: 'hey do you support running this camel route?'
> > Not a trivial thing to respond to, it might require a build step in there
> > for traditional Camel runtimes.
> >
> >
> > On Fri, Jul 13, 2018 at 4:32 AM Antonin Stefanutti <
> [hidden email]>
> > wrote:
> >
> > > Hi Nicola,
> > >
> > > I love the idea.
> > >
> > > I just wonder whether YAML/JSON is an expressive enough format in the
> > long
> > > term. But as you’ve mentioned, starting simple would enable
> experimenting
> > > some very interesting / promising optimisations. So it seems worth
> taking
> > > that path, instead of trying to embed a complex DSL or the existing XML
> > DSL
> > > into the CRD.
> > >
> > > Definitely +1
> > >
> > > > On 13 Jul 2018, at 01:30, Nicola Ferraro <[hidden email]>
> wrote:
> > > >
> > > > Hi Cameleers,
> > > > it's now passed some time since I started thinking about a new
> project
> > > that
> > > > we can begin here at Apache Camel, and I'd like to have your opinion.
> > > >
> > > > We've already been targeting cloud-native applications with Camel,
> > > > especially on top of Kubernetes, that is becoming "the standard"
> cloud
> > > > platform. But writing a Camel integration and running it on
> Kubernetes
> > > > requires some effort: choosing the base platform (spring-boot, karaf,
> > > > simple main?), adding health checks (actuator?), packaging a docker
> > image
> > > > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > > > publishing the image on a docker registry, then finally deploying the
> > > > resources on a Kubernetes cluster.
> > > >
> > > > The resulting integration container is then far from being optimal
> > from a
> > > > resource consumption point of view: it is likely that a Camel
> > Spring-Boot
> > > > application will require at least 200MB of RAM and also some CPU
> shares
> > > > because of polling threads used by many components.
> > > >
> > > > In case people use a CI/CD pipeline, it will take also a long time to
> > get
> > > > from a code update to having a Kubernetes POD up and running.
> > > > Apart from compilation and image push/pull time, also startup time is
> > > often
> > > > ~10 seconds for Camel + Spring-Boot in a container with standard
> limits
> > > on
> > > > resources, making it difficult to propose this combination for
> > > "serverless
> > > > integration" (this term is becoming increasingly more popular).
> > > >
> > > > So, my proposal is to start to investigate a "more cloud-native"
> > approach
> > > > to integration: *making Camel integrations first-class citizens in
> > > > Kubernetes, and making them super fast and lightweight.*
> > > >
> > > > We can base the project on Kubernetes Custom Resource Definitions
> (CRD)
> > > > <
> > >
> >
> https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
> > > >,
> > > > for example a Integration CRD and have a Kubernetes "operator"
> > > > <https://coreos.com/operators/> taking care of:
> > > > - Optimizing the integration that we want to run
> > > > - Packaging in a container
> > > > - Running it on Kubernetes
> > > > - Managing its entire lifecycle
> > > >
> > > > A Kubernetes-native integration may look like:
> > > >
> > > > -------------------
> > > > kind: "Integration"
> > > > apiVersion: "camel.apache.org/v1alpha1"
> > > > metadata:
> > > > name: "example"
> > > > spec:
> > > > replicas: 1
> > > > routes:
> > > >  - id: timer
> > > >    route:
> > > >     - type: endpoint
> > > >       uri: timer:tick
> > > >     - type: endpoint
> > > >       uri: log:info
> > > > -------------------
> > > >
> > > > For those who are not familiar with Kubernetes resources, this kind
> of
> > > > YAML/JSON resource definitions are really common.
> > > > The example route is embedded in the Kubernetes resource declaration
> > and
> > > > follows a basic "flow DSL". We may start from a basic one and evolve
> it
> > > as
> > > > new requirements arrive from the community.
> > > >
> > > > I've made a very simple (but working) POC here:
> > > > https://github.com/nicolaferraro/integration-operator.
> > > >
> > > > This idea of a "Cloud-Native Camel" on Kubernetes (project codename
> can
> > > be "
> > > > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> > > features.
> > > >
> > > > For example, we can propose "Kamel" as "ideal" platform for
> "serverless
> > > > integration" (I see many people reinventing the wheel out there): the
> > > > operator can reduce resource consumption of a single integration by
> > > > optimizing the runtime and also pause/resume integrations when they
> are
> > > not
> > > > used, that is the basic idea behind "serverless" (e.g. think to
> > > > HTTP-triggered integrations, but not only).
> > > > Focusing on serverless will bring more emphasis on push-based
> > > notifications
> > > > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> > > used in
> > > > Camel components, that prefer a poll based approach being it simpler
> to
> > > use
> > > > in classic deployments, but not so good in the cloud, where more
> > > resources
> > > > become higher direct costs for the users.
> > > >
> > > > The presence of the simplified DSL enables also experimenting on
> > > "*reduced*
> > > > subsets of Camel" implemented in languages other than Java, for
> example
> > > one
> > > > language that has a reactive approach on thread scheduling and a
> really
> > > low
> > > > memory footprint, like Go.
> > > >
> > > > But apart from this kind of experiments (that are valid IMO), the
> > "Kamel"
> > > > optimizer will have free room to choose the right platform for the
> > > > integration that the user wants to run, including, in the future,
> doing
> > > AOT
> > > > compilation using Graal/VM (less memory, faster startup) if the
> > features
> > > > (components) used in the integration are supporting it (maybe we can
> > add
> > > > AOT compilation in the roadmap for Camel 3).
> > > > A silly optimization: integrations starting from "timer:..." may be
> > > > scheduled directly with Kubernetes CronJobs, so they will consume
> > > resources
> > > > only when actually running.
> > > >
> > > > Being the final integrations lightweight and being the DSL
> > > > language-independent, we may see a increased adoption of Camel also
> as
> > > > agile integration layer for not-only-java applications (both "cloud"
> > and
> > > > "serverless" applications).
> > > >
> > > > I'm the first one that would like to work on a project ilke this.
> I've
> > > > worked on many Kubernetes/Openshift based applications and frameworks
> > in
> > > > the past years, also on operators and CRDs, and I think this way of
> > > > redesigning integrations has a lot of potential.
> > > >
> > > > Integrations will not be necessarily limited to the simplified DSL,
> but
> > > we
> > > > can add extension points for scripting and even custom libraries
> > > (although
> > > > limiting the freedom of the optimizer).
> > > >
> > > > The most important thing: it may become a great project, since it's
> > > driven
> > > > by a great community.
> > > >
> > > > So, what do you think? Is it crazy enough?
> > > >
> > > > Nicola
> > >
> > >
> >
> > --
> > Hiram Chirino
> > Engineering | Red Hat, Inc.
> > [hidden email] | fusesource.com | redhat.com
> > skype: hiramchirino | twitter: @hiramchirino
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

gnodet
Le lun. 30 juil. 2018 à 17:49, Nicola Ferraro <[hidden email]> a
écrit :

> Hi Cameleers,
> it seems from the comments that this "Kamel" subproject is something we
> want to start and I think that also the main camel core will benefit from
> the new features it will bring.
>
> I would like to donate the current "Kamel" code to Apache Camel, in order
> to have a initial brick to start from.
>
> From your reactions, the name "Kamel" seems cool and I'd like to keep it.
> However if you talk to people about the "Kamel" project, they won't
> understand if you're talking about "Kamel with the K" or "Camel with the
> C".
> For this reasons I propose to keep "Kamel", but also use "Camel K" [ˈkæməl
> keɪ] as friendly name when talking about it. This is in line with the
> repository we should create for the subproject, that needs to start with
> "camel-" (it will be "apache/camel-k" in that case).
>
> These days I've been experimenting different ideas with Luca Burgazzoli. In
> particular, there have been some concerns here on the power of a
> declarative DSL (a new one or the existing XML one) and we've found a
> strategy that will allow us to use also the Java DSL in Kamel integrations.
> Without adding too many details here, at integration build time we can run
> the user code in a build container and inspect the produced routes to get
> metadata for the optimizer. This is one of the first things we should do
> next.
>
> This possibility of running code that produces integrations opened a lot of
> unexpected paths, that I've written down in the project roadmap (
> https://github.com/nicolaferraro/integration-operator) and summarized
> below.
>
> One nice feature (kudos to Luca) is that we can simplify life for Kamel
> users up to the point that they'll just need to write their routes on a
> Java file and run them with e.g.:
> "kamel run Routes.java"
>
> Where "kamel" is a binary we release within the project. Kubernetes custom
> resources will be used under the hood, but the "kamel" binary is a utility
> that will provide a user experience comparable to that of current
> serverless platforms (or even better).
>
> But there's not just that.
>
> Luca wrote this weekend a prototype for having a polyglot Camel (
> https://github.com/lburgazzoli/camel-routes-loader).
>
> With that, you will be able to e.g. write integrations in groovy and use
> the same Kamel engine:
> "kamel run routes.groovy"
>
> But also JavaScript:
>
> https://github.com/lburgazzoli/camel-routes-loader/blob/ca986541f7c422ee02c21727cdfe4293d64a364e/src/test/resources/ext/camel/routes.js#L2
>
> This is a prototype right now, but a similar approach based on GraalVM has
> a lot of potential, because users can not only use their preferred language
> to write routes and processors, but also bind them to functionalities
> available in their preferred libraries.
>
> How much this will be feasible depends on the adoption of GraalVM, but I'm
> seeing many frameworks adding metadata to make GraalVM work with
> reflection. We've also done some work in Camel, there are some Jiras for it
> and first tests made by Guillaume signal that it's something feasible, at
> least for the Camel core and a subset of components. GraalVM would be
> important also to reduce memory footprint and improve startup time, as
> already said.
>

I'll continue experimenting and I'll report back.
I did some initial experiments leveraging some modifications I did  to
speed up the start up time [1] and on a single route [2].
There may be some limitations down the road of course, but at least it
shows that it's feasible. Fwiw, the experiments lead to a startup time of
14 ms compared to a few hundreds milliseconds (roughly 680 with [1]).


>
> If the GraalVM approach works (it is working for other frameworks), instead
> of rewriting a subset of Camel in Go (as the original proposal mentions),
> we can just sanitize and recompile our existing codebase: this way we fully
> leverage the strength of Apache Camel.
>

Right, if we can avoid rewriting Camel, that would be much better !

[1] https://issues.apache.org/jira/browse/CAMEL-12688
[2]
https://github.com/gnodet/openwhisk-runtime-camel/blob/master/camel-openwhisk-example/src/main/java/org/jboss/fuse/openwhisk/camel/example/SimpleCamelFunction.java


>
> Another important thing in the roadmap is actually how we'll integrate with
> current FaaS platforms. We will evaluate together if it's better to have a
> tighter integration with some of them or to leverage knative for some
> aspects of Kamel...
>
> But, let's get started!
>
> Il 19 lug 2018 10:45, "Nicola Ferraro" <[hidden email]> ha scritto:
>
> It's clear to me that we need to add support for our existing XML DSL, that
> is powerful. But there are multiple reasons why I'd like to also "add" a
> limited yaml/json notation to "Kamel".
>
> The first one (and simplest) is that json/yaml is the primary encoding for
> all resources exchanged with the Kubernetes API server. I'm not saying that
> for this reason the Integration resource cannot contain a XML
> (/Java/Ballerina) section, but from a pure presentational point of view,
> having the possibility to write simple use cases (even a "hello world") in
> pure json/yaml is better than always requiring a mix of json and XML.
>
> The second one is simplicity. Writing a optimizer (the module that
> materializes the integration into running code, choosing a specific runtime
> and configuration) for a turing-complete language like Java is not easy:
> even if you manage to create a good parser, it may be able to do
> optimizations only if you write routes in a particular way, without complex
> constructs.
> XML is ok from this point of view. The thing is that both json/yaml and XML
> are just two different ways to serialize object trees, that can be then
> statically analyzed.
> The point is not XML vs json/yaml, it's more about tailoring a new
> minimalistic DSL to the emerging use cases vs proposing "only" our classic
> way of writing integrations. I think XML can be the "advanced" way. We can
> experiment optimizations easily with the new DSL, and enable them also on
> XML if it's worth.
>
> Scripting should be part of the spec, but I'd try to use programming
> languages only for processing/transformation, not for the route definition.
>
> Third one is performance. Apart from the fact that json parsers are in
> general said to be faster than XML parsers... Given the "fast startup"
> target that we want to reach, we may think e.g. to translate the new DSL
> into Java or Go code, then compiling it. This allows doing parsing at build
> time in order to avoid it on startup. This kind of improvements are much
> easier with a limited DSL but much more difficult with a existing
> fully-fledged DSL..
>
> Nicola
>
>
>
> On Wed, Jul 18, 2018 at 11:51 PM Pontus Ullgren <[hidden email]> wrote:
>
> > I also like the idea but with some comments.
> >
> > As Hiram Chirino I'm not sure YAML/JSON is the best language for this.
> > Perhaps a more fluent DSL like the Camel Java DSL or perhaps something
> like
> > Ballerina language would be better suited ?
> > Also in my experience even simple integrations, that is simple real world
> > integration and not just hello world demos, requires
> > you to add one or more Java class or scripting in addition to the core
> > components. So for it to be useful there must be some way to add custom
> > code in some way for aggregation/enrichment strategies.
> >
> > If you go with a GO implementation I would also like some fallback for
> > using Java since I find it unlikely that the existing base of components
> > will be ported any time soon.
> >
> > As for the name as other I like the "Kamel"  name however a being a
> newborn
> > kotlin fan I know of one pre-existing project with that name
> > that is close enough to allow for potential confusion.
> > https://github.com/topicusoverheid/kamel
> >
> > Looking forward to see the result.
> >
> > // Pontus
> >
> >
> >
> >
> >
> >
> > On Mon, 16 Jul 2018 at 15:15 Hiram Chirino <[hidden email]>
> wrote:
> >
> > > Love the idea.
> > >
> > > Personally, I'd keep using the existing Camel XML DSL if possible.  You
> > can
> > > still embed it in the CRD.  The operator that deploys it could inspect
> it
> > > and figure whats the most optimized runtime that can support the DSL.
> > > Perhaps if it's only using the restricted set of camel components
> > supported
> > > by camel-go (https://github.com/lburgazzoli/camel-go) then is uses
> that.
> > > Otherwise it falls back to using camel spring boot.
> > >
> > > For that to work I think we need the different runtime implementations
> to
> > > provide a way to ask them: 'hey do you support running this camel
> route?'
> > > Not a trivial thing to respond to, it might require a build step in
> there
> > > for traditional Camel runtimes.
> > >
> > >
> > > On Fri, Jul 13, 2018 at 4:32 AM Antonin Stefanutti <
> > [hidden email]>
> > > wrote:
> > >
> > > > Hi Nicola,
> > > >
> > > > I love the idea.
> > > >
> > > > I just wonder whether YAML/JSON is an expressive enough format in the
> > > long
> > > > term. But as you’ve mentioned, starting simple would enable
> > experimenting
> > > > some very interesting / promising optimisations. So it seems worth
> > taking
> > > > that path, instead of trying to embed a complex DSL or the existing
> XML
> > > DSL
> > > > into the CRD.
> > > >
> > > > Definitely +1
> > > >
> > > > > On 13 Jul 2018, at 01:30, Nicola Ferraro <[hidden email]>
> > wrote:
> > > > >
> > > > > Hi Cameleers,
> > > > > it's now passed some time since I started thinking about a new
> > project
> > > > that
> > > > > we can begin here at Apache Camel, and I'd like to have your
> opinion.
> > > > >
> > > > > We've already been targeting cloud-native applications with Camel,
> > > > > especially on top of Kubernetes, that is becoming "the standard"
> > cloud
> > > > > platform. But writing a Camel integration and running it on
> > Kubernetes
> > > > > requires some effort: choosing the base platform (spring-boot,
> karaf,
> > > > > simple main?), adding health checks (actuator?), packaging a docker
> > > image
> > > > > and creating the Kubernetes resources (fabric8-maven-plugin,
> helm?),
> > > > > publishing the image on a docker registry, then finally deploying
> the
> > > > > resources on a Kubernetes cluster.
> > > > >
> > > > > The resulting integration container is then far from being optimal
> > > from a
> > > > > resource consumption point of view: it is likely that a Camel
> > > Spring-Boot
> > > > > application will require at least 200MB of RAM and also some CPU
> > shares
> > > > > because of polling threads used by many components.
> > > > >
> > > > > In case people use a CI/CD pipeline, it will take also a long time
> to
> > > get
> > > > > from a code update to having a Kubernetes POD up and running.
> > > > > Apart from compilation and image push/pull time, also startup time
> is
> > > > often
> > > > > ~10 seconds for Camel + Spring-Boot in a container with standard
> > limits
> > > > on
> > > > > resources, making it difficult to propose this combination for
> > > > "serverless
> > > > > integration" (this term is becoming increasingly more popular).
> > > > >
> > > > > So, my proposal is to start to investigate a "more cloud-native"
> > > approach
> > > > > to integration: *making Camel integrations first-class citizens in
> > > > > Kubernetes, and making them super fast and lightweight.*
> > > > >
> > > > > We can base the project on Kubernetes Custom Resource Definitions
> > (CRD)
> > > > > <
> > > >
> > >
> >
> https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
> > > > >,
> > > > > for example a Integration CRD and have a Kubernetes "operator"
> > > > > <https://coreos.com/operators/> taking care of:
> > > > > - Optimizing the integration that we want to run
> > > > > - Packaging in a container
> > > > > - Running it on Kubernetes
> > > > > - Managing its entire lifecycle
> > > > >
> > > > > A Kubernetes-native integration may look like:
> > > > >
> > > > > -------------------
> > > > > kind: "Integration"
> > > > > apiVersion: "camel.apache.org/v1alpha1"
> > > > > metadata:
> > > > > name: "example"
> > > > > spec:
> > > > > replicas: 1
> > > > > routes:
> > > > >  - id: timer
> > > > >    route:
> > > > >     - type: endpoint
> > > > >       uri: timer:tick
> > > > >     - type: endpoint
> > > > >       uri: log:info
> > > > > -------------------
> > > > >
> > > > > For those who are not familiar with Kubernetes resources, this kind
> > of
> > > > > YAML/JSON resource definitions are really common.
> > > > > The example route is embedded in the Kubernetes resource
> declaration
> > > and
> > > > > follows a basic "flow DSL". We may start from a basic one and
> evolve
> > it
> > > > as
> > > > > new requirements arrive from the community.
> > > > >
> > > > > I've made a very simple (but working) POC here:
> > > > > https://github.com/nicolaferraro/integration-operator.
> > > > >
> > > > > This idea of a "Cloud-Native Camel" on Kubernetes (project codename
> > can
> > > > be "
> > > > > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> > > > features.
> > > > >
> > > > > For example, we can propose "Kamel" as "ideal" platform for
> > "serverless
> > > > > integration" (I see many people reinventing the wheel out there):
> the
> > > > > operator can reduce resource consumption of a single integration by
> > > > > optimizing the runtime and also pause/resume integrations when they
> > are
> > > > not
> > > > > used, that is the basic idea behind "serverless" (e.g. think to
> > > > > HTTP-triggered integrations, but not only).
> > > > > Focusing on serverless will bring more emphasis on push-based
> > > > notifications
> > > > > (webhooks, cloud events <https://cloudevents.io/>), that are
> rarely
> > > > used in
> > > > > Camel components, that prefer a poll based approach being it
> simpler
> > to
> > > > use
> > > > > in classic deployments, but not so good in the cloud, where more
> > > > resources
> > > > > become higher direct costs for the users.
> > > > >
> > > > > The presence of the simplified DSL enables also experimenting on
> > > > "*reduced*
> > > > > subsets of Camel" implemented in languages other than Java, for
> > example
> > > > one
> > > > > language that has a reactive approach on thread scheduling and a
> > really
> > > > low
> > > > > memory footprint, like Go.
> > > > >
> > > > > But apart from this kind of experiments (that are valid IMO), the
> > > "Kamel"
> > > > > optimizer will have free room to choose the right platform for the
> > > > > integration that the user wants to run, including, in the future,
> > doing
> > > > AOT
> > > > > compilation using Graal/VM (less memory, faster startup) if the
> > > features
> > > > > (components) used in the integration are supporting it (maybe we
> can
> > > add
> > > > > AOT compilation in the roadmap for Camel 3).
> > > > > A silly optimization: integrations starting from "timer:..." may be
> > > > > scheduled directly with Kubernetes CronJobs, so they will consume
> > > > resources
> > > > > only when actually running.
> > > > >
> > > > > Being the final integrations lightweight and being the DSL
> > > > > language-independent, we may see a increased adoption of Camel also
> > as
> > > > > agile integration layer for not-only-java applications (both
> "cloud"
> > > and
> > > > > "serverless" applications).
> > > > >
> > > > > I'm the first one that would like to work on a project ilke this.
> > I've
> > > > > worked on many Kubernetes/Openshift based applications and
> frameworks
> > > in
> > > > > the past years, also on operators and CRDs, and I think this way of
> > > > > redesigning integrations has a lot of potential.
> > > > >
> > > > > Integrations will not be necessarily limited to the simplified DSL,
> > but
> > > > we
> > > > > can add extension points for scripting and even custom libraries
> > > > (although
> > > > > limiting the freedom of the optimizer).
> > > > >
> > > > > The most important thing: it may become a great project, since it's
> > > > driven
> > > > > by a great community.
> > > > >
> > > > > So, what do you think? Is it crazy enough?
> > > > >
> > > > > Nicola
> > > >
> > > >
> > >
> > > --
> > > Hiram Chirino
> > > Engineering | Red Hat, Inc.
> > > [hidden email] | fusesource.com | redhat.com
> > > skype: hiramchirino | twitter: @hiramchirino
> > >
> >
>


--
------------------------
Guillaume Nodet
Reply | Threaded
Open this post in threaded view
|

Re: A "Kamel" crazy idea

Andrea Cosentino-2
In reply to this post by Nicola Ferraro
This is a wonderful idea and I believe we can improve our community with a subproject like this.

So +1 for me.

--
Andrea Cosentino 
----------------------------------
Apache Camel PMC Chair
Apache Karaf Committer
Apache Servicemix PMC Member
Email: [hidden email]
Twitter: @oscerd2
Github: oscerd






On Monday, July 30, 2018, 5:49:50 PM GMT+2, Nicola Ferraro <[hidden email]> wrote:





Hi Cameleers,
it seems from the comments that this "Kamel" subproject is something we
want to start and I think that also the main camel core will benefit from
the new features it will bring.

I would like to donate the current "Kamel" code to Apache Camel, in order
to have a initial brick to start from.

From your reactions, the name "Kamel" seems cool and I'd like to keep it.
However if you talk to people about the "Kamel" project, they won't
understand if you're talking about "Kamel with the K" or "Camel with the C".
For this reasons I propose to keep "Kamel", but also use "Camel K" [ˈkæməl
keɪ] as friendly name when talking about it. This is in line with the
repository we should create for the subproject, that needs to start with
"camel-" (it will be "apache/camel-k" in that case).

These days I've been experimenting different ideas with Luca Burgazzoli. In
particular, there have been some concerns here on the power of a
declarative DSL (a new one or the existing XML one) and we've found a
strategy that will allow us to use also the Java DSL in Kamel integrations.
Without adding too many details here, at integration build time we can run
the user code in a build container and inspect the produced routes to get
metadata for the optimizer. This is one of the first things we should do
next.

This possibility of running code that produces integrations opened a lot of
unexpected paths, that I've written down in the project roadmap (
https://github.com/nicolaferraro/integration-operator) and summarized below.

One nice feature (kudos to Luca) is that we can simplify life for Kamel
users up to the point that they'll just need to write their routes on a
Java file and run them with e.g.:
"kamel run Routes.java"

Where "kamel" is a binary we release within the project. Kubernetes custom
resources will be used under the hood, but the "kamel" binary is a utility
that will provide a user experience comparable to that of current
serverless platforms (or even better).

But there's not just that.

Luca wrote this weekend a prototype for having a polyglot Camel (
https://github.com/lburgazzoli/camel-routes-loader).

With that, you will be able to e.g. write integrations in groovy and use
the same Kamel engine:
"kamel run routes.groovy"

But also JavaScript:
https://github.com/lburgazzoli/camel-routes-loader/blob/ca986541f7c422ee02c21727cdfe4293d64a364e/src/test/resources/ext/camel/routes.js#L2

This is a prototype right now, but a similar approach based on GraalVM has
a lot of potential, because users can not only use their preferred language
to write routes and processors, but also bind them to functionalities
available in their preferred libraries.

How much this will be feasible depends on the adoption of GraalVM, but I'm
seeing many frameworks adding metadata to make GraalVM work with
reflection. We've also done some work in Camel, there are some Jiras for it
and first tests made by Guillaume signal that it's something feasible, at
least for the Camel core and a subset of components. GraalVM would be
important also to reduce memory footprint and improve startup time, as
already said.

If the GraalVM approach works (it is working for other frameworks), instead
of rewriting a subset of Camel in Go (as the original proposal mentions),
we can just sanitize and recompile our existing codebase: this way we fully
leverage the strength of Apache Camel.

Another important thing in the roadmap is actually how we'll integrate with
current FaaS platforms. We will evaluate together if it's better to have a
tighter integration with some of them or to leverage knative for some
aspects of Kamel...

But, let's get started!

Il 19 lug 2018 10:45, "Nicola Ferraro" <[hidden email]> ha scritto:

It's clear to me that we need to add support for our existing XML DSL, that
is powerful. But there are multiple reasons why I'd like to also "add" a
limited yaml/json notation to "Kamel".

The first one (and simplest) is that json/yaml is the primary encoding for
all resources exchanged with the Kubernetes API server. I'm not saying that
for this reason the Integration resource cannot contain a XML
(/Java/Ballerina) section, but from a pure presentational point of view,
having the possibility to write simple use cases (even a "hello world") in
pure json/yaml is better than always requiring a mix of json and XML.

The second one is simplicity. Writing a optimizer (the module that
materializes the integration into running code, choosing a specific runtime
and configuration) for a turing-complete language like Java is not easy:
even if you manage to create a good parser, it may be able to do
optimizations only if you write routes in a particular way, without complex
constructs.
XML is ok from this point of view. The thing is that both json/yaml and XML
are just two different ways to serialize object trees, that can be then
statically analyzed.
The point is not XML vs json/yaml, it's more about tailoring a new
minimalistic DSL to the emerging use cases vs proposing "only" our classic
way of writing integrations. I think XML can be the "advanced" way. We can
experiment optimizations easily with the new DSL, and enable them also on
XML if it's worth.

Scripting should be part of the spec, but I'd try to use programming
languages only for processing/transformation, not for the route definition.

Third one is performance. Apart from the fact that json parsers are in
general said to be faster than XML parsers... Given the "fast startup"
target that we want to reach, we may think e.g. to translate the new DSL
into Java or Go code, then compiling it. This allows doing parsing at build
time in order to avoid it on startup. This kind of improvements are much
easier with a limited DSL but much more difficult with a existing
fully-fledged DSL..

Nicola



On Wed, Jul 18, 2018 at 11:51 PM Pontus Ullgren <[hidden email]> wrote:

> I also like the idea but with some comments.
>
> As Hiram Chirino I'm not sure YAML/JSON is the best language for this.
> Perhaps a more fluent DSL like the Camel Java DSL or perhaps something like
> Ballerina language would be better suited ?
> Also in my experience even simple integrations, that is simple real world
> integration and not just hello world demos, requires
> you to add one or more Java class or scripting in addition to the core
> components. So for it to be useful there must be some way to add custom
> code in some way for aggregation/enrichment strategies.
>
> If you go with a GO implementation I would also like some fallback for
> using Java since I find it unlikely that the existing base of components
> will be ported any time soon.
>
> As for the name as other I like the "Kamel"  name however a being a newborn
> kotlin fan I know of one pre-existing project with that name
> that is close enough to allow for potential confusion.
> https://github.com/topicusoverheid/kamel
>
> Looking forward to see the result.
>
> // Pontus
>
>
>
>
>
>
> On Mon, 16 Jul 2018 at 15:15 Hiram Chirino <[hidden email]> wrote:
>
> > Love the idea.
> >
> > Personally, I'd keep using the existing Camel XML DSL if possible.  You
> can
> > still embed it in the CRD.  The operator that deploys it could inspect it
> > and figure whats the most optimized runtime that can support the DSL.
> > Perhaps if it's only using the restricted set of camel components
> supported
> > by camel-go (https://github.com/lburgazzoli/camel-go) then is uses that.
> > Otherwise it falls back to using camel spring boot.
> >
> > For that to work I think we need the different runtime implementations to
> > provide a way to ask them: 'hey do you support running this camel route?'
> > Not a trivial thing to respond to, it might require a build step in there
> > for traditional Camel runtimes.
> >
> >
> > On Fri, Jul 13, 2018 at 4:32 AM Antonin Stefanutti <
> [hidden email]>
> > wrote:
> >
> > > Hi Nicola,
> > >
> > > I love the idea.
> > >
> > > I just wonder whether YAML/JSON is an expressive enough format in the
> > long
> > > term. But as you’ve mentioned, starting simple would enable
> experimenting
> > > some very interesting / promising optimisations. So it seems worth
> taking
> > > that path, instead of trying to embed a complex DSL or the existing XML
> > DSL
> > > into the CRD.
> > >
> > > Definitely +1
> > >
> > > > On 13 Jul 2018, at 01:30, Nicola Ferraro <[hidden email]>
> wrote:
> > > >
> > > > Hi Cameleers,
> > > > it's now passed some time since I started thinking about a new
> project
> > > that
> > > > we can begin here at Apache Camel, and I'd like to have your opinion.
> > > >
> > > > We've already been targeting cloud-native applications with Camel,
> > > > especially on top of Kubernetes, that is becoming "the standard"
> cloud
> > > > platform. But writing a Camel integration and running it on
> Kubernetes
> > > > requires some effort: choosing the base platform (spring-boot, karaf,
> > > > simple main?), adding health checks (actuator?), packaging a docker
> > image
> > > > and creating the Kubernetes resources (fabric8-maven-plugin, helm?),
> > > > publishing the image on a docker registry, then finally deploying the
> > > > resources on a Kubernetes cluster.
> > > >
> > > > The resulting integration container is then far from being optimal
> > from a
> > > > resource consumption point of view: it is likely that a Camel
> > Spring-Boot
> > > > application will require at least 200MB of RAM and also some CPU
> shares
> > > > because of polling threads used by many components.
> > > >
> > > > In case people use a CI/CD pipeline, it will take also a long time to
> > get
> > > > from a code update to having a Kubernetes POD up and running.
> > > > Apart from compilation and image push/pull time, also startup time is
> > > often
> > > > ~10 seconds for Camel + Spring-Boot in a container with standard
> limits
> > > on
> > > > resources, making it difficult to propose this combination for
> > > "serverless
> > > > integration" (this term is becoming increasingly more popular).
> > > >
> > > > So, my proposal is to start to investigate a "more cloud-native"
> > approach
> > > > to integration: *making Camel integrations first-class citizens in
> > > > Kubernetes, and making them super fast and lightweight.*
> > > >
> > > > We can base the project on Kubernetes Custom Resource Definitions
> (CRD)
> > > > <
> > >
> >
> https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
> > > >,
> > > > for example a Integration CRD and have a Kubernetes "operator"
> > > > <https://coreos.com/operators/> taking care of:
> > > > - Optimizing the integration that we want to run
> > > > - Packaging in a container
> > > > - Running it on Kubernetes
> > > > - Managing its entire lifecycle
> > > >
> > > > A Kubernetes-native integration may look like:
> > > >
> > > > -------------------
> > > > kind: "Integration"
> > > > apiVersion: "camel.apache.org/v1alpha1"
> > > > metadata:
> > > > name: "example"
> > > > spec:
> > > > replicas: 1
> > > > routes:
> > > >  - id: timer
> > > >    route:
> > > >    - type: endpoint
> > > >      uri: timer:tick
> > > >    - type: endpoint
> > > >      uri: log:info
> > > > -------------------
> > > >
> > > > For those who are not familiar with Kubernetes resources, this kind
> of
> > > > YAML/JSON resource definitions are really common.
> > > > The example route is embedded in the Kubernetes resource declaration
> > and
> > > > follows a basic "flow DSL". We may start from a basic one and evolve
> it
> > > as
> > > > new requirements arrive from the community.
> > > >
> > > > I've made a very simple (but working) POC here:
> > > > https://github.com/nicolaferraro/integration-operator.
> > > >
> > > > This idea of a "Cloud-Native Camel" on Kubernetes (project codename
> can
> > > be "
> > > > *Kamel*", if you like it :D), will be an enabler for a lot of nice
> > > features.
> > > >
> > > > For example, we can propose "Kamel" as "ideal" platform for
> "serverless
> > > > integration" (I see many people reinventing the wheel out there): the
> > > > operator can reduce resource consumption of a single integration by
> > > > optimizing the runtime and also pause/resume integrations when they
> are
> > > not
> > > > used, that is the basic idea behind "serverless" (e.g. think to
> > > > HTTP-triggered integrations, but not only).
> > > > Focusing on serverless will bring more emphasis on push-based
> > > notifications
> > > > (webhooks, cloud events <https://cloudevents.io/>), that are rarely
> > > used in
> > > > Camel components, that prefer a poll based approach being it simpler
> to
> > > use
> > > > in classic deployments, but not so good in the cloud, where more
> > > resources
> > > > become higher direct costs for the users.
> > > >
> > > > The presence of the simplified DSL enables also experimenting on
> > > "*reduced*
> > > > subsets of Camel" implemented in languages other than Java, for
> example
> > > one
> > > > language that has a reactive approach on thread scheduling and a
> really
> > > low
> > > > memory footprint, like Go.
> > > >
> > > > But apart from this kind of experiments (that are valid IMO), the
> > "Kamel"
> > > > optimizer will have free room to choose the right platform for the
> > > > integration that the user wants to run, including, in the future,
> doing
> > > AOT
> > > > compilation using Graal/VM (less memory, faster startup) if the
> > features
> > > > (components) used in the integration are supporting it (maybe we can
> > add
> > > > AOT compilation in the roadmap for Camel 3).
> > > > A silly optimization: integrations starting from "timer:..." may be
> > > > scheduled directly with Kubernetes CronJobs, so they will consume
> > > resources
> > > > only when actually running.
> > > >
> > > > Being the final integrations lightweight and being the DSL
> > > > language-independent, we may see a increased adoption of Camel also
> as
> > > > agile integration layer for not-only-java applications (both "cloud"
> > and
> > > > "serverless" applications).
> > > >
> > > > I'm the first one that would like to work on a project ilke this.
> I've
> > > > worked on many Kubernetes/Openshift based applications and frameworks
> > in
> > > > the past years, also on operators and CRDs, and I think this way of
> > > > redesigning integrations has a lot of potential.
> > > >
> > > > Integrations will not be necessarily limited to the simplified DSL,
> but
> > > we
> > > > can add extension points for scripting and even custom libraries
> > > (although
> > > > limiting the freedom of the optimizer).
> > > >
> > > > The most important thing: it may become a great project, since it's
> > > driven
> > > > by a great community.
> > > >
> > > > So, what do you think? Is it crazy enough?
> > > >
> > > > Nicola
> > >
> > >
> >
> > --
> > Hiram Chirino
> > Engineering | Red Hat, Inc.
> > [hidden email] | fusesource.com | redhat.com
> > skype: hiramchirino | twitter: @hiramchirino
> >
>
12