Some questions on CamelK

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Some questions on CamelK

skin27
Hi all,

In the last presentation there was also a short demo on CamelK. That looked
very interesting, but I have some questions on how does it work, especially
in more complex use cases.

Here are some questions I have:

Question 1: Dependency resolution

In CamelK you can provide the following:

from("imap://[hidden email]")
  .to("seda:output")

Because imap is not part of core this will start the dependency resolution.
How does this work? How does CamelK know it needs the mail component as
dependency? Is there some kind of fixed mapping between the scheme and
component?

Say for example I have the following route:

from("seda:input")
  .to("seda:output")

This is then run on Kubernetes/CamelK. Then I change this to:

from("imap://[hidden email]")
  .to("seda:output")

Is this dependency add as "Maven" dependency and then recompiled? Or is the
jar downloaded and the added? Or something else...


Question 2: Multiple files

In the Java example:
https://camel.apache.org/camel-k/latest/languages/java.html

There is one simple route with one file. How does it work in more complex
cases when there are multiple files?

For example I have the following package:

Registry.java
Route1.java
Route2.java
Processor.java
Connection.java

In Route1.java data is retrieved with the SQL component out of an Oracle
database, also this routs calls a processor. In route 2 the pojo is
marshalled to JSON and send to a HTTP endpoint. In registry things like
SSL, metrics are configured. In the processor some mapping is done and in
the connection the Oracle driver is configured.

Can I run a whole package with the Kamel CLI? How does Kamel gets for
example the dependency for the Oracle driver?



Question 3: Configure and Management from Java.

What I believe is that the Kamel CLI client is written in Go. So the flow
is as follows:

A) Write some Camel code in the available languages.

https://camel.apache.org/camel-k/latest/languages/languages.html

B) Use Kamel CLI to run/deploy it on Kubernetes

C) Use something like hawt.io to manage it.

Is this also available from Java (or as Camel component) to
configure/deploy it from Java code and to manage it from Java code?


Question 4: Scaling

Kubernetes is ideal for horizontal scaling. How does this work with CamelK?
Say I have the following two routes:

from("activemq:topic:foo")
  .to("sftp://sample.com/someDir")

from("sftp://sample.com/someDir")
  .to("activemq:topic:foo")

When the routes are scaled. How does CamelK prevent that in the first route
multiple messages arrive in the same FTP directory. And in the second
example there are multiple routes listening to the same directory and get
file locks. Does it have possibilities how to scale the routes?

Kind regards,

Raymond
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

Claus Ibsen-2
Hi

Its a bunch of questions I will give a few points with the limited time I have.

We want to do a webinar in the future about Camel K with more deep
dive, especially around its 1.0 GA release timeframe.

Ad 1)
You can read about dependency resolution here
https://camel.apache.org/camel-k/latest/configuration/dependencies.html

Camel K uses the camel-catalog to know how to map from component names
to maven artifacts.
https://github.com/apache/camel/tree/master/catalog/camel-catalog


Ad 2)
You can specify multiple files from the CLI

kamel run File1.java File2.java

Being able to run with *.java or something is AFAIK not currently
supported, but we are open for ideas.

Ad 3)
Camel K uses standard Kubernetes CRD (custom resource definitions).
So any tool/language that can talk to the k8s API server (rest calls)
you can use.
And of course also the kubernetes client - kubectl.

For Java there is the kubernetes-java client that has java api for k8s.

Ad 4)
Scaling is not Camel K specific. You can setup replicas and quotes and
limits etc which are standard k8s concepts.
But for scaling then Knative is the interesting add-on for k8s that
event based scaling. Camel K works with that too.

However knative is likely more for modern cloud workloads, and not for
traditional enterprise systems like JMS messaging, or FTP data
transfers.
But Camel can bridge that world and allow Camel components to act as
sink and sources for knative eventing.

For k8s concepts then I also recommend the k8s patterns book by Bilgin
and Roland.
https://k8spatterns.io/




On Mon, Mar 16, 2020 at 6:42 PM ski n <[hidden email]> wrote:

>
> Hi all,
>
> In the last presentation there was also a short demo on CamelK. That looked
> very interesting, but I have some questions on how does it work, especially
> in more complex use cases.
>
> Here are some questions I have:
>
> Question 1: Dependency resolution
>
> In CamelK you can provide the following:
>
> from("imap://[hidden email]")
>   .to("seda:output")
>
> Because imap is not part of core this will start the dependency resolution.
> How does this work? How does CamelK know it needs the mail component as
> dependency? Is there some kind of fixed mapping between the scheme and
> component?
>
> Say for example I have the following route:
>
> from("seda:input")
>   .to("seda:output")
>
> This is then run on Kubernetes/CamelK. Then I change this to:
>
> from("imap://[hidden email]")
>   .to("seda:output")
>
> Is this dependency add as "Maven" dependency and then recompiled? Or is the
> jar downloaded and the added? Or something else...
>
>
> Question 2: Multiple files
>
> In the Java example:
> https://camel.apache.org/camel-k/latest/languages/java.html
>
> There is one simple route with one file. How does it work in more complex
> cases when there are multiple files?
>
> For example I have the following package:
>
> Registry.java
> Route1.java
> Route2.java
> Processor.java
> Connection.java
>
> In Route1.java data is retrieved with the SQL component out of an Oracle
> database, also this routs calls a processor. In route 2 the pojo is
> marshalled to JSON and send to a HTTP endpoint. In registry things like
> SSL, metrics are configured. In the processor some mapping is done and in
> the connection the Oracle driver is configured.
>
> Can I run a whole package with the Kamel CLI? How does Kamel gets for
> example the dependency for the Oracle driver?
>
>
>
> Question 3: Configure and Management from Java.
>
> What I believe is that the Kamel CLI client is written in Go. So the flow
> is as follows:
>
> A) Write some Camel code in the available languages.
>
> https://camel.apache.org/camel-k/latest/languages/languages.html
>
> B) Use Kamel CLI to run/deploy it on Kubernetes
>
> C) Use something like hawt.io to manage it.
>
> Is this also available from Java (or as Camel component) to
> configure/deploy it from Java code and to manage it from Java code?
>
>
> Question 4: Scaling
>
> Kubernetes is ideal for horizontal scaling. How does this work with CamelK?
> Say I have the following two routes:
>
> from("activemq:topic:foo")
>   .to("sftp://sample.com/someDir")
>
> from("sftp://sample.com/someDir")
>   .to("activemq:topic:foo")
>
> When the routes are scaled. How does CamelK prevent that in the first route
> multiple messages arrive in the same FTP directory. And in the second
> example there are multiple routes listening to the same directory and get
> file locks. Does it have possibilities how to scale the routes?
>
> Kind regards,
>
> Raymond



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

Nicola Ferraro
In reply to this post by skin27
Hi, responses inline.

Il lun 16 mar 2020, 18:42 ski n <[hidden email]> ha scritto:

> Hi all,
>
> In the last presentation there was also a short demo on CamelK. That looked
> very interesting, but I have some questions on how does it work, especially
> in more complex use cases.
>
> Here are some questions I have:
>
> Question 1: Dependency resolution
>
> In CamelK you can provide the following:
>
> from("imap://[hidden email]")
>   .to("seda:output")
>
> Because imap is not part of core this will start the dependency resolution.
> How does this work? How does CamelK know it needs the mail component as
> dependency? Is there some kind of fixed mapping between the scheme and
> component?
>

Component IDs are known and listed in an artifact called "camel catalog"
for each release. So camel k does the reverse mapping from the catalog and
adds the needed camel library.


> Say for example I have the following route:
>
> from("seda:input")
>   .to("seda:output")
>
> This is then run on Kubernetes/CamelK. Then I change this to:
>
> from("imap://[hidden email]")
>   .to("seda:output")
>
> Is this dependency add as "Maven" dependency and then recompiled? Or is the
> jar downloaded and the added? Or something else...
>

The integration goes in a "Building Kit" state where a container image is
built for the new integration. The algorithm is complex, but it's likely in
this case that the image for seda is enhanced with new jars for the imap
component and that results in a new image. Building the image takes some
more time, but it's done transparently. If, instead, you don't change the
used components, no new jar is added and the redeployment is much faster.


>
> Question 2: Multiple files
>
> In the Java example:
> https://camel.apache.org/camel-k/latest/languages/java.html
>
> There is one simple route with one file. How does it work in more complex
> cases when there are multiple files?
>
> For example I have the following package:
>
> Registry.java
> Route1.java
> Route2.java
> Processor.java
> Connection.java
>
> In Route1.java data is retrieved with the SQL component out of an Oracle
> database, also this routs calls a processor. In route 2 the pojo is
> marshalled to JSON and send to a HTTP endpoint. In registry things like
> SSL, metrics are configured. In the processor some mapping is done and in
> the connection the Oracle driver is configured.
>
> Can I run a whole package with the Kamel CLI? How does Kamel gets for
> example the dependency for the Oracle driver?
>


Camel k creates a base images that is suitable for all the routes (i.e.
with all the dependencies required by all routes), since they are run in
the same JVM in the end.
Additional dependencies can be provided using the "-d
mvn:group:artifact:version": that way you can add the Oracle driver.



>
>
>
>
>
> Question 3: Configure and Management from Java.
>
> What I believe is that the Kamel CLI client is written in Go. So the flow
> is as follows:
>
> A) Write some Camel code in the available languages.
>
> https://camel.apache.org/camel-k/latest/languages/languages.html
>
> B) Use Kamel CLI to run/deploy it on Kubernetes
>
> C) Use something like hawt.io to manage it.
>
> Is this also available from Java (or as Camel component) to
> configure/deploy it from Java code and to manage it from Java code?
>

No, "kamel" should be used to deploy integrations. But integrations are
Kubernetes custom resources, so they also have a YAML/json representation
in the cluster and you can use Kubernetes tools to manage them (like
kubectl). But the workflow should not start from Java. Also there's no need
to create a project structure (like a maven structure): we prefer a
non-structured approach and ideally camel k integrations should be made of
one/few files for portability.


> Question 4: Scaling
>
> Kubernetes is ideal for horizontal scaling. How does this work with CamelK?
> Say I have the following two routes:
>
> from("activemq:topic:foo")
>   .to("sftp://sample.com/someDir")
>
> from("sftp://sample.com/someDir")
>   .to("activemq:topic:foo")
>
> When the routes are scaled. How does CamelK prevent that in the first route
> multiple messages arrive in the same FTP directory. And in the second
> example there are multiple routes listening to the same directory and get
> file locks. Does it have possibilities how to scale the routes?
>

There's support for master component of you need to run only one instance
of the route. But in this case you want to scale, so you've to reason using
knowledge from classic camel.

E.g. in the first case I'd use an approach to name files so to avoid
collision (the POD_NAME environment variable is populated with the name of
the pod instance if you need a way to avoid collision).
For the second route, I expect that sftp provides fine grained lock on the
files, so it should scale to multiple instances correctly.


> Kind regards,
>
> Raymond
>
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

skin27
In reply to this post by Claus Ibsen-2
 Ad 1) Dependency resolution

As I understand it.

a) The user doesn't need to to specify Camel dependencies, but other
dependency (like i.e. Guava) needs to be explicitly defined
b) If it's a Camel dependency then CamelK will map the scheme to the maven
artifact
c) The Camel and all other artifacts are then downloaded from maven central
if needed and compiled (or injected at runtime?)

Seems like a very useful feature for other implementations like Camel
Standalone, Wildfly/EAP, Spring boot, Karaf as well.

Ad 2) Running multiple files

Would be nice to specify just a root directory and Kamel will scans this
directory recursively.

Ad 3) Interacting with CamelK

So the option is either to use "Kamel CLI" or for Java use the
kubernetes-java client that has java api for k8s.

OK, thanks. Wished to stay within the Camel Framework as much as possible.
My estimation is:

Option 1)  Easy to use Kamel, but hard hard to combine this with
traditional Camel apps ( Camel Standalone, Wildfly/EAP, Spring boot, Karaf)
Option 2)  Hard to use K8's API (need to write a CamelK client wrapper),
but easy to combine with traditional Camel.

My question comes from that there is already a lot of Camel implementations
out there. It's already a big step to go from for example Wildfly to Spring
Boot, but to Kubernetes seems to be a bigger hurdle.

For example now I have a Spring boot Camel application. Everything
Registry, Route, Processors works perfectly. Say one company wants to use
this application on a traditional VM and a second company want to use the
same application in OpenShift. When I move that application to Kubernetes
can I move the workload from my Spring boot application running in
Openshift to KNative/CamelK and still manage it from my CamelContext/Spring
Boot? Or do I need two code bases then?

As you mentioned before companies have all kind of different
workloads/protocols. The nice things about Camel I can all be
configured/managed this form one place/framework. Seems like I need to
rewrite/migrate the event based routes first to the CamelK way of working.

So my main point is that while you may ideally switch to an event
based/microservice world where you run locally and in the cloud everything
on Kubernetes or Openshift. However if both worlds needs to be supported
then you stand with one leg in one world and with your other leg in the
other. Moving back and forward. Looking for reuse and reconcile both worlds
a little :)


Kind regards,

Raymond



Op ma 16 mrt. 2020 om 19:48 schreef Claus Ibsen <[hidden email]>:

> Hi
>
> Its a bunch of questions I will give a few points with the limited time I
> have.
>
> We want to do a webinar in the future about Camel K with more deep
> dive, especially around its 1.0 GA release timeframe.
>
> Ad 1)
> You can read about dependency resolution here
> https://camel.apache.org/camel-k/latest/configuration/dependencies.html
>
> Camel K uses the camel-catalog to know how to map from component names
> to maven artifacts.
> https://github.com/apache/camel/tree/master/catalog/camel-catalog
>
>
> Ad 2)
> You can specify multiple files from the CLI
>
> kamel run File1.java File2.java
>
> Being able to run with *.java or something is AFAIK not currently
> supported, but we are open for ideas.
>
> Ad 3)
> Camel K uses standard Kubernetes CRD (custom resource definitions).
> So any tool/language that can talk to the k8s API server (rest calls)
> you can use.
> And of course also the kubernetes client - kubectl.
>
> For Java there is the kubernetes-java client that has java api for k8s.
>
> Ad 4)
> Scaling is not Camel K specific. You can setup replicas and quotes and
> limits etc which are standard k8s concepts.
> But for scaling then Knative is the interesting add-on for k8s that
> event based scaling. Camel K works with that too.
>
> However knative is likely more for modern cloud workloads, and not for
> traditional enterprise systems like JMS messaging, or FTP data
> transfers.
> But Camel can bridge that world and allow Camel components to act as
> sink and sources for knative eventing.
>
> For k8s concepts then I also recommend the k8s patterns book by Bilgin
> and Roland.
> https://k8spatterns.io/
>
>
>
>
> On Mon, Mar 16, 2020 at 6:42 PM ski n <[hidden email]> wrote:
> >
> > Hi all,
> >
> > In the last presentation there was also a short demo on CamelK. That
> looked
> > very interesting, but I have some questions on how does it work,
> especially
> > in more complex use cases.
> >
> > Here are some questions I have:
> >
> > Question 1: Dependency resolution
> >
> > In CamelK you can provide the following:
> >
> > from("imap://[hidden email]")
> >   .to("seda:output")
> >
> > Because imap is not part of core this will start the dependency
> resolution.
> > How does this work? How does CamelK know it needs the mail component as
> > dependency? Is there some kind of fixed mapping between the scheme and
> > component?
> >
> > Say for example I have the following route:
> >
> > from("seda:input")
> >   .to("seda:output")
> >
> > This is then run on Kubernetes/CamelK. Then I change this to:
> >
> > from("imap://[hidden email]")
> >   .to("seda:output")
> >
> > Is this dependency add as "Maven" dependency and then recompiled? Or is
> the
> > jar downloaded and the added? Or something else...
> >
> >
> > Question 2: Multiple files
> >
> > In the Java example:
> > https://camel.apache.org/camel-k/latest/languages/java.html
> >
> > There is one simple route with one file. How does it work in more complex
> > cases when there are multiple files?
> >
> > For example I have the following package:
> >
> > Registry.java
> > Route1.java
> > Route2.java
> > Processor.java
> > Connection.java
> >
> > In Route1.java data is retrieved with the SQL component out of an Oracle
> > database, also this routs calls a processor. In route 2 the pojo is
> > marshalled to JSON and send to a HTTP endpoint. In registry things like
> > SSL, metrics are configured. In the processor some mapping is done and in
> > the connection the Oracle driver is configured.
> >
> > Can I run a whole package with the Kamel CLI? How does Kamel gets for
> > example the dependency for the Oracle driver?
> >
> >
> >
> > Question 3: Configure and Management from Java.
> >
> > What I believe is that the Kamel CLI client is written in Go. So the flow
> > is as follows:
> >
> > A) Write some Camel code in the available languages.
> >
> > https://camel.apache.org/camel-k/latest/languages/languages.html
> >
> > B) Use Kamel CLI to run/deploy it on Kubernetes
> >
> > C) Use something like hawt.io to manage it.
> >
> > Is this also available from Java (or as Camel component) to
> > configure/deploy it from Java code and to manage it from Java code?
> >
> >
> > Question 4: Scaling
> >
> > Kubernetes is ideal for horizontal scaling. How does this work with
> CamelK?
> > Say I have the following two routes:
> >
> > from("activemq:topic:foo")
> >   .to("sftp://sample.com/someDir")
> >
> > from("sftp://sample.com/someDir")
> >   .to("activemq:topic:foo")
> >
> > When the routes are scaled. How does CamelK prevent that in the first
> route
> > multiple messages arrive in the same FTP directory. And in the second
> > example there are multiple routes listening to the same directory and get
> > file locks. Does it have possibilities how to scale the routes?
> >
> > Kind regards,
> >
> > Raymond
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

Claus Ibsen-2
Camel K is not for a traditional Java development project with many
dependencies and source code files, and complex builds and so on.
Its for low-code/no-code lightweight integration in the cloud (=
kubernetes) for users whom may not be hard-core Java developers.

For traditional java development projects then look at camel-quarkus
or still use spring boot (but SB is heavy for Java in containers) for
cloud workloads (= kubernetes).
That becomes just a standard container image with a JVM base that you run.

Camel K runs via "intelligence" via the Camel K operator that "figures
out how to run your stuff".

With Camel K its still 100% Camel and you have CamelContext and all of
that. But it focused on hiding that more, and just make the route DSL
front and center, where you do "low code" to specify what to integrate
together.





On Tue, Mar 17, 2020 at 9:13 AM ski n <[hidden email]> wrote:

>
>  Ad 1) Dependency resolution
>
> As I understand it.
>
> a) The user doesn't need to to specify Camel dependencies, but other
> dependency (like i.e. Guava) needs to be explicitly defined
> b) If it's a Camel dependency then CamelK will map the scheme to the maven
> artifact
> c) The Camel and all other artifacts are then downloaded from maven central
> if needed and compiled (or injected at runtime?)
>
> Seems like a very useful feature for other implementations like Camel
> Standalone, Wildfly/EAP, Spring boot, Karaf as well.
>
> Ad 2) Running multiple files
>
> Would be nice to specify just a root directory and Kamel will scans this
> directory recursively.
>
> Ad 3) Interacting with CamelK
>
> So the option is either to use "Kamel CLI" or for Java use the
> kubernetes-java client that has java api for k8s.
>
> OK, thanks. Wished to stay within the Camel Framework as much as possible.
> My estimation is:
>
> Option 1)  Easy to use Kamel, but hard hard to combine this with
> traditional Camel apps ( Camel Standalone, Wildfly/EAP, Spring boot, Karaf)
> Option 2)  Hard to use K8's API (need to write a CamelK client wrapper),
> but easy to combine with traditional Camel.
>
> My question comes from that there is already a lot of Camel implementations
> out there. It's already a big step to go from for example Wildfly to Spring
> Boot, but to Kubernetes seems to be a bigger hurdle.
>
> For example now I have a Spring boot Camel application. Everything
> Registry, Route, Processors works perfectly. Say one company wants to use
> this application on a traditional VM and a second company want to use the
> same application in OpenShift. When I move that application to Kubernetes
> can I move the workload from my Spring boot application running in
> Openshift to KNative/CamelK and still manage it from my CamelContext/Spring
> Boot? Or do I need two code bases then?
>
> As you mentioned before companies have all kind of different
> workloads/protocols. The nice things about Camel I can all be
> configured/managed this form one place/framework. Seems like I need to
> rewrite/migrate the event based routes first to the CamelK way of working.
>
> So my main point is that while you may ideally switch to an event
> based/microservice world where you run locally and in the cloud everything
> on Kubernetes or Openshift. However if both worlds needs to be supported
> then you stand with one leg in one world and with your other leg in the
> other. Moving back and forward. Looking for reuse and reconcile both worlds
> a little :)
>
>
> Kind regards,
>
> Raymond
>
>
>
> Op ma 16 mrt. 2020 om 19:48 schreef Claus Ibsen <[hidden email]>:
>
> > Hi
> >
> > Its a bunch of questions I will give a few points with the limited time I
> > have.
> >
> > We want to do a webinar in the future about Camel K with more deep
> > dive, especially around its 1.0 GA release timeframe.
> >
> > Ad 1)
> > You can read about dependency resolution here
> > https://camel.apache.org/camel-k/latest/configuration/dependencies.html
> >
> > Camel K uses the camel-catalog to know how to map from component names
> > to maven artifacts.
> > https://github.com/apache/camel/tree/master/catalog/camel-catalog
> >
> >
> > Ad 2)
> > You can specify multiple files from the CLI
> >
> > kamel run File1.java File2.java
> >
> > Being able to run with *.java or something is AFAIK not currently
> > supported, but we are open for ideas.
> >
> > Ad 3)
> > Camel K uses standard Kubernetes CRD (custom resource definitions).
> > So any tool/language that can talk to the k8s API server (rest calls)
> > you can use.
> > And of course also the kubernetes client - kubectl.
> >
> > For Java there is the kubernetes-java client that has java api for k8s.
> >
> > Ad 4)
> > Scaling is not Camel K specific. You can setup replicas and quotes and
> > limits etc which are standard k8s concepts.
> > But for scaling then Knative is the interesting add-on for k8s that
> > event based scaling. Camel K works with that too.
> >
> > However knative is likely more for modern cloud workloads, and not for
> > traditional enterprise systems like JMS messaging, or FTP data
> > transfers.
> > But Camel can bridge that world and allow Camel components to act as
> > sink and sources for knative eventing.
> >
> > For k8s concepts then I also recommend the k8s patterns book by Bilgin
> > and Roland.
> > https://k8spatterns.io/
> >
> >
> >
> >
> > On Mon, Mar 16, 2020 at 6:42 PM ski n <[hidden email]> wrote:
> > >
> > > Hi all,
> > >
> > > In the last presentation there was also a short demo on CamelK. That
> > looked
> > > very interesting, but I have some questions on how does it work,
> > especially
> > > in more complex use cases.
> > >
> > > Here are some questions I have:
> > >
> > > Question 1: Dependency resolution
> > >
> > > In CamelK you can provide the following:
> > >
> > > from("imap://[hidden email]")
> > >   .to("seda:output")
> > >
> > > Because imap is not part of core this will start the dependency
> > resolution.
> > > How does this work? How does CamelK know it needs the mail component as
> > > dependency? Is there some kind of fixed mapping between the scheme and
> > > component?
> > >
> > > Say for example I have the following route:
> > >
> > > from("seda:input")
> > >   .to("seda:output")
> > >
> > > This is then run on Kubernetes/CamelK. Then I change this to:
> > >
> > > from("imap://[hidden email]")
> > >   .to("seda:output")
> > >
> > > Is this dependency add as "Maven" dependency and then recompiled? Or is
> > the
> > > jar downloaded and the added? Or something else...
> > >
> > >
> > > Question 2: Multiple files
> > >
> > > In the Java example:
> > > https://camel.apache.org/camel-k/latest/languages/java.html
> > >
> > > There is one simple route with one file. How does it work in more complex
> > > cases when there are multiple files?
> > >
> > > For example I have the following package:
> > >
> > > Registry.java
> > > Route1.java
> > > Route2.java
> > > Processor.java
> > > Connection.java
> > >
> > > In Route1.java data is retrieved with the SQL component out of an Oracle
> > > database, also this routs calls a processor. In route 2 the pojo is
> > > marshalled to JSON and send to a HTTP endpoint. In registry things like
> > > SSL, metrics are configured. In the processor some mapping is done and in
> > > the connection the Oracle driver is configured.
> > >
> > > Can I run a whole package with the Kamel CLI? How does Kamel gets for
> > > example the dependency for the Oracle driver?
> > >
> > >
> > >
> > > Question 3: Configure and Management from Java.
> > >
> > > What I believe is that the Kamel CLI client is written in Go. So the flow
> > > is as follows:
> > >
> > > A) Write some Camel code in the available languages.
> > >
> > > https://camel.apache.org/camel-k/latest/languages/languages.html
> > >
> > > B) Use Kamel CLI to run/deploy it on Kubernetes
> > >
> > > C) Use something like hawt.io to manage it.
> > >
> > > Is this also available from Java (or as Camel component) to
> > > configure/deploy it from Java code and to manage it from Java code?
> > >
> > >
> > > Question 4: Scaling
> > >
> > > Kubernetes is ideal for horizontal scaling. How does this work with
> > CamelK?
> > > Say I have the following two routes:
> > >
> > > from("activemq:topic:foo")
> > >   .to("sftp://sample.com/someDir")
> > >
> > > from("sftp://sample.com/someDir")
> > >   .to("activemq:topic:foo")
> > >
> > > When the routes are scaled. How does CamelK prevent that in the first
> > route
> > > multiple messages arrive in the same FTP directory. And in the second
> > > example there are multiple routes listening to the same directory and get
> > > file locks. Does it have possibilities how to scale the routes?
> > >
> > > Kind regards,
> > >
> > > Raymond
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
> >



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

skin27
Yes, it's clear how Kubernetes/Camel K is used. I just speak from the field
which mostly goes like this:

1) IT strategy is to move (everything) to the cloud
2) Engineers propose Kubernetes
3) The integration specialists propose CamelK
4) The architect propose an "event-driven architecture"

Everyone is enthusiastic, first integration will be some event-driven thing
between microservices. Success! However everyone wants after that
everything works in this one solution. So in practice everyone wants to
reuse non-cloud protocols and patterns. I already see their next request:

"Please get files from our Windows file share and send an email based on
the file"

From a theoretical perspective it easy to say no, but in practice...Also,
to fulfill real business case (event typical cloud use cases) a lot of
complexity needs to be added. In practice no one understands the
limitations. Maybe I would be good that on the Camel page there will be
more explanation on the use cases and limitations of CamelK. Also which
Camel components works good with CamelK and which not.

As a side note:

You spoke of no code/low code solution. When I think of low-code I would
think more of Fuse Online/Syndesis then CamelK. Here it is more of less
code/DSL/convention-over-configuration type of thing, but at the end it's
still code.







Op di 17 mrt. 2020 om 09:23 schreef Claus Ibsen <[hidden email]>:

> Camel K is not for a traditional Java development project with many
> dependencies and source code files, and complex builds and so on.
> Its for low-code/no-code lightweight integration in the cloud (=
> kubernetes) for users whom may not be hard-core Java developers.
>
> For traditional java development projects then look at camel-quarkus
> or still use spring boot (but SB is heavy for Java in containers) for
> cloud workloads (= kubernetes).
> That becomes just a standard container image with a JVM base that you run.
>
> Camel K runs via "intelligence" via the Camel K operator that "figures
> out how to run your stuff".
>
> With Camel K its still 100% Camel and you have CamelContext and all of
> that. But it focused on hiding that more, and just make the route DSL
> front and center, where you do "low code" to specify what to integrate
> together.
>
>
>
>
>
> On Tue, Mar 17, 2020 at 9:13 AM ski n <[hidden email]> wrote:
> >
> >  Ad 1) Dependency resolution
> >
> > As I understand it.
> >
> > a) The user doesn't need to to specify Camel dependencies, but other
> > dependency (like i.e. Guava) needs to be explicitly defined
> > b) If it's a Camel dependency then CamelK will map the scheme to the
> maven
> > artifact
> > c) The Camel and all other artifacts are then downloaded from maven
> central
> > if needed and compiled (or injected at runtime?)
> >
> > Seems like a very useful feature for other implementations like Camel
> > Standalone, Wildfly/EAP, Spring boot, Karaf as well.
> >
> > Ad 2) Running multiple files
> >
> > Would be nice to specify just a root directory and Kamel will scans this
> > directory recursively.
> >
> > Ad 3) Interacting with CamelK
> >
> > So the option is either to use "Kamel CLI" or for Java use the
> > kubernetes-java client that has java api for k8s.
> >
> > OK, thanks. Wished to stay within the Camel Framework as much as
> possible.
> > My estimation is:
> >
> > Option 1)  Easy to use Kamel, but hard hard to combine this with
> > traditional Camel apps ( Camel Standalone, Wildfly/EAP, Spring boot,
> Karaf)
> > Option 2)  Hard to use K8's API (need to write a CamelK client wrapper),
> > but easy to combine with traditional Camel.
> >
> > My question comes from that there is already a lot of Camel
> implementations
> > out there. It's already a big step to go from for example Wildfly to
> Spring
> > Boot, but to Kubernetes seems to be a bigger hurdle.
> >
> > For example now I have a Spring boot Camel application. Everything
> > Registry, Route, Processors works perfectly. Say one company wants to use
> > this application on a traditional VM and a second company want to use the
> > same application in OpenShift. When I move that application to Kubernetes
> > can I move the workload from my Spring boot application running in
> > Openshift to KNative/CamelK and still manage it from my
> CamelContext/Spring
> > Boot? Or do I need two code bases then?
> >
> > As you mentioned before companies have all kind of different
> > workloads/protocols. The nice things about Camel I can all be
> > configured/managed this form one place/framework. Seems like I need to
> > rewrite/migrate the event based routes first to the CamelK way of
> working.
> >
> > So my main point is that while you may ideally switch to an event
> > based/microservice world where you run locally and in the cloud
> everything
> > on Kubernetes or Openshift. However if both worlds needs to be supported
> > then you stand with one leg in one world and with your other leg in the
> > other. Moving back and forward. Looking for reuse and reconcile both
> worlds
> > a little :)
> >
> >
> > Kind regards,
> >
> > Raymond
> >
> >
> >
> > Op ma 16 mrt. 2020 om 19:48 schreef Claus Ibsen <[hidden email]>:
> >
> > > Hi
> > >
> > > Its a bunch of questions I will give a few points with the limited
> time I
> > > have.
> > >
> > > We want to do a webinar in the future about Camel K with more deep
> > > dive, especially around its 1.0 GA release timeframe.
> > >
> > > Ad 1)
> > > You can read about dependency resolution here
> > >
> https://camel.apache.org/camel-k/latest/configuration/dependencies.html
> > >
> > > Camel K uses the camel-catalog to know how to map from component names
> > > to maven artifacts.
> > > https://github.com/apache/camel/tree/master/catalog/camel-catalog
> > >
> > >
> > > Ad 2)
> > > You can specify multiple files from the CLI
> > >
> > > kamel run File1.java File2.java
> > >
> > > Being able to run with *.java or something is AFAIK not currently
> > > supported, but we are open for ideas.
> > >
> > > Ad 3)
> > > Camel K uses standard Kubernetes CRD (custom resource definitions).
> > > So any tool/language that can talk to the k8s API server (rest calls)
> > > you can use.
> > > And of course also the kubernetes client - kubectl.
> > >
> > > For Java there is the kubernetes-java client that has java api for k8s.
> > >
> > > Ad 4)
> > > Scaling is not Camel K specific. You can setup replicas and quotes and
> > > limits etc which are standard k8s concepts.
> > > But for scaling then Knative is the interesting add-on for k8s that
> > > event based scaling. Camel K works with that too.
> > >
> > > However knative is likely more for modern cloud workloads, and not for
> > > traditional enterprise systems like JMS messaging, or FTP data
> > > transfers.
> > > But Camel can bridge that world and allow Camel components to act as
> > > sink and sources for knative eventing.
> > >
> > > For k8s concepts then I also recommend the k8s patterns book by Bilgin
> > > and Roland.
> > > https://k8spatterns.io/
> > >
> > >
> > >
> > >
> > > On Mon, Mar 16, 2020 at 6:42 PM ski n <[hidden email]>
> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > In the last presentation there was also a short demo on CamelK. That
> > > looked
> > > > very interesting, but I have some questions on how does it work,
> > > especially
> > > > in more complex use cases.
> > > >
> > > > Here are some questions I have:
> > > >
> > > > Question 1: Dependency resolution
> > > >
> > > > In CamelK you can provide the following:
> > > >
> > > > from("imap://[hidden email]")
> > > >   .to("seda:output")
> > > >
> > > > Because imap is not part of core this will start the dependency
> > > resolution.
> > > > How does this work? How does CamelK know it needs the mail component
> as
> > > > dependency? Is there some kind of fixed mapping between the scheme
> and
> > > > component?
> > > >
> > > > Say for example I have the following route:
> > > >
> > > > from("seda:input")
> > > >   .to("seda:output")
> > > >
> > > > This is then run on Kubernetes/CamelK. Then I change this to:
> > > >
> > > > from("imap://[hidden email]")
> > > >   .to("seda:output")
> > > >
> > > > Is this dependency add as "Maven" dependency and then recompiled? Or
> is
> > > the
> > > > jar downloaded and the added? Or something else...
> > > >
> > > >
> > > > Question 2: Multiple files
> > > >
> > > > In the Java example:
> > > > https://camel.apache.org/camel-k/latest/languages/java.html
> > > >
> > > > There is one simple route with one file. How does it work in more
> complex
> > > > cases when there are multiple files?
> > > >
> > > > For example I have the following package:
> > > >
> > > > Registry.java
> > > > Route1.java
> > > > Route2.java
> > > > Processor.java
> > > > Connection.java
> > > >
> > > > In Route1.java data is retrieved with the SQL component out of an
> Oracle
> > > > database, also this routs calls a processor. In route 2 the pojo is
> > > > marshalled to JSON and send to a HTTP endpoint. In registry things
> like
> > > > SSL, metrics are configured. In the processor some mapping is done
> and in
> > > > the connection the Oracle driver is configured.
> > > >
> > > > Can I run a whole package with the Kamel CLI? How does Kamel gets for
> > > > example the dependency for the Oracle driver?
> > > >
> > > >
> > > >
> > > > Question 3: Configure and Management from Java.
> > > >
> > > > What I believe is that the Kamel CLI client is written in Go. So the
> flow
> > > > is as follows:
> > > >
> > > > A) Write some Camel code in the available languages.
> > > >
> > > > https://camel.apache.org/camel-k/latest/languages/languages.html
> > > >
> > > > B) Use Kamel CLI to run/deploy it on Kubernetes
> > > >
> > > > C) Use something like hawt.io to manage it.
> > > >
> > > > Is this also available from Java (or as Camel component) to
> > > > configure/deploy it from Java code and to manage it from Java code?
> > > >
> > > >
> > > > Question 4: Scaling
> > > >
> > > > Kubernetes is ideal for horizontal scaling. How does this work with
> > > CamelK?
> > > > Say I have the following two routes:
> > > >
> > > > from("activemq:topic:foo")
> > > >   .to("sftp://sample.com/someDir")
> > > >
> > > > from("sftp://sample.com/someDir")
> > > >   .to("activemq:topic:foo")
> > > >
> > > > When the routes are scaled. How does CamelK prevent that in the first
> > > route
> > > > multiple messages arrive in the same FTP directory. And in the second
> > > > example there are multiple routes listening to the same directory
> and get
> > > > file locks. Does it have possibilities how to scale the routes?
> > > >
> > > > Kind regards,
> > > >
> > > > Raymond
> > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -----------------
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

Claus Ibsen-2
On Tue, Mar 17, 2020 at 12:06 PM ski n <[hidden email]> wrote:

>
> Yes, it's clear how Kubernetes/Camel K is used. I just speak from the field
> which mostly goes like this:
>
> 1) IT strategy is to move (everything) to the cloud
> 2) Engineers propose Kubernetes
> 3) The integration specialists propose CamelK
> 4) The architect propose an "event-driven architecture"
>
> Everyone is enthusiastic, first integration will be some event-driven thing
> between microservices. Success! However everyone wants after that
> everything works in this one solution. So in practice everyone wants to
> reuse non-cloud protocols and patterns. I already see their next request:
>

Ah yeah that is a great observation.
The IT world is not shiny greenfield with only latest and greatest
shiny cloud systems, and protocols.



> "Please get files from our Windows file share and send an email based on
> the file"
>

Yeah you can move a lot of data via files ;)

So knative has sources which you can plugin, but of course its mainly
a few cloud protocols so far, and kafka.
But Camel K can bridge to that with all its 200+ components. But you
need to setup that via knative yaml and also
transform the data to cloud events. So its not super easy plug and
play. But it has potential.


> From a theoretical perspective it easy to say no, but in practice...Also,
> to fulfill real business case (event typical cloud use cases) a lot of
> complexity needs to be added. In practice no one understands the
> limitations. Maybe I would be good that on the Camel page there will be
> more explanation on the use cases and limitations of CamelK. Also which
> Camel components works good with CamelK and which not.
>

All the components work with Camel K (well all the ones you can run
standalone, so skip osgi, servlet and a few others).
It all just runs on a JVM.


> As a side note:
>
> You spoke of no code/low code solution. When I think of low-code I would
> think more of Fuse Online/Syndesis then CamelK. Here it is more of less
> code/DSL/convention-over-configuration type of thing, but at the end it's
> still code.
>

Yeah syndesis is targeting this, but possible mostly the no-code and
non developer space, with its web UI driven "glue together systems"
style.


>
>
>
>
>
>
> Op di 17 mrt. 2020 om 09:23 schreef Claus Ibsen <[hidden email]>:
>
> > Camel K is not for a traditional Java development project with many
> > dependencies and source code files, and complex builds and so on.
> > Its for low-code/no-code lightweight integration in the cloud (=
> > kubernetes) for users whom may not be hard-core Java developers.
> >
> > For traditional java development projects then look at camel-quarkus
> > or still use spring boot (but SB is heavy for Java in containers) for
> > cloud workloads (= kubernetes).
> > That becomes just a standard container image with a JVM base that you run.
> >
> > Camel K runs via "intelligence" via the Camel K operator that "figures
> > out how to run your stuff".
> >
> > With Camel K its still 100% Camel and you have CamelContext and all of
> > that. But it focused on hiding that more, and just make the route DSL
> > front and center, where you do "low code" to specify what to integrate
> > together.
> >
> >
> >
> >
> >
> > On Tue, Mar 17, 2020 at 9:13 AM ski n <[hidden email]> wrote:
> > >
> > >  Ad 1) Dependency resolution
> > >
> > > As I understand it.
> > >
> > > a) The user doesn't need to to specify Camel dependencies, but other
> > > dependency (like i.e. Guava) needs to be explicitly defined
> > > b) If it's a Camel dependency then CamelK will map the scheme to the
> > maven
> > > artifact
> > > c) The Camel and all other artifacts are then downloaded from maven
> > central
> > > if needed and compiled (or injected at runtime?)
> > >
> > > Seems like a very useful feature for other implementations like Camel
> > > Standalone, Wildfly/EAP, Spring boot, Karaf as well.
> > >
> > > Ad 2) Running multiple files
> > >
> > > Would be nice to specify just a root directory and Kamel will scans this
> > > directory recursively.
> > >
> > > Ad 3) Interacting with CamelK
> > >
> > > So the option is either to use "Kamel CLI" or for Java use the
> > > kubernetes-java client that has java api for k8s.
> > >
> > > OK, thanks. Wished to stay within the Camel Framework as much as
> > possible.
> > > My estimation is:
> > >
> > > Option 1)  Easy to use Kamel, but hard hard to combine this with
> > > traditional Camel apps ( Camel Standalone, Wildfly/EAP, Spring boot,
> > Karaf)
> > > Option 2)  Hard to use K8's API (need to write a CamelK client wrapper),
> > > but easy to combine with traditional Camel.
> > >
> > > My question comes from that there is already a lot of Camel
> > implementations
> > > out there. It's already a big step to go from for example Wildfly to
> > Spring
> > > Boot, but to Kubernetes seems to be a bigger hurdle.
> > >
> > > For example now I have a Spring boot Camel application. Everything
> > > Registry, Route, Processors works perfectly. Say one company wants to use
> > > this application on a traditional VM and a second company want to use the
> > > same application in OpenShift. When I move that application to Kubernetes
> > > can I move the workload from my Spring boot application running in
> > > Openshift to KNative/CamelK and still manage it from my
> > CamelContext/Spring
> > > Boot? Or do I need two code bases then?
> > >
> > > As you mentioned before companies have all kind of different
> > > workloads/protocols. The nice things about Camel I can all be
> > > configured/managed this form one place/framework. Seems like I need to
> > > rewrite/migrate the event based routes first to the CamelK way of
> > working.
> > >
> > > So my main point is that while you may ideally switch to an event
> > > based/microservice world where you run locally and in the cloud
> > everything
> > > on Kubernetes or Openshift. However if both worlds needs to be supported
> > > then you stand with one leg in one world and with your other leg in the
> > > other. Moving back and forward. Looking for reuse and reconcile both
> > worlds
> > > a little :)
> > >
> > >
> > > Kind regards,
> > >
> > > Raymond
> > >
> > >
> > >
> > > Op ma 16 mrt. 2020 om 19:48 schreef Claus Ibsen <[hidden email]>:
> > >
> > > > Hi
> > > >
> > > > Its a bunch of questions I will give a few points with the limited
> > time I
> > > > have.
> > > >
> > > > We want to do a webinar in the future about Camel K with more deep
> > > > dive, especially around its 1.0 GA release timeframe.
> > > >
> > > > Ad 1)
> > > > You can read about dependency resolution here
> > > >
> > https://camel.apache.org/camel-k/latest/configuration/dependencies.html
> > > >
> > > > Camel K uses the camel-catalog to know how to map from component names
> > > > to maven artifacts.
> > > > https://github.com/apache/camel/tree/master/catalog/camel-catalog
> > > >
> > > >
> > > > Ad 2)
> > > > You can specify multiple files from the CLI
> > > >
> > > > kamel run File1.java File2.java
> > > >
> > > > Being able to run with *.java or something is AFAIK not currently
> > > > supported, but we are open for ideas.
> > > >
> > > > Ad 3)
> > > > Camel K uses standard Kubernetes CRD (custom resource definitions).
> > > > So any tool/language that can talk to the k8s API server (rest calls)
> > > > you can use.
> > > > And of course also the kubernetes client - kubectl.
> > > >
> > > > For Java there is the kubernetes-java client that has java api for k8s.
> > > >
> > > > Ad 4)
> > > > Scaling is not Camel K specific. You can setup replicas and quotes and
> > > > limits etc which are standard k8s concepts.
> > > > But for scaling then Knative is the interesting add-on for k8s that
> > > > event based scaling. Camel K works with that too.
> > > >
> > > > However knative is likely more for modern cloud workloads, and not for
> > > > traditional enterprise systems like JMS messaging, or FTP data
> > > > transfers.
> > > > But Camel can bridge that world and allow Camel components to act as
> > > > sink and sources for knative eventing.
> > > >
> > > > For k8s concepts then I also recommend the k8s patterns book by Bilgin
> > > > and Roland.
> > > > https://k8spatterns.io/
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Mar 16, 2020 at 6:42 PM ski n <[hidden email]>
> > wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > In the last presentation there was also a short demo on CamelK. That
> > > > looked
> > > > > very interesting, but I have some questions on how does it work,
> > > > especially
> > > > > in more complex use cases.
> > > > >
> > > > > Here are some questions I have:
> > > > >
> > > > > Question 1: Dependency resolution
> > > > >
> > > > > In CamelK you can provide the following:
> > > > >
> > > > > from("imap://[hidden email]")
> > > > >   .to("seda:output")
> > > > >
> > > > > Because imap is not part of core this will start the dependency
> > > > resolution.
> > > > > How does this work? How does CamelK know it needs the mail component
> > as
> > > > > dependency? Is there some kind of fixed mapping between the scheme
> > and
> > > > > component?
> > > > >
> > > > > Say for example I have the following route:
> > > > >
> > > > > from("seda:input")
> > > > >   .to("seda:output")
> > > > >
> > > > > This is then run on Kubernetes/CamelK. Then I change this to:
> > > > >
> > > > > from("imap://[hidden email]")
> > > > >   .to("seda:output")
> > > > >
> > > > > Is this dependency add as "Maven" dependency and then recompiled? Or
> > is
> > > > the
> > > > > jar downloaded and the added? Or something else...
> > > > >
> > > > >
> > > > > Question 2: Multiple files
> > > > >
> > > > > In the Java example:
> > > > > https://camel.apache.org/camel-k/latest/languages/java.html
> > > > >
> > > > > There is one simple route with one file. How does it work in more
> > complex
> > > > > cases when there are multiple files?
> > > > >
> > > > > For example I have the following package:
> > > > >
> > > > > Registry.java
> > > > > Route1.java
> > > > > Route2.java
> > > > > Processor.java
> > > > > Connection.java
> > > > >
> > > > > In Route1.java data is retrieved with the SQL component out of an
> > Oracle
> > > > > database, also this routs calls a processor. In route 2 the pojo is
> > > > > marshalled to JSON and send to a HTTP endpoint. In registry things
> > like
> > > > > SSL, metrics are configured. In the processor some mapping is done
> > and in
> > > > > the connection the Oracle driver is configured.
> > > > >
> > > > > Can I run a whole package with the Kamel CLI? How does Kamel gets for
> > > > > example the dependency for the Oracle driver?
> > > > >
> > > > >
> > > > >
> > > > > Question 3: Configure and Management from Java.
> > > > >
> > > > > What I believe is that the Kamel CLI client is written in Go. So the
> > flow
> > > > > is as follows:
> > > > >
> > > > > A) Write some Camel code in the available languages.
> > > > >
> > > > > https://camel.apache.org/camel-k/latest/languages/languages.html
> > > > >
> > > > > B) Use Kamel CLI to run/deploy it on Kubernetes
> > > > >
> > > > > C) Use something like hawt.io to manage it.
> > > > >
> > > > > Is this also available from Java (or as Camel component) to
> > > > > configure/deploy it from Java code and to manage it from Java code?
> > > > >
> > > > >
> > > > > Question 4: Scaling
> > > > >
> > > > > Kubernetes is ideal for horizontal scaling. How does this work with
> > > > CamelK?
> > > > > Say I have the following two routes:
> > > > >
> > > > > from("activemq:topic:foo")
> > > > >   .to("sftp://sample.com/someDir")
> > > > >
> > > > > from("sftp://sample.com/someDir")
> > > > >   .to("activemq:topic:foo")
> > > > >
> > > > > When the routes are scaled. How does CamelK prevent that in the first
> > > > route
> > > > > multiple messages arrive in the same FTP directory. And in the second
> > > > > example there are multiple routes listening to the same directory
> > and get
> > > > > file locks. Does it have possibilities how to scale the routes?
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Raymond
> > > >
> > > >
> > > >
> > > > --
> > > > Claus Ibsen
> > > > -----------------
> > > > http://davsclaus.com @davsclaus
> > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > >
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > http://davsclaus.com @davsclaus
> > Camel in Action 2: https://www.manning.com/ibsen2
> >



--
Claus Ibsen
-----------------
http://davsclaus.com @davsclaus
Camel in Action 2: https://www.manning.com/ibsen2
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

skin27
Thanks for your time, looking forward for the in depth CamelK webinar!

Op di 17 mrt. 2020 om 14:11 schreef Claus Ibsen <[hidden email]>:

> On Tue, Mar 17, 2020 at 12:06 PM ski n <[hidden email]> wrote:
> >
> > Yes, it's clear how Kubernetes/Camel K is used. I just speak from the
> field
> > which mostly goes like this:
> >
> > 1) IT strategy is to move (everything) to the cloud
> > 2) Engineers propose Kubernetes
> > 3) The integration specialists propose CamelK
> > 4) The architect propose an "event-driven architecture"
> >
> > Everyone is enthusiastic, first integration will be some event-driven
> thing
> > between microservices. Success! However everyone wants after that
> > everything works in this one solution. So in practice everyone wants to
> > reuse non-cloud protocols and patterns. I already see their next request:
> >
>
> Ah yeah that is a great observation.
> The IT world is not shiny greenfield with only latest and greatest
> shiny cloud systems, and protocols.
>
>
>
> > "Please get files from our Windows file share and send an email based on
> > the file"
> >
>
> Yeah you can move a lot of data via files ;)
>
> So knative has sources which you can plugin, but of course its mainly
> a few cloud protocols so far, and kafka.
> But Camel K can bridge to that with all its 200+ components. But you
> need to setup that via knative yaml and also
> transform the data to cloud events. So its not super easy plug and
> play. But it has potential.
>
>
> > From a theoretical perspective it easy to say no, but in practice...Also,
> > to fulfill real business case (event typical cloud use cases) a lot of
> > complexity needs to be added. In practice no one understands the
> > limitations. Maybe I would be good that on the Camel page there will be
> > more explanation on the use cases and limitations of CamelK. Also which
> > Camel components works good with CamelK and which not.
> >
>
> All the components work with Camel K (well all the ones you can run
> standalone, so skip osgi, servlet and a few others).
> It all just runs on a JVM.
>
>
> > As a side note:
> >
> > You spoke of no code/low code solution. When I think of low-code I would
> > think more of Fuse Online/Syndesis then CamelK. Here it is more of less
> > code/DSL/convention-over-configuration type of thing, but at the end it's
> > still code.
> >
>
> Yeah syndesis is targeting this, but possible mostly the no-code and
> non developer space, with its web UI driven "glue together systems"
> style.
>
>
> >
> >
> >
> >
> >
> >
> > Op di 17 mrt. 2020 om 09:23 schreef Claus Ibsen <[hidden email]>:
> >
> > > Camel K is not for a traditional Java development project with many
> > > dependencies and source code files, and complex builds and so on.
> > > Its for low-code/no-code lightweight integration in the cloud (=
> > > kubernetes) for users whom may not be hard-core Java developers.
> > >
> > > For traditional java development projects then look at camel-quarkus
> > > or still use spring boot (but SB is heavy for Java in containers) for
> > > cloud workloads (= kubernetes).
> > > That becomes just a standard container image with a JVM base that you
> run.
> > >
> > > Camel K runs via "intelligence" via the Camel K operator that "figures
> > > out how to run your stuff".
> > >
> > > With Camel K its still 100% Camel and you have CamelContext and all of
> > > that. But it focused on hiding that more, and just make the route DSL
> > > front and center, where you do "low code" to specify what to integrate
> > > together.
> > >
> > >
> > >
> > >
> > >
> > > On Tue, Mar 17, 2020 at 9:13 AM ski n <[hidden email]>
> wrote:
> > > >
> > > >  Ad 1) Dependency resolution
> > > >
> > > > As I understand it.
> > > >
> > > > a) The user doesn't need to to specify Camel dependencies, but other
> > > > dependency (like i.e. Guava) needs to be explicitly defined
> > > > b) If it's a Camel dependency then CamelK will map the scheme to the
> > > maven
> > > > artifact
> > > > c) The Camel and all other artifacts are then downloaded from maven
> > > central
> > > > if needed and compiled (or injected at runtime?)
> > > >
> > > > Seems like a very useful feature for other implementations like Camel
> > > > Standalone, Wildfly/EAP, Spring boot, Karaf as well.
> > > >
> > > > Ad 2) Running multiple files
> > > >
> > > > Would be nice to specify just a root directory and Kamel will scans
> this
> > > > directory recursively.
> > > >
> > > > Ad 3) Interacting with CamelK
> > > >
> > > > So the option is either to use "Kamel CLI" or for Java use the
> > > > kubernetes-java client that has java api for k8s.
> > > >
> > > > OK, thanks. Wished to stay within the Camel Framework as much as
> > > possible.
> > > > My estimation is:
> > > >
> > > > Option 1)  Easy to use Kamel, but hard hard to combine this with
> > > > traditional Camel apps ( Camel Standalone, Wildfly/EAP, Spring boot,
> > > Karaf)
> > > > Option 2)  Hard to use K8's API (need to write a CamelK client
> wrapper),
> > > > but easy to combine with traditional Camel.
> > > >
> > > > My question comes from that there is already a lot of Camel
> > > implementations
> > > > out there. It's already a big step to go from for example Wildfly to
> > > Spring
> > > > Boot, but to Kubernetes seems to be a bigger hurdle.
> > > >
> > > > For example now I have a Spring boot Camel application. Everything
> > > > Registry, Route, Processors works perfectly. Say one company wants
> to use
> > > > this application on a traditional VM and a second company want to
> use the
> > > > same application in OpenShift. When I move that application to
> Kubernetes
> > > > can I move the workload from my Spring boot application running in
> > > > Openshift to KNative/CamelK and still manage it from my
> > > CamelContext/Spring
> > > > Boot? Or do I need two code bases then?
> > > >
> > > > As you mentioned before companies have all kind of different
> > > > workloads/protocols. The nice things about Camel I can all be
> > > > configured/managed this form one place/framework. Seems like I need
> to
> > > > rewrite/migrate the event based routes first to the CamelK way of
> > > working.
> > > >
> > > > So my main point is that while you may ideally switch to an event
> > > > based/microservice world where you run locally and in the cloud
> > > everything
> > > > on Kubernetes or Openshift. However if both worlds needs to be
> supported
> > > > then you stand with one leg in one world and with your other leg in
> the
> > > > other. Moving back and forward. Looking for reuse and reconcile both
> > > worlds
> > > > a little :)
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Raymond
> > > >
> > > >
> > > >
> > > > Op ma 16 mrt. 2020 om 19:48 schreef Claus Ibsen <
> [hidden email]>:
> > > >
> > > > > Hi
> > > > >
> > > > > Its a bunch of questions I will give a few points with the limited
> > > time I
> > > > > have.
> > > > >
> > > > > We want to do a webinar in the future about Camel K with more deep
> > > > > dive, especially around its 1.0 GA release timeframe.
> > > > >
> > > > > Ad 1)
> > > > > You can read about dependency resolution here
> > > > >
> > >
> https://camel.apache.org/camel-k/latest/configuration/dependencies.html
> > > > >
> > > > > Camel K uses the camel-catalog to know how to map from component
> names
> > > > > to maven artifacts.
> > > > > https://github.com/apache/camel/tree/master/catalog/camel-catalog
> > > > >
> > > > >
> > > > > Ad 2)
> > > > > You can specify multiple files from the CLI
> > > > >
> > > > > kamel run File1.java File2.java
> > > > >
> > > > > Being able to run with *.java or something is AFAIK not currently
> > > > > supported, but we are open for ideas.
> > > > >
> > > > > Ad 3)
> > > > > Camel K uses standard Kubernetes CRD (custom resource definitions).
> > > > > So any tool/language that can talk to the k8s API server (rest
> calls)
> > > > > you can use.
> > > > > And of course also the kubernetes client - kubectl.
> > > > >
> > > > > For Java there is the kubernetes-java client that has java api for
> k8s.
> > > > >
> > > > > Ad 4)
> > > > > Scaling is not Camel K specific. You can setup replicas and quotes
> and
> > > > > limits etc which are standard k8s concepts.
> > > > > But for scaling then Knative is the interesting add-on for k8s that
> > > > > event based scaling. Camel K works with that too.
> > > > >
> > > > > However knative is likely more for modern cloud workloads, and not
> for
> > > > > traditional enterprise systems like JMS messaging, or FTP data
> > > > > transfers.
> > > > > But Camel can bridge that world and allow Camel components to act
> as
> > > > > sink and sources for knative eventing.
> > > > >
> > > > > For k8s concepts then I also recommend the k8s patterns book by
> Bilgin
> > > > > and Roland.
> > > > > https://k8spatterns.io/
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Mon, Mar 16, 2020 at 6:42 PM ski n <[hidden email]>
> > > wrote:
> > > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > In the last presentation there was also a short demo on CamelK.
> That
> > > > > looked
> > > > > > very interesting, but I have some questions on how does it work,
> > > > > especially
> > > > > > in more complex use cases.
> > > > > >
> > > > > > Here are some questions I have:
> > > > > >
> > > > > > Question 1: Dependency resolution
> > > > > >
> > > > > > In CamelK you can provide the following:
> > > > > >
> > > > > > from("imap://[hidden email]")
> > > > > >   .to("seda:output")
> > > > > >
> > > > > > Because imap is not part of core this will start the dependency
> > > > > resolution.
> > > > > > How does this work? How does CamelK know it needs the mail
> component
> > > as
> > > > > > dependency? Is there some kind of fixed mapping between the
> scheme
> > > and
> > > > > > component?
> > > > > >
> > > > > > Say for example I have the following route:
> > > > > >
> > > > > > from("seda:input")
> > > > > >   .to("seda:output")
> > > > > >
> > > > > > This is then run on Kubernetes/CamelK. Then I change this to:
> > > > > >
> > > > > > from("imap://[hidden email]")
> > > > > >   .to("seda:output")
> > > > > >
> > > > > > Is this dependency add as "Maven" dependency and then
> recompiled? Or
> > > is
> > > > > the
> > > > > > jar downloaded and the added? Or something else...
> > > > > >
> > > > > >
> > > > > > Question 2: Multiple files
> > > > > >
> > > > > > In the Java example:
> > > > > > https://camel.apache.org/camel-k/latest/languages/java.html
> > > > > >
> > > > > > There is one simple route with one file. How does it work in more
> > > complex
> > > > > > cases when there are multiple files?
> > > > > >
> > > > > > For example I have the following package:
> > > > > >
> > > > > > Registry.java
> > > > > > Route1.java
> > > > > > Route2.java
> > > > > > Processor.java
> > > > > > Connection.java
> > > > > >
> > > > > > In Route1.java data is retrieved with the SQL component out of an
> > > Oracle
> > > > > > database, also this routs calls a processor. In route 2 the pojo
> is
> > > > > > marshalled to JSON and send to a HTTP endpoint. In registry
> things
> > > like
> > > > > > SSL, metrics are configured. In the processor some mapping is
> done
> > > and in
> > > > > > the connection the Oracle driver is configured.
> > > > > >
> > > > > > Can I run a whole package with the Kamel CLI? How does Kamel
> gets for
> > > > > > example the dependency for the Oracle driver?
> > > > > >
> > > > > >
> > > > > >
> > > > > > Question 3: Configure and Management from Java.
> > > > > >
> > > > > > What I believe is that the Kamel CLI client is written in Go. So
> the
> > > flow
> > > > > > is as follows:
> > > > > >
> > > > > > A) Write some Camel code in the available languages.
> > > > > >
> > > > > > https://camel.apache.org/camel-k/latest/languages/languages.html
> > > > > >
> > > > > > B) Use Kamel CLI to run/deploy it on Kubernetes
> > > > > >
> > > > > > C) Use something like hawt.io to manage it.
> > > > > >
> > > > > > Is this also available from Java (or as Camel component) to
> > > > > > configure/deploy it from Java code and to manage it from Java
> code?
> > > > > >
> > > > > >
> > > > > > Question 4: Scaling
> > > > > >
> > > > > > Kubernetes is ideal for horizontal scaling. How does this work
> with
> > > > > CamelK?
> > > > > > Say I have the following two routes:
> > > > > >
> > > > > > from("activemq:topic:foo")
> > > > > >   .to("sftp://sample.com/someDir")
> > > > > >
> > > > > > from("sftp://sample.com/someDir")
> > > > > >   .to("activemq:topic:foo")
> > > > > >
> > > > > > When the routes are scaled. How does CamelK prevent that in the
> first
> > > > > route
> > > > > > multiple messages arrive in the same FTP directory. And in the
> second
> > > > > > example there are multiple routes listening to the same directory
> > > and get
> > > > > > file locks. Does it have possibilities how to scale the routes?
> > > > > >
> > > > > > Kind regards,
> > > > > >
> > > > > > Raymond
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Claus Ibsen
> > > > > -----------------
> > > > > http://davsclaus.com @davsclaus
> > > > > Camel in Action 2: https://www.manning.com/ibsen2
> > > > >
> > >
> > >
> > >
> > > --
> > > Claus Ibsen
> > > -----------------
> > > http://davsclaus.com @davsclaus
> > > Camel in Action 2: https://www.manning.com/ibsen2
> > >
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

skin27
In reply to this post by Nicola Ferraro
Thanks for all the answers, it's a lot clearer now what's going on in the
background.

Currently, I think what's mainly missing is more extensive documentation.
Mainly a tutorial (now it only explains something about pipelines). The
tutorial could start with a simple route and then add more complex
examples. A more extensive route, multiple routes, processors,
errorhandlers, metrics, services and so on until a complete integration is
created.

The code for every example could be added in a tab for every support
language (so all examples are supported languages are treated equally). A
little bit like vert.x does on it website (See polyglot -->
https://vertx.io/)

Kind regards,

Raymond

Op wo 18 mrt. 2020 om 20:11 schreef Nicola Ferraro <[hidden email]>:

> Hi, responses inline.
>
> Il lun 16 mar 2020, 18:42 ski n <[hidden email]> ha scritto:
>
> > Hi all,
> >
> > In the last presentation there was also a short demo on CamelK. That
> looked
> > very interesting, but I have some questions on how does it work,
> especially
> > in more complex use cases.
> >
> > Here are some questions I have:
> >
> > Question 1: Dependency resolution
> >
> > In CamelK you can provide the following:
> >
> > from("imap://[hidden email]")
> >   .to("seda:output")
> >
> > Because imap is not part of core this will start the dependency
> resolution.
> > How does this work? How does CamelK know it needs the mail component as
> > dependency? Is there some kind of fixed mapping between the scheme and
> > component?
> >
>
> Component IDs are known and listed in an artifact called "camel catalog"
> for each release. So camel k does the reverse mapping from the catalog and
> adds the needed camel library.
>
>
> > Say for example I have the following route:
> >
> > from("seda:input")
> >   .to("seda:output")
> >
> > This is then run on Kubernetes/CamelK. Then I change this to:
> >
> > from("imap://[hidden email]")
> >   .to("seda:output")
> >
> > Is this dependency add as "Maven" dependency and then recompiled? Or is
> the
> > jar downloaded and the added? Or something else...
> >
>
> The integration goes in a "Building Kit" state where a container image is
> built for the new integration. The algorithm is complex, but it's likely in
> this case that the image for seda is enhanced with new jars for the imap
> component and that results in a new image. Building the image takes some
> more time, but it's done transparently. If, instead, you don't change the
> used components, no new jar is added and the redeployment is much faster.
>
>
> >
> > Question 2: Multiple files
> >
> > In the Java example:
> > https://camel.apache.org/camel-k/latest/languages/java.html
> >
> > There is one simple route with one file. How does it work in more complex
> > cases when there are multiple files?
> >
> > For example I have the following package:
> >
> > Registry.java
> > Route1.java
> > Route2.java
> > Processor.java
> > Connection.java
> >
> > In Route1.java data is retrieved with the SQL component out of an Oracle
> > database, also this routs calls a processor. In route 2 the pojo is
> > marshalled to JSON and send to a HTTP endpoint. In registry things like
> > SSL, metrics are configured. In the processor some mapping is done and in
> > the connection the Oracle driver is configured.
> >
> > Can I run a whole package with the Kamel CLI? How does Kamel gets for
> > example the dependency for the Oracle driver?
> >
>
>
> Camel k creates a base images that is suitable for all the routes (i.e.
> with all the dependencies required by all routes), since they are run in
> the same JVM in the end.
> Additional dependencies can be provided using the "-d
> mvn:group:artifact:version": that way you can add the Oracle driver.
>
>
>
> >
> >
> >
> >
> >
> > Question 3: Configure and Management from Java.
> >
> > What I believe is that the Kamel CLI client is written in Go. So the flow
> > is as follows:
> >
> > A) Write some Camel code in the available languages.
> >
> > https://camel.apache.org/camel-k/latest/languages/languages.html
> >
> > B) Use Kamel CLI to run/deploy it on Kubernetes
> >
> > C) Use something like hawt.io to manage it.
> >
> > Is this also available from Java (or as Camel component) to
> > configure/deploy it from Java code and to manage it from Java code?
> >
>
> No, "kamel" should be used to deploy integrations. But integrations are
> Kubernetes custom resources, so they also have a YAML/json representation
> in the cluster and you can use Kubernetes tools to manage them (like
> kubectl). But the workflow should not start from Java. Also there's no need
> to create a project structure (like a maven structure): we prefer a
> non-structured approach and ideally camel k integrations should be made of
> one/few files for portability.
>
>
> > Question 4: Scaling
> >
> > Kubernetes is ideal for horizontal scaling. How does this work with
> CamelK?
> > Say I have the following two routes:
> >
> > from("activemq:topic:foo")
> >   .to("sftp://sample.com/someDir")
> >
> > from("sftp://sample.com/someDir")
> >   .to("activemq:topic:foo")
> >
> > When the routes are scaled. How does CamelK prevent that in the first
> route
> > multiple messages arrive in the same FTP directory. And in the second
> > example there are multiple routes listening to the same directory and get
> > file locks. Does it have possibilities how to scale the routes?
> >
>
> There's support for master component of you need to run only one instance
> of the route. But in this case you want to scale, so you've to reason using
> knowledge from classic camel.
>
> E.g. in the first case I'd use an approach to name files so to avoid
> collision (the POD_NAME environment variable is populated with the name of
> the pod instance if you need a way to avoid collision).
> For the second route, I expect that sftp provides fine grained lock on the
> files, so it should scale to multiple instances correctly.
>
>
> > Kind regards,
> >
> > Raymond
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

Zoran Regvart-2
Hi Raymond,

On Mon, Mar 23, 2020 at 7:38 PM ski n <[hidden email]> wrote:
> The code for every example could be added in a tab for every support
> language (so all examples are supported languages are treated equally). A
> little bit like vert.x does on it website (See polyglot -->
> https://vertx.io/)

I think this is good feedback, can you provide the full link to the
example on the vert.x website, I have trouble finding this "polyglot"
documentation on vert.x.

zoran
--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: Some questions on CamelK

skin27
Hi Zoran,

" I think this is good feedback, can you provide the full link to the
example on the vert.x website, I have trouble finding this "polyglot"
documentation on vert.x. "

When you go to the main website of vert.x then there is an item "Polyglot".
Next to it is a sample example of creating a http server in multiple
languages. In this way every example is in every language.

Raymond


Op wo 25 mrt. 2020 om 09:38 schreef Zoran Regvart <[hidden email]>:

> Hi Raymond,
>
> On Mon, Mar 23, 2020 at 7:38 PM ski n <[hidden email]> wrote:
> > The code for every example could be added in a tab for every support
> > language (so all examples are supported languages are treated equally). A
> > little bit like vert.x does on it website (See polyglot -->
> > https://vertx.io/)
>
> I think this is good feedback, can you provide the full link to the
> example on the vert.x website, I have trouble finding this "polyglot"
> documentation on vert.x.
>
> zoran
> --
> Zoran Regvart
>