[Discussion] Sharing test containers/infrastructure among projects

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Discussion] Sharing test containers/infrastructure among projects

Otavio Rodolfo Piske
Hello everyone!

Luca B, Andrea T. and I had a brief discussion / brainstorm today about
adjusting part of our testing infrastructure code so that it is easier for
all the projects to levage testing bits used throughout the project. The
issue that led to this idea can be found here [1].

As it is common these days, a lot of projects use TestContainers [2] to set
up the test infrastructure required for the tests to run. This is how the
integration tests for the Camel Kafka Connector work [3].

In the Camel Kafka Connector, the test infrastructure contains
"AbstractServices". They can be resolved either to containers managed by
TestContainers or remote instances of the services. These services are
injected into the test code using JUnit 5 extensions. It uses a generic
approach and it is not tightly coupled to Kafka Connect or the Camel Kafka
Connector project.

In more concrete terms, the idea is to make this test infrastructure more
easily accessible for other projects. This would involve moving this code
to another project (Camel? Where?), adjusting the few parts that may be
coupled (if any at all) and moving the part of the testing documentation
[4] (check the "Simulating the Test Infrastructure" part).

A few potential benefits of this idea are reducing our effort in handling
the test infrastructure throughout the code base and simplifying how we
assert the behavior of each project for each infrastructure (ie.: we may
want to make sure that components behave the same way across all projects).

What do you think?

1. https://github.com/apache/camel-quarkus/issues/1783
2. https://www.testcontainers.org/
3. https://github.com/apache/camel-kafka-connector/tree/master/tests
4. https://camel.apache.org/camel-kafka-connector/latest/testing.html

Kind regards
--
Otavio R. Piske
http://orpiske.net
Reply | Threaded
Open this post in threaded view
|

Re: [Discussion] Sharing test containers/infrastructure among projects

lburgazzoli
Hi Otavio,

as you know, big +1 on that idea.

As a strategy I'm thinking about:

1. try to contribute to TestContainers to make it easy to configure some
common tasks the implementations they ship and eventually provide some new
ones.
2. create some new modules, don't ask me for a name, with the core
infrastructure code and some specific impl.
3. see if we can easily hook such service in the camel-test* and eventually
deprecate camel-testcontainers*

---
Luca Burgazzoli


On Fri, Sep 11, 2020 at 3:49 PM Otavio Rodolfo Piske <[hidden email]>
wrote:

> Hello everyone!
>
> Luca B, Andrea T. and I had a brief discussion / brainstorm today about
> adjusting part of our testing infrastructure code so that it is easier for
> all the projects to levage testing bits used throughout the project. The
> issue that led to this idea can be found here [1].
>
> As it is common these days, a lot of projects use TestContainers [2] to set
> up the test infrastructure required for the tests to run. This is how the
> integration tests for the Camel Kafka Connector work [3].
>
> In the Camel Kafka Connector, the test infrastructure contains
> "AbstractServices". They can be resolved either to containers managed by
> TestContainers or remote instances of the services. These services are
> injected into the test code using JUnit 5 extensions. It uses a generic
> approach and it is not tightly coupled to Kafka Connect or the Camel Kafka
> Connector project.
>
> In more concrete terms, the idea is to make this test infrastructure more
> easily accessible for other projects. This would involve moving this code
> to another project (Camel? Where?), adjusting the few parts that may be
> coupled (if any at all) and moving the part of the testing documentation
> [4] (check the "Simulating the Test Infrastructure" part).
>
> A few potential benefits of this idea are reducing our effort in handling
> the test infrastructure throughout the code base and simplifying how we
> assert the behavior of each project for each infrastructure (ie.: we may
> want to make sure that components behave the same way across all projects).
>
> What do you think?
>
> 1. https://github.com/apache/camel-quarkus/issues/1783
> 2. https://www.testcontainers.org/
> 3. https://github.com/apache/camel-kafka-connector/tree/master/tests
> 4. https://camel.apache.org/camel-kafka-connector/latest/testing.html
>
> Kind regards
> --
> Otavio R. Piske
> http://orpiske.net
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discussion] Sharing test containers/infrastructure among projects

Otavio Rodolfo Piske
Thanks Luca,

I think we could try getting started with this by doing #2 or #3 and then,
investigate what/if any stuff, we can contribute to Test Containers.

As far as I know, they have some restrictions to what they accept. An
important one is that they only accept TestContainers with official images
(which rule out a few of ours) and this rules out a few of those that we
have (ie.: Artemis, Qpid Interconnect, Apache Hadoop, etc). We would need
Apache to start to provide official images for those otherwise they
wouldn't accept it. So, I think that's the biggest barrier for #1.

If that's OK, I will send a PR with a proposal for moving the current ones
in Camel Kafka Connector to Camel. Something along the lines of strategy #2
and #3.

Kind regards

On Wed, Sep 16, 2020 at 9:29 AM Luca Burgazzoli <[hidden email]>
wrote:

> Hi Otavio,
>
> as you know, big +1 on that idea.
>
> As a strategy I'm thinking about:
>
> 1. try to contribute to TestContainers to make it easy to configure some
> common tasks the implementations they ship and eventually provide some new
> ones.
> 2. create some new modules, don't ask me for a name, with the core
> infrastructure code and some specific impl.
> 3. see if we can easily hook such service in the camel-test* and eventually
> deprecate camel-testcontainers*
>
> ---
> Luca Burgazzoli
>
>
> On Fri, Sep 11, 2020 at 3:49 PM Otavio Rodolfo Piske <[hidden email]
> >
> wrote:
>
> > Hello everyone!
> >
> > Luca B, Andrea T. and I had a brief discussion / brainstorm today about
> > adjusting part of our testing infrastructure code so that it is easier
> for
> > all the projects to levage testing bits used throughout the project. The
> > issue that led to this idea can be found here [1].
> >
> > As it is common these days, a lot of projects use TestContainers [2] to
> set
> > up the test infrastructure required for the tests to run. This is how the
> > integration tests for the Camel Kafka Connector work [3].
> >
> > In the Camel Kafka Connector, the test infrastructure contains
> > "AbstractServices". They can be resolved either to containers managed by
> > TestContainers or remote instances of the services. These services are
> > injected into the test code using JUnit 5 extensions. It uses a generic
> > approach and it is not tightly coupled to Kafka Connect or the Camel
> Kafka
> > Connector project.
> >
> > In more concrete terms, the idea is to make this test infrastructure more
> > easily accessible for other projects. This would involve moving this code
> > to another project (Camel? Where?), adjusting the few parts that may be
> > coupled (if any at all) and moving the part of the testing documentation
> > [4] (check the "Simulating the Test Infrastructure" part).
> >
> > A few potential benefits of this idea are reducing our effort in handling
> > the test infrastructure throughout the code base and simplifying how we
> > assert the behavior of each project for each infrastructure (ie.: we may
> > want to make sure that components behave the same way across all
> projects).
> >
> > What do you think?
> >
> > 1. https://github.com/apache/camel-quarkus/issues/1783
> > 2. https://www.testcontainers.org/
> > 3. https://github.com/apache/camel-kafka-connector/tree/master/tests
> > 4. https://camel.apache.org/camel-kafka-connector/latest/testing.html
> >
> > Kind regards
> > --
> > Otavio R. Piske
> > http://orpiske.net
> >
>


--
Otavio R. Piske
http://orpiske.net