Code coverage

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Code coverage

blanx
Hello fellow Cameleers!

I am looking into twitter component, doing some small refactoring. I
noticed something interesting, code coverage is a little above 50%, in my
opinion that is a really poor %. What do you think that we add some checks
or  when doing PR reviews to also check coverage of added code? This way we
will promote that tests  are mandatory in order to  approve PR. Of course
there will be some cases when tests are not available to be written, anyway
i think this will help us reduce the number of bugs and give us freedom to
add, change and refactor with more confidence.

--
- Đorđe Bajić
Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Jan Bednář
Hi,
It would be great IMO, but I think you need to actually run the tests
for coverage report. We currently skip tests for github PR, because it
takes many hours to test whole codebase - these are running during
nightly build.

Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):

> Hello fellow Cameleers!
>
> I am looking into twitter component, doing some small refactoring. I
> noticed something interesting, code coverage is a little above 50%, in my
> opinion that is a really poor %. What do you think that we add some checks
> or  when doing PR reviews to also check coverage of added code? This way we
> will promote that tests  are mandatory in order to  approve PR. Of course
> there will be some cases when tests are not available to be written, anyway
> i think this will help us reduce the number of bugs and give us freedom to
> add, change and refactor with more confidence.
>

Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

blanx
Hi Jan!

Yes i understand that tests are gonna last long. Idk if there is
possibility to specify to run only tests for that particular component or
project inside the camel?


On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:

> Hi,
> It would be great IMO, but I think you need to actually run the tests
> for coverage report. We currently skip tests for github PR, because it
> takes many hours to test whole codebase - these are running during
> nightly build.
>
> Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> > Hello fellow Cameleers!
> >
> > I am looking into twitter component, doing some small refactoring. I
> > noticed something interesting, code coverage is a little above 50%, in my
> > opinion that is a really poor %. What do you think that we add some
> checks
> > or  when doing PR reviews to also check coverage of added code? This way
> we
> > will promote that tests  are mandatory in order to  approve PR. Of course
> > there will be some cases when tests are not available to be written,
> anyway
> > i think this will help us reduce the number of bugs and give us freedom
> to
> > add, change and refactor with more confidence.
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Andrea Cosentino-3
Hello,

No, incremental build are not supported. The Camel build is too complex for
that.

Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <[hidden email]>
ha scritto:

> Hi Jan!
>
> Yes i understand that tests are gonna last long. Idk if there is
> possibility to specify to run only tests for that particular component or
> project inside the camel?
>
>
> On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:
>
> > Hi,
> > It would be great IMO, but I think you need to actually run the tests
> > for coverage report. We currently skip tests for github PR, because it
> > takes many hours to test whole codebase - these are running during
> > nightly build.
> >
> > Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> > > Hello fellow Cameleers!
> > >
> > > I am looking into twitter component, doing some small refactoring. I
> > > noticed something interesting, code coverage is a little above 50%, in
> my
> > > opinion that is a really poor %. What do you think that we add some
> > checks
> > > or  when doing PR reviews to also check coverage of added code? This
> way
> > we
> > > will promote that tests  are mandatory in order to  approve PR. Of
> course
> > > there will be some cases when tests are not available to be written,
> > anyway
> > > i think this will help us reduce the number of bugs and give us freedom
> > to
> > > add, change and refactor with more confidence.
> > >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

blanx
Hello Andrea, Jan,

In that case, maybe PR reviewers can run tests locally on that branch and
check? What do you guys think?

- Djordje

On Thu, 8 Oct 2020, 08:09 Andrea Cosentino, <[hidden email]> wrote:

> Hello,
>
> No, incremental build are not supported. The Camel build is too complex for
> that.
>
> Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <
> [hidden email]>
> ha scritto:
>
> > Hi Jan!
> >
> > Yes i understand that tests are gonna last long. Idk if there is
> > possibility to specify to run only tests for that particular component or
> > project inside the camel?
> >
> >
> > On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:
> >
> > > Hi,
> > > It would be great IMO, but I think you need to actually run the tests
> > > for coverage report. We currently skip tests for github PR, because it
> > > takes many hours to test whole codebase - these are running during
> > > nightly build.
> > >
> > > Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> > > > Hello fellow Cameleers!
> > > >
> > > > I am looking into twitter component, doing some small refactoring. I
> > > > noticed something interesting, code coverage is a little above 50%,
> in
> > my
> > > > opinion that is a really poor %. What do you think that we add some
> > > checks
> > > > or  when doing PR reviews to also check coverage of added code? This
> > way
> > > we
> > > > will promote that tests  are mandatory in order to  approve PR. Of
> > course
> > > > there will be some cases when tests are not available to be written,
> > > anyway
> > > > i think this will help us reduce the number of bugs and give us
> freedom
> > > to
> > > > add, change and refactor with more confidence.
> > > >
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Andrea Cosentino-3
I don't think it is feasible. Nobody would do it. It's time consuming.

Il gio 8 ott 2020, 09:21 Djordje Bajić <[hidden email]> ha scritto:

> Hello Andrea, Jan,
>
> In that case, maybe PR reviewers can run tests locally on that branch and
> check? What do you guys think?
>
> - Djordje
>
> On Thu, 8 Oct 2020, 08:09 Andrea Cosentino, <[hidden email]> wrote:
>
> > Hello,
> >
> > No, incremental build are not supported. The Camel build is too complex
> for
> > that.
> >
> > Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <
> > [hidden email]>
> > ha scritto:
> >
> > > Hi Jan!
> > >
> > > Yes i understand that tests are gonna last long. Idk if there is
> > > possibility to specify to run only tests for that particular component
> or
> > > project inside the camel?
> > >
> > >
> > > On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:
> > >
> > > > Hi,
> > > > It would be great IMO, but I think you need to actually run the tests
> > > > for coverage report. We currently skip tests for github PR, because
> it
> > > > takes many hours to test whole codebase - these are running during
> > > > nightly build.
> > > >
> > > > Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> > > > > Hello fellow Cameleers!
> > > > >
> > > > > I am looking into twitter component, doing some small refactoring.
> I
> > > > > noticed something interesting, code coverage is a little above 50%,
> > in
> > > my
> > > > > opinion that is a really poor %. What do you think that we add some
> > > > checks
> > > > > or  when doing PR reviews to also check coverage of added code?
> This
> > > way
> > > > we
> > > > > will promote that tests  are mandatory in order to  approve PR. Of
> > > course
> > > > > there will be some cases when tests are not available to be
> written,
> > > > anyway
> > > > > i think this will help us reduce the number of bugs and give us
> > freedom
> > > > to
> > > > > add, change and refactor with more confidence.
> > > > >
> > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Maria Arias de Reyna Dominguez
Hi,

In any case, maybe a nightly/weekly code coverage is useful to check
which parts of the code are less "tested" and we should put more
effort on them. Even if we can't do it by PR, it will show some light
on the current status of the code.

On Thu, Oct 8, 2020 at 9:30 AM Andrea Cosentino <[hidden email]> wrote:

>
> I don't think it is feasible. Nobody would do it. It's time consuming.
>
> Il gio 8 ott 2020, 09:21 Djordje Bajić <[hidden email]> ha scritto:
>
> > Hello Andrea, Jan,
> >
> > In that case, maybe PR reviewers can run tests locally on that branch and
> > check? What do you guys think?
> >
> > - Djordje
> >
> > On Thu, 8 Oct 2020, 08:09 Andrea Cosentino, <[hidden email]> wrote:
> >
> > > Hello,
> > >
> > > No, incremental build are not supported. The Camel build is too complex
> > for
> > > that.
> > >
> > > Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <
> > > [hidden email]>
> > > ha scritto:
> > >
> > > > Hi Jan!
> > > >
> > > > Yes i understand that tests are gonna last long. Idk if there is
> > > > possibility to specify to run only tests for that particular component
> > or
> > > > project inside the camel?
> > > >
> > > >
> > > > On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:
> > > >
> > > > > Hi,
> > > > > It would be great IMO, but I think you need to actually run the tests
> > > > > for coverage report. We currently skip tests for github PR, because
> > it
> > > > > takes many hours to test whole codebase - these are running during
> > > > > nightly build.
> > > > >
> > > > > Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> > > > > > Hello fellow Cameleers!
> > > > > >
> > > > > > I am looking into twitter component, doing some small refactoring.
> > I
> > > > > > noticed something interesting, code coverage is a little above 50%,
> > > in
> > > > my
> > > > > > opinion that is a really poor %. What do you think that we add some
> > > > > checks
> > > > > > or  when doing PR reviews to also check coverage of added code?
> > This
> > > > way
> > > > > we
> > > > > > will promote that tests  are mandatory in order to  approve PR. Of
> > > > course
> > > > > > there will be some cases when tests are not available to be
> > written,
> > > > > anyway
> > > > > > i think this will help us reduce the number of bugs and give us
> > > freedom
> > > > > to
> > > > > > add, change and refactor with more confidence.
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >

Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Omar Al-Safi
Hi,

We have lgtm.com integrated which helps a bit to check from time to time
but not on every PR since the Camel build is complex. However, I think a
weekly coverage report is not a bad idea, at least it would maybe help a
bit.

Regards,
Omar

On Thu, Oct 8, 2020 at 9:51 AM Maria Arias de Reyna Dominguez <
[hidden email]> wrote:

> Hi,
>
> In any case, maybe a nightly/weekly code coverage is useful to check
> which parts of the code are less "tested" and we should put more
> effort on them. Even if we can't do it by PR, it will show some light
> on the current status of the code.
>
> On Thu, Oct 8, 2020 at 9:30 AM Andrea Cosentino <[hidden email]> wrote:
> >
> > I don't think it is feasible. Nobody would do it. It's time consuming.
> >
> > Il gio 8 ott 2020, 09:21 Djordje Bajić <[hidden email]> ha
> scritto:
> >
> > > Hello Andrea, Jan,
> > >
> > > In that case, maybe PR reviewers can run tests locally on that branch
> and
> > > check? What do you guys think?
> > >
> > > - Djordje
> > >
> > > On Thu, 8 Oct 2020, 08:09 Andrea Cosentino, <[hidden email]> wrote:
> > >
> > > > Hello,
> > > >
> > > > No, incremental build are not supported. The Camel build is too
> complex
> > > for
> > > > that.
> > > >
> > > > Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <
> > > > [hidden email]>
> > > > ha scritto:
> > > >
> > > > > Hi Jan!
> > > > >
> > > > > Yes i understand that tests are gonna last long. Idk if there is
> > > > > possibility to specify to run only tests for that particular
> component
> > > or
> > > > > project inside the camel?
> > > > >
> > > > >
> > > > > On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:
> > > > >
> > > > > > Hi,
> > > > > > It would be great IMO, but I think you need to actually run the
> tests
> > > > > > for coverage report. We currently skip tests for github PR,
> because
> > > it
> > > > > > takes many hours to test whole codebase - these are running
> during
> > > > > > nightly build.
> > > > > >
> > > > > > Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> > > > > > > Hello fellow Cameleers!
> > > > > > >
> > > > > > > I am looking into twitter component, doing some small
> refactoring.
> > > I
> > > > > > > noticed something interesting, code coverage is a little above
> 50%,
> > > > in
> > > > > my
> > > > > > > opinion that is a really poor %. What do you think that we add
> some
> > > > > > checks
> > > > > > > or  when doing PR reviews to also check coverage of added code?
> > > This
> > > > > way
> > > > > > we
> > > > > > > will promote that tests  are mandatory in order to  approve
> PR. Of
> > > > > course
> > > > > > > there will be some cases when tests are not available to be
> > > written,
> > > > > > anyway
> > > > > > > i think this will help us reduce the number of bugs and give us
> > > > freedom
> > > > > > to
> > > > > > > add, change and refactor with more confidence.
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

drekbour
It does feel like a failing. For exactly the reason below - smaller leaf
components (of which there are many) and PRs (which are infinite into
the future) "get away" with weaker testing because of the weight of
historic coverage within the core elements. This is entropy at work and
something a long-lived project might be bothered by.

Have you tried using something like Sandboni to optimise the tests
executed based on the git commits unique to the PR? Any enforced
coverage percentage then becomes specific to the tests selected so
avoids this situation.

Marc

https://github.com/jpmorganchase/sandboni-core

On 08/10/2020 08:59, Omar Al-Safi wrote:

> Hi,
>
> We have lgtm.com integrated which helps a bit to check from time to time
> but not on every PR since the Camel build is complex. However, I think a
> weekly coverage report is not a bad idea, at least it would maybe help a
> bit.
>
> Regards,
> Omar
>
> On Thu, Oct 8, 2020 at 9:51 AM Maria Arias de Reyna Dominguez <
> [hidden email]> wrote:
>
>> Hi,
>>
>> In any case, maybe a nightly/weekly code coverage is useful to check
>> which parts of the code are less "tested" and we should put more
>> effort on them. Even if we can't do it by PR, it will show some light
>> on the current status of the code.
>>
>> On Thu, Oct 8, 2020 at 9:30 AM Andrea Cosentino <[hidden email]> wrote:
>>> I don't think it is feasible. Nobody would do it. It's time consuming.
>>>
>>> Il gio 8 ott 2020, 09:21 Djordje Bajić <[hidden email]> ha
>> scritto:
>>>> Hello Andrea, Jan,
>>>>
>>>> In that case, maybe PR reviewers can run tests locally on that branch
>> and
>>>> check? What do you guys think?
>>>>
>>>> - Djordje
>>>>
>>>> On Thu, 8 Oct 2020, 08:09 Andrea Cosentino, <[hidden email]> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> No, incremental build are not supported. The Camel build is too
>> complex
>>>> for
>>>>> that.
>>>>>
>>>>> Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <
>>>>> [hidden email]>
>>>>> ha scritto:
>>>>>
>>>>>> Hi Jan!
>>>>>>
>>>>>> Yes i understand that tests are gonna last long. Idk if there is
>>>>>> possibility to specify to run only tests for that particular
>> component
>>>> or
>>>>>> project inside the camel?
>>>>>>
>>>>>>
>>>>>> On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> It would be great IMO, but I think you need to actually run the
>> tests
>>>>>>> for coverage report. We currently skip tests for github PR,
>> because
>>>> it
>>>>>>> takes many hours to test whole codebase - these are running
>> during
>>>>>>> nightly build.
>>>>>>>
>>>>>>> Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
>>>>>>>> Hello fellow Cameleers!
>>>>>>>>
>>>>>>>> I am looking into twitter component, doing some small
>> refactoring.
>>>> I
>>>>>>>> noticed something interesting, code coverage is a little above
>> 50%,
>>>>> in
>>>>>> my
>>>>>>>> opinion that is a really poor %. What do you think that we add
>> some
>>>>>>> checks
>>>>>>>> or  when doing PR reviews to also check coverage of added code?
>>>> This
>>>>>> way
>>>>>>> we
>>>>>>>> will promote that tests  are mandatory in order to  approve
>> PR. Of
>>>>>> course
>>>>>>>> there will be some cases when tests are not available to be
>>>> written,
>>>>>>> anyway
>>>>>>>> i think this will help us reduce the number of bugs and give us
>>>>> freedom
>>>>>>> to
>>>>>>>> add, change and refactor with more confidence.
>>>>>>>>
>>>>>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Andrea Cosentino-3
It's all easy in words. The reality is just that we need incremental
builds, but the structure is too complex to be able to have them.

We can add test coverage but just as weekly or daily report.

Like jenkins build, except the usual maintainers, nobody will care.

Il gio 8 ott 2020, 10:38 Marc Carter <[hidden email]> ha scritto:

> It does feel like a failing. For exactly the reason below - smaller leaf
> components (of which there are many) and PRs (which are infinite into
> the future) "get away" with weaker testing because of the weight of
> historic coverage within the core elements. This is entropy at work and
> something a long-lived project might be bothered by.
>
> Have you tried using something like Sandboni to optimise the tests
> executed based on the git commits unique to the PR? Any enforced
> coverage percentage then becomes specific to the tests selected so
> avoids this situation.
>
> Marc
>
> https://github.com/jpmorganchase/sandboni-core
>
> On 08/10/2020 08:59, Omar Al-Safi wrote:
> > Hi,
> >
> > We have lgtm.com integrated which helps a bit to check from time to time
> > but not on every PR since the Camel build is complex. However, I think a
> > weekly coverage report is not a bad idea, at least it would maybe help a
> > bit.
> >
> > Regards,
> > Omar
> >
> > On Thu, Oct 8, 2020 at 9:51 AM Maria Arias de Reyna Dominguez <
> > [hidden email]> wrote:
> >
> >> Hi,
> >>
> >> In any case, maybe a nightly/weekly code coverage is useful to check
> >> which parts of the code are less "tested" and we should put more
> >> effort on them. Even if we can't do it by PR, it will show some light
> >> on the current status of the code.
> >>
> >> On Thu, Oct 8, 2020 at 9:30 AM Andrea Cosentino <[hidden email]>
> wrote:
> >>> I don't think it is feasible. Nobody would do it. It's time consuming.
> >>>
> >>> Il gio 8 ott 2020, 09:21 Djordje Bajić <[hidden email]> ha
> >> scritto:
> >>>> Hello Andrea, Jan,
> >>>>
> >>>> In that case, maybe PR reviewers can run tests locally on that branch
> >> and
> >>>> check? What do you guys think?
> >>>>
> >>>> - Djordje
> >>>>
> >>>> On Thu, 8 Oct 2020, 08:09 Andrea Cosentino, <[hidden email]>
> wrote:
> >>>>
> >>>>> Hello,
> >>>>>
> >>>>> No, incremental build are not supported. The Camel build is too
> >> complex
> >>>> for
> >>>>> that.
> >>>>>
> >>>>> Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <
> >>>>> [hidden email]>
> >>>>> ha scritto:
> >>>>>
> >>>>>> Hi Jan!
> >>>>>>
> >>>>>> Yes i understand that tests are gonna last long. Idk if there is
> >>>>>> possibility to specify to run only tests for that particular
> >> component
> >>>> or
> >>>>>> project inside the camel?
> >>>>>>
> >>>>>>
> >>>>>> On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>> It would be great IMO, but I think you need to actually run the
> >> tests
> >>>>>>> for coverage report. We currently skip tests for github PR,
> >> because
> >>>> it
> >>>>>>> takes many hours to test whole codebase - these are running
> >> during
> >>>>>>> nightly build.
> >>>>>>>
> >>>>>>> Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> >>>>>>>> Hello fellow Cameleers!
> >>>>>>>>
> >>>>>>>> I am looking into twitter component, doing some small
> >> refactoring.
> >>>> I
> >>>>>>>> noticed something interesting, code coverage is a little above
> >> 50%,
> >>>>> in
> >>>>>> my
> >>>>>>>> opinion that is a really poor %. What do you think that we add
> >> some
> >>>>>>> checks
> >>>>>>>> or  when doing PR reviews to also check coverage of added code?
> >>>> This
> >>>>>> way
> >>>>>>> we
> >>>>>>>> will promote that tests  are mandatory in order to  approve
> >> PR. Of
> >>>>>> course
> >>>>>>>> there will be some cases when tests are not available to be
> >>>> written,
> >>>>>>> anyway
> >>>>>>>> i think this will help us reduce the number of bugs and give us
> >>>>> freedom
> >>>>>>> to
> >>>>>>>> add, change and refactor with more confidence.
> >>>>>>>>
> >>>>>>>
> >>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

Otavio Rodolfo Piske
IMHO, I like the idea in principle, data is important and certainly can
help us target some areas where the coverage is low.

So, I think it would be useful to have the report ... but I believe making
it mandatory as part of PRs would be too soon.

Before making it mandatory, I think we need to adjust the build so it's
quicker and easier to run the tests, reduce the test effort by sharing more
testing code between the sub-projects and make sure the tests are solid.

On Thu, Oct 8, 2020 at 10:41 AM Andrea Cosentino <[hidden email]> wrote:

> It's all easy in words. The reality is just that we need incremental
> builds, but the structure is too complex to be able to have them.
>
> We can add test coverage but just as weekly or daily report.
>
> Like jenkins build, except the usual maintainers, nobody will care.
>
> Il gio 8 ott 2020, 10:38 Marc Carter <[hidden email]> ha scritto:
>
> > It does feel like a failing. For exactly the reason below - smaller leaf
> > components (of which there are many) and PRs (which are infinite into
> > the future) "get away" with weaker testing because of the weight of
> > historic coverage within the core elements. This is entropy at work and
> > something a long-lived project might be bothered by.
> >
> > Have you tried using something like Sandboni to optimise the tests
> > executed based on the git commits unique to the PR? Any enforced
> > coverage percentage then becomes specific to the tests selected so
> > avoids this situation.
> >
> > Marc
> >
> > https://github.com/jpmorganchase/sandboni-core
> >
> > On 08/10/2020 08:59, Omar Al-Safi wrote:
> > > Hi,
> > >
> > > We have lgtm.com integrated which helps a bit to check from time to
> time
> > > but not on every PR since the Camel build is complex. However, I think
> a
> > > weekly coverage report is not a bad idea, at least it would maybe help
> a
> > > bit.
> > >
> > > Regards,
> > > Omar
> > >
> > > On Thu, Oct 8, 2020 at 9:51 AM Maria Arias de Reyna Dominguez <
> > > [hidden email]> wrote:
> > >
> > >> Hi,
> > >>
> > >> In any case, maybe a nightly/weekly code coverage is useful to check
> > >> which parts of the code are less "tested" and we should put more
> > >> effort on them. Even if we can't do it by PR, it will show some light
> > >> on the current status of the code.
> > >>
> > >> On Thu, Oct 8, 2020 at 9:30 AM Andrea Cosentino <[hidden email]>
> > wrote:
> > >>> I don't think it is feasible. Nobody would do it. It's time
> consuming.
> > >>>
> > >>> Il gio 8 ott 2020, 09:21 Djordje Bajić <[hidden email]> ha
> > >> scritto:
> > >>>> Hello Andrea, Jan,
> > >>>>
> > >>>> In that case, maybe PR reviewers can run tests locally on that
> branch
> > >> and
> > >>>> check? What do you guys think?
> > >>>>
> > >>>> - Djordje
> > >>>>
> > >>>> On Thu, 8 Oct 2020, 08:09 Andrea Cosentino, <[hidden email]>
> > wrote:
> > >>>>
> > >>>>> Hello,
> > >>>>>
> > >>>>> No, incremental build are not supported. The Camel build is too
> > >> complex
> > >>>> for
> > >>>>> that.
> > >>>>>
> > >>>>> Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <
> > >>>>> [hidden email]>
> > >>>>> ha scritto:
> > >>>>>
> > >>>>>> Hi Jan!
> > >>>>>>
> > >>>>>> Yes i understand that tests are gonna last long. Idk if there is
> > >>>>>> possibility to specify to run only tests for that particular
> > >> component
> > >>>> or
> > >>>>>> project inside the camel?
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]> wrote:
> > >>>>>>
> > >>>>>>> Hi,
> > >>>>>>> It would be great IMO, but I think you need to actually run the
> > >> tests
> > >>>>>>> for coverage report. We currently skip tests for github PR,
> > >> because
> > >>>> it
> > >>>>>>> takes many hours to test whole codebase - these are running
> > >> during
> > >>>>>>> nightly build.
> > >>>>>>>
> > >>>>>>> Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> > >>>>>>>> Hello fellow Cameleers!
> > >>>>>>>>
> > >>>>>>>> I am looking into twitter component, doing some small
> > >> refactoring.
> > >>>> I
> > >>>>>>>> noticed something interesting, code coverage is a little above
> > >> 50%,
> > >>>>> in
> > >>>>>> my
> > >>>>>>>> opinion that is a really poor %. What do you think that we add
> > >> some
> > >>>>>>> checks
> > >>>>>>>> or  when doing PR reviews to also check coverage of added code?
> > >>>> This
> > >>>>>> way
> > >>>>>>> we
> > >>>>>>>> will promote that tests  are mandatory in order to  approve
> > >> PR. Of
> > >>>>>> course
> > >>>>>>>> there will be some cases when tests are not available to be
> > >>>> written,
> > >>>>>>> anyway
> > >>>>>>>> i think this will help us reduce the number of bugs and give us
> > >>>>> freedom
> > >>>>>>> to
> > >>>>>>>> add, change and refactor with more confidence.
> > >>>>>>>>
> > >>>>>>>
> > >>
> >
> >
>


--
Otavio R. Piske
http://orpiske.net
Reply | Threaded
Open this post in threaded view
|

Re: Code coverage

blanx
Hello all!

First off all thanks for such a great discussion. My main goal was to start
some conversation and to check if someone has some ideas about code
coverage and what would be next steps.

Mandatory check that I suggested is a shot in the dark right now, since
habits don't change that easily and Jenkins build is too complex (which i
didn't know), but some small step in that direction could improve the
current state.

Also it would be great to have some reports (daily, weekly, whatever), that
way, there may be some people who are willing to write tests for some
components/core..etc that are not covered well.
This could also enable some new contributors to get involved in development
much faster than the usual.

Just my two cents.

- Djordje





On Thu, Oct 8, 2020 at 7:55 PM Otavio Rodolfo Piske <[hidden email]>
wrote:

> IMHO, I like the idea in principle, data is important and certainly can
> help us target some areas where the coverage is low.
>
> So, I think it would be useful to have the report ... but I believe making
> it mandatory as part of PRs would be too soon.
>
> Before making it mandatory, I think we need to adjust the build so it's
> quicker and easier to run the tests, reduce the test effort by sharing more
> testing code between the sub-projects and make sure the tests are solid.
>
> On Thu, Oct 8, 2020 at 10:41 AM Andrea Cosentino <[hidden email]>
> wrote:
>
> > It's all easy in words. The reality is just that we need incremental
> > builds, but the structure is too complex to be able to have them.
> >
> > We can add test coverage but just as weekly or daily report.
> >
> > Like jenkins build, except the usual maintainers, nobody will care.
> >
> > Il gio 8 ott 2020, 10:38 Marc Carter <[hidden email]> ha scritto:
> >
> > > It does feel like a failing. For exactly the reason below - smaller
> leaf
> > > components (of which there are many) and PRs (which are infinite into
> > > the future) "get away" with weaker testing because of the weight of
> > > historic coverage within the core elements. This is entropy at work and
> > > something a long-lived project might be bothered by.
> > >
> > > Have you tried using something like Sandboni to optimise the tests
> > > executed based on the git commits unique to the PR? Any enforced
> > > coverage percentage then becomes specific to the tests selected so
> > > avoids this situation.
> > >
> > > Marc
> > >
> > > https://github.com/jpmorganchase/sandboni-core
> > >
> > > On 08/10/2020 08:59, Omar Al-Safi wrote:
> > > > Hi,
> > > >
> > > > We have lgtm.com integrated which helps a bit to check from time to
> > time
> > > > but not on every PR since the Camel build is complex. However, I
> think
> > a
> > > > weekly coverage report is not a bad idea, at least it would maybe
> help
> > a
> > > > bit.
> > > >
> > > > Regards,
> > > > Omar
> > > >
> > > > On Thu, Oct 8, 2020 at 9:51 AM Maria Arias de Reyna Dominguez <
> > > > [hidden email]> wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> In any case, maybe a nightly/weekly code coverage is useful to check
> > > >> which parts of the code are less "tested" and we should put more
> > > >> effort on them. Even if we can't do it by PR, it will show some
> light
> > > >> on the current status of the code.
> > > >>
> > > >> On Thu, Oct 8, 2020 at 9:30 AM Andrea Cosentino <[hidden email]>
> > > wrote:
> > > >>> I don't think it is feasible. Nobody would do it. It's time
> > consuming.
> > > >>>
> > > >>> Il gio 8 ott 2020, 09:21 Djordje Bajić <[hidden email]> ha
> > > >> scritto:
> > > >>>> Hello Andrea, Jan,
> > > >>>>
> > > >>>> In that case, maybe PR reviewers can run tests locally on that
> > branch
> > > >> and
> > > >>>> check? What do you guys think?
> > > >>>>
> > > >>>> - Djordje
> > > >>>>
> > > >>>> On Thu, 8 Oct 2020, 08:09 Andrea Cosentino, <[hidden email]>
> > > wrote:
> > > >>>>
> > > >>>>> Hello,
> > > >>>>>
> > > >>>>> No, incremental build are not supported. The Camel build is too
> > > >> complex
> > > >>>> for
> > > >>>>> that.
> > > >>>>>
> > > >>>>> Il giorno gio 8 ott 2020 alle ore 08:07 Djordje Bajić <
> > > >>>>> [hidden email]>
> > > >>>>> ha scritto:
> > > >>>>>
> > > >>>>>> Hi Jan!
> > > >>>>>>
> > > >>>>>> Yes i understand that tests are gonna last long. Idk if there is
> > > >>>>>> possibility to specify to run only tests for that particular
> > > >> component
> > > >>>> or
> > > >>>>>> project inside the camel?
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Wed, 7 Oct 2020, 21:09 Jan Bednář, <[hidden email]>
> wrote:
> > > >>>>>>
> > > >>>>>>> Hi,
> > > >>>>>>> It would be great IMO, but I think you need to actually run the
> > > >> tests
> > > >>>>>>> for coverage report. We currently skip tests for github PR,
> > > >> because
> > > >>>> it
> > > >>>>>>> takes many hours to test whole codebase - these are running
> > > >> during
> > > >>>>>>> nightly build.
> > > >>>>>>>
> > > >>>>>>> Dne 7.10.2020 v 15:08 Djordje Bajić napsal(a):
> > > >>>>>>>> Hello fellow Cameleers!
> > > >>>>>>>>
> > > >>>>>>>> I am looking into twitter component, doing some small
> > > >> refactoring.
> > > >>>> I
> > > >>>>>>>> noticed something interesting, code coverage is a little above
> > > >> 50%,
> > > >>>>> in
> > > >>>>>> my
> > > >>>>>>>> opinion that is a really poor %. What do you think that we add
> > > >> some
> > > >>>>>>> checks
> > > >>>>>>>> or  when doing PR reviews to also check coverage of added
> code?
> > > >>>> This
> > > >>>>>> way
> > > >>>>>>> we
> > > >>>>>>>> will promote that tests  are mandatory in order to  approve
> > > >> PR. Of
> > > >>>>>> course
> > > >>>>>>>> there will be some cases when tests are not available to be
> > > >>>> written,
> > > >>>>>>> anyway
> > > >>>>>>>> i think this will help us reduce the number of bugs and give
> us
> > > >>>>> freedom
> > > >>>>>>> to
> > > >>>>>>>> add, change and refactor with more confidence.
> > > >>>>>>>>
> > > >>>>>>>
> > > >>
> > >
> > >
> >
>
>
> --
> Otavio R. Piske
> http://orpiske.net
>


--
- Đorđe Bajić