Daily build to test with JDK 9 and 10

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Daily build to test with JDK 9 and 10

Zoran Regvart-2
Hello Cameleers,
I've created a build that will run once every day to run tests on JDK
9 and 10[1], the configuration for the build is in
`Jenkinsfile.daily`[2].

Suggestions and contributions are welcome :)

zoran

[1] https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/
[2] https://gitbox.apache.org/repos/asf?p=camel.git;a=blob;f=Jenkinsfile.daily;hb=HEAD
--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: Daily build to test with JDK 9 and 10

jbonofre
Hi Zoran,

it sounds good to me.

Thanks !

Regards
JB

On 09/05/2018 11:45, Zoran Regvart wrote:

> Hello Cameleers,
> I've created a build that will run once every day to run tests on JDK
> 9 and 10[1], the configuration for the build is in
> `Jenkinsfile.daily`[2].
>
> Suggestions and contributions are welcome :)
>
> zoran
>
> [1] https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/
> [2] https://gitbox.apache.org/repos/asf?p=camel.git;a=blob;f=Jenkinsfile.daily;hb=HEAD
>
Reply | Threaded
Open this post in threaded view
|

Re: Daily build to test with JDK 9 and 10

Zoran Regvart-2
In reply to this post by Zoran Regvart-2
Hi Cameleers,
so it's not that bad[1] 77 tests failed in total. I think we need to
play with opening up some modules a bit. I'm a bit sad that it takes 9
hours for the job to run, when we add JDK 11 I would expect it to take
13 hours, would be nice if there was a way to just run the tests in
parallel on multiple JDKs.

zoran

[1] https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/lastCompletedBuild/testReport/

On Wed, May 9, 2018 at 11:45 AM, Zoran Regvart <[hidden email]> wrote:

> Hello Cameleers,
> I've created a build that will run once every day to run tests on JDK
> 9 and 10[1], the configuration for the build is in
> `Jenkinsfile.daily`[2].
>
> Suggestions and contributions are welcome :)
>
> zoran
>
> [1] https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/
> [2] https://gitbox.apache.org/repos/asf?p=camel.git;a=blob;f=Jenkinsfile.daily;hb=HEAD
> --
> Zoran Regvart



--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: Daily build to test with JDK 9 and 10

Willem.Jiang
Administrator
Not sure if we can leverage travis to do this kind of work.
As camel build is a little bit longer one by one, if we can run the
component tests in parallel that you could save us lot of time.


Willem Jiang

Blog: http://willemjiang.blogspot.com (English)
          http://jnn.iteye.com  (Chinese)
Twitter: willemjiang
Weibo: 姜宁willem

On Thu, May 10, 2018 at 6:15 AM, Zoran Regvart <[hidden email]> wrote:

> Hi Cameleers,
> so it's not that bad[1] 77 tests failed in total. I think we need to
> play with opening up some modules a bit. I'm a bit sad that it takes 9
> hours for the job to run, when we add JDK 11 I would expect it to take
> 13 hours, would be nice if there was a way to just run the tests in
> parallel on multiple JDKs.
>
> zoran
>
> [1] https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/
> lastCompletedBuild/testReport/
>
> On Wed, May 9, 2018 at 11:45 AM, Zoran Regvart <[hidden email]> wrote:
> > Hello Cameleers,
> > I've created a build that will run once every day to run tests on JDK
> > 9 and 10[1], the configuration for the build is in
> > `Jenkinsfile.daily`[2].
> >
> > Suggestions and contributions are welcome :)
> >
> > zoran
> >
> > [1] https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/
> > [2] https://gitbox.apache.org/repos/asf?p=camel.git;a=blob;
> f=Jenkinsfile.daily;hb=HEAD
> > --
> > Zoran Regvart
>
>
>
> --
> Zoran Regvart
>
Reply | Threaded
Open this post in threaded view
|

Re: Daily build to test with JDK 9 and 10

Pascal Schumacher
In reply to this post by Zoran Regvart-2
It is great to have a build for JDK 9 and 10!

(Next step is JDK 11, which is already available according to
https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix 
:))

Making the job executing the test for JDK 9 and JDK 10 in parallel was easy:

https://github.com/apache/camel/commit/1c26b28145553e8db45f5b5412903d16ef1f6948

The job is still running
(https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/17/flowGraphTable/),
but peeking into the logout I guess this easy solution will not work.
Both stages are executed on the same node and are therefore using the
same file system which causes problems/conflicts with e.g. Apache Derby.

- Pascal

Am 10.05.2018 um 00:15 schrieb Zoran Regvart:

> Hi Cameleers,
> so it's not that bad[1] 77 tests failed in total. I think we need to
> play with opening up some modules a bit. I'm a bit sad that it takes 9
> hours for the job to run, when we add JDK 11 I would expect it to take
> 13 hours, would be nice if there was a way to just run the tests in
> parallel on multiple JDKs.
>
> zoran
>
> [1] https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/lastCompletedBuild/testReport/

Reply | Threaded
Open this post in threaded view
|

Re: Daily build to test with JDK 9 and 10

Zoran Regvart-2
Hi Pascal,
thank you for looking into this. I don't think that using parallel
will help us, here's why: we're building on JDK 8 and we need to use
binaries built on JDK 8. Running same tests in parallel will cause a
lot of flakiness as they might use the same network ports or touch the
same files. So that leaves us with copying the build workspace for
each JDK we want to additionally test before the parallel step -- and
that would be a lot of IO. That's why I mentioned a copy-on-write
filesystem. With something like that, and Luca mentioned using docker,
we could cheaply fork the file system to run the two stages in
parallel.

Hope that makes sense...

zoran

On Thu, May 10, 2018 at 4:10 PM, Pascal Schumacher
<[hidden email]> wrote:

> It is great to have a build for JDK 9 and 10!
>
> (Next step is JDK 11, which is already available according to
> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
> :))
>
> Making the job executing the test for JDK 9 and JDK 10 in parallel was easy:
>
> https://github.com/apache/camel/commit/1c26b28145553e8db45f5b5412903d16ef1f6948
>
> The job is still running
> (https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/17/flowGraphTable/),
> but peeking into the logout I guess this easy solution will not work. Both
> stages are executed on the same node and are therefore using the same file
> system which causes problems/conflicts with e.g. Apache Derby.
>
> - Pascal
>
>
> Am 10.05.2018 um 00:15 schrieb Zoran Regvart:
>>
>> Hi Cameleers,
>> so it's not that bad[1] 77 tests failed in total. I think we need to
>> play with opening up some modules a bit. I'm a bit sad that it takes 9
>> hours for the job to run, when we add JDK 11 I would expect it to take
>> 13 hours, would be nice if there was a way to just run the tests in
>> parallel on multiple JDKs.
>>
>> zoran
>>
>> [1]
>> https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/lastCompletedBuild/testReport/
>
>



--
Zoran Regvart
Reply | Threaded
Open this post in threaded view
|

Re: Daily build to test with JDK 9 and 10

Pascal Schumacher
Hi Zoran,

it is exactly as you said. Just parallelizing the test part of the build
did not work for the reasons you mentioned. Therefore I have reverted
the change.

- Pascal

Am 11.05.2018 um 00:05 schrieb Zoran Regvart:

> Hi Pascal,
> thank you for looking into this. I don't think that using parallel
> will help us, here's why: we're building on JDK 8 and we need to use
> binaries built on JDK 8. Running same tests in parallel will cause a
> lot of flakiness as they might use the same network ports or touch the
> same files. So that leaves us with copying the build workspace for
> each JDK we want to additionally test before the parallel step -- and
> that would be a lot of IO. That's why I mentioned a copy-on-write
> filesystem. With something like that, and Luca mentioned using docker,
> we could cheaply fork the file system to run the two stages in
> parallel.
>
> Hope that makes sense...
>
> zoran
>
> On Thu, May 10, 2018 at 4:10 PM, Pascal Schumacher
> <[hidden email]> wrote:
>> It is great to have a build for JDK 9 and 10!
>>
>> (Next step is JDK 11, which is already available according to
>> https://cwiki.apache.org/confluence/display/INFRA/JDK+Installation+Matrix
>> :))
>>
>> Making the job executing the test for JDK 9 and JDK 10 in parallel was easy:
>>
>> https://github.com/apache/camel/commit/1c26b28145553e8db45f5b5412903d16ef1f6948
>>
>> The job is still running
>> (https://builds.apache.org/view/A-D/view/Camel/job/Camel.daily/17/flowGraphTable/),
>> but peeking into the logout I guess this easy solution will not work. Both
>> stages are executed on the same node and are therefore using the same file
>> system which causes problems/conflicts with e.g. Apache Derby.
>>
>> - Pascal
>>
>>
>> Am 10.05.2018 um 00:15 schrieb Zoran Regvart:
>>> Hi Cameleers,
>>> so it's not that bad[1] 77 tests failed in total. I think we need to
>>> play with opening up some modules a bit. I'm a bit sad that it takes 9
>>> hours for the job to run, when we add JDK 11 I would expect it to take
>>> 13 hours, would be nice if there was a way to just run the tests in
>>> parallel on multiple JDKs.
>>>
>>> zoran
>>>
>>> [1]
>>> https://builds.apache.org/view/C/view/Apache%20Camel/job/Camel.daily/lastCompletedBuild/testReport/
>>
>
>