[ocaml-platform] OCamlot

[Sylvain Le Gall]

2013/2/13 Anil Madhavapeddy <avsm2 at cl.cam.ac.uk>:
> On 13 Feb 2013, at 22:42, Anil Madhavapeddy <avsm2 at cl.cam.ac.uk> wrote:
>
>> Opamalot is more of a coordination service.  It interfaces with OPAM to get the (non-trivial) package database, version constraints and compiler variants out of it, and comes up with a prioritised schedule of builds and tests that need to be run on different platforms.  Some of these could (and should) run on third-party hosted services such as Travis, just to get some diversity.  Others, however, really require a Xen pool so that more exotic stuff can be done.
>
> It's also worth noting the difference in execution speeds for adopting a copy-on-write approach.  We  need to test 300+ packages (which will grow) across 3.12.1/4.00.0/4.00.1/4.01.0dev and experimental compiler branches, ideally without recompiling the compiler for each package.  If a package fails to build on a fast architecture (x86), it should immediately be dropped from a slow one.

Humm, to be honest, I will:
1. limit the number of compiler version (4.00.1 or 4.01.0 when it will be out)
2. limit the number of packages

I think 300+ packages in OPAM term is the number of tarball. Let's
just take either the last one or the last one that we were to build on
all platform. I don't want to have OUnit 1.1.0 and 2.0.0...

Concerning Travis/Jenkins/whatever, I think we should consider this:
- just create a shell script that do the job and don't interface
strongly with the build system in the core job
- most of the builder I know, can have some restriction, esp
concerning accessing the network. This is not always true but we
should try to avoid downloading stuff
- create a big git repository that basically contains an opam
repository + the builder shell script and do the job inside it
- try to output the results of tests/build in a "common test result
format" (either TAP or JUnit)
- consider that most platform don't allow you to install stuff that
are 3rd party (e.g. C libraries that comes with the OS, like libzmq0
or libfoo.so.XX), I don't think we should build them on our own (let's
focus on OCaml)

I think this can be done independently of the platform we use for
continuous integration, when we will have gained some experience, we
can enhance the process for a specific/preferred platform.

I start to play quite nicely with Jenkins and I think you don't have
to build any plugin to make it works:
- use a matrix project to build on all platform, have a first step on
1 particular platform that need to succeed (integrated, I use it)
- use a shell script to start the build, no maven, no ant (I use it)
- ...

I will look at memory issue with java worker... Let's say I have a
couple of Java specialist sitting next to me. I will need to run one
on my RPi...

Regards
Sylvain