[ocaml-platform] The OCaml Platform

[Sylvain Le Gall]

2013/2/12 Anil Madhavapeddy <anil at recoil.org>:
> On 12 Feb 2013, at 15:23, Sylvain Le Gall <sylvain+ocaml at le-gall.net> wrote:
>>
>> In the case of Lwt, the solution is pretty straight forward, you just
>> use a thread (Thread) to wrap 1. and produce 2.
>>
>> I think the overall question is "do we wish to have a common style for
>> non-stdlib features", e.g. monads, file descriptor et al. I don't
>> think this is mandatory but can be a nice long term goal.
>>
>> But maybe I am going too far, and the point of OCaml Platform is just
>> to have  a "set of libraries", tested, packaged together and released
>> every 6 months (which is already an ambitious goal). If this is the
>> case, please ignore my question -- I'll be already extermly happy to
>> see an OCaml Platform in 6 months.
>
> You raise a good question, but there's no easy answer without basics
> in place first. After a discussion with JSC/Citrix folks, the very first
> focus of the platform is on filling in the blanks on some basic things:
>
> - unified documentation: a cross-referenced documentation set across
> multiple libraries, that can be output in different formats. Leo has a
> prototype for this based on the new bin-annot files, and will release
> it shortly.

Please coordinate with OASIS project, if we need to generate this
bin-annot file, this is just a matter of changing 2 options and re-run
"oasis setup" before building.

>
> - test harness: something that combines unit tests (oUnit) with coverage
> and fuzz tests (QuickCheck), and benchmarks (like Criterion in Haskell).
> This is something we hope to work with Xavier Clerc on, as he has made
> some excellent contributions here.  The test harness should also encode
> dependencies on an environment, so that we can run them automatically
> in a virtual environment (absolutely required by Citrix, who have an
> incredible test suite for Xen that they want to extend to the OCaml code).
>

Quick note on this point (oUnit):
I am currently working on OUnit 2.0, which will feature:
- HTML output of tests
- JUnit output of tests
- parrallel run of tests (up to your number of core)
- timing...

One of the goal is to integrate well with Jenkins, so that you can
seee the reason why your test is failing directly in the artifact
generated by a test run.

> - build: most of the existing build solutions only scale up to small
> code bases, so investigating a more unified approach based on compiler-libs
> to write an integrated compiler front end that can auto-download
> dependencies at the configuration phase, do smart incremental build,
> and hook in the test frameworks.
>

I would dare to say on this point, that you should focus on "breaking
the longest path". This is an issue we have dealt with since a number
of years in Debian. (each new version of OCaml require to recompile
all packages on all architecture of debian).

Have a look at that:
http://pkg-ocaml-maint.alioth.debian.org/build_order.txt

Each package of a level can be build in parallel. So the compilation
time is roughly the sum of the longest package to build in each level.

The first level is "ocaml', which is among one of the longest step. I
would say that the full set can be build in ~2h on a recent computer.

It is pretty straightforward to write a makefile that will do all this
compilation for you (and it will take you ~30 min to do so). I would
recommed you to start by this very simple solution, that will probably
scale to ~100 packages with no problem.

Knowing the time it takes to compile Haskell or C++, I would say that
we are already pretty good at that -- no need to put it as a top
priority.

> This is all tooling, and can be done independently of the process of
> selecting libraries.  For that selection, we need to think of this a bit
> like a microkernel: pick the absolute smallest set of libraries we need
> (including stripping down the OCaml standard library), and then working
> our way up from there into a usable set of package.  Along the way, we'll
> need to define 'platform tiers' to determine which ones absolutely have
> to supported.
>
> An important note is that there'll be a reward for adapting your code
> to run on the Platform.  The test system will regularly run *your* unit
> tests on all the support platforms and send you reports.  Don't have a
> FreeBSD/mips box, or a Raspberry Pi?  Opamalot will take care of that
> for you and let you know if it comes and passes tests.
>
> Amir and Leo are currently writing up the results of the various recent
> meetings and will publish it soon.
>
> -anil