[ocaml-platform] The OCaml Platform

[Anil Madhavapeddy]

The initial process is simple: satisfy the stakeholders who have
committed to wanting the Platform, and are putting resources into
making sure it happens. From the last Consortium meeting, this is
JSC, Citrix and Lexifi in the first instance, expanding to other
Consortium members as appropriate. OCamlPro also have their own
list of people who've expressed interest, of course, and are
heavily involved.

There will no doubt be various expanding circles of support from this
core, but we'll firm that up after we have a concrete thing to discuss.
The actual development will be as open as possible, but I'm keen to
avoid spending lots of time on the issues you describe below without a
baseline.

This isn't to say we don't want people to be involved; quite the
opposite. Until we have a first cut, simply contributing your packages
to the OPAM database will be very helpful to making sure that we have
a representative collection of OCaml packages from which to make
data-driven decisions.

-anil

On 13 Feb 2013, at 13:13, Gabriel Scherer <gabriel.scherer at gmail.com> wrote:

> Putting my "Community" and "Free Software" hats on, I wonder about
> what the contribution and management process for this "Platform" will
> look like. Who exactly is in charge of the project? Who owns the code
> it distributes, takes the responsability to react to issues, fix bugs
> and evolves the software?
> 
> Aiming at "coherence" means having a higher coupling between provided
> code than what exists in the community at large (where "community"
> here means the wide set of people that publicly releases FLOSS OCaml
> software). Is the Platform considered to be a single, unified software
> project, or a bundle of several independent software projects?
> 
> I know of several models to handle such sort of "starting component of
> a programming language ecosystem" (at various degrees of Scale):
> - single ownership: a single entity owns and maintains the code: this
> is the case of Core, maintained by Jane Street (with an internal
> development process), and a well-defined boundary between maintainers
> and external contributors (who sign a Contributor License Agreement);
> this is also the setting of the "standard library" of most programming
> languages (say Scala, Python, or most languages with a distinguished
> canonical implementation; also the case of OCaml, but the scope of the
> stdlib is much narrower in this case)
> - multiple ownership: it is a bundle of individually-owned packages,
> the platform mostly choosing to distribute them or not and providing
> quality control; my understanding is that this is typically the model
> of the Perl CPAN, Haskell's Hackage, etc.; these projects usually have
> a wider scope and weaker consistency goals than what seems aimed at
> here
> - shared ownership: the code ownership is loosely shared between the
> various contributors to the project; this is the case of the Linux
> kernel, the standard libraries of some languages with strong
> contribution dynamics (possibly GHC's library), Batteries, and most
> projects that use a Wiki model to build code (eg. Rosetta Code)
> 
> What kind of contribution dynamics are you aiming for? When the works
> starts to take a shape that is visible to the outside, should I start
> telling people "if you have loose time, please consider contributing
> to the Platform"?
> 
> On Tue, Feb 12, 2013 at 7:01 PM, Sylvain Le Gall
> <sylvain+ocaml at le-gall.net> wrote:
>> 2013/2/12 Anil Madhavapeddy <anil at recoil.org>:
>>> On 12 Feb 2013, at 15:23, Sylvain Le Gall <sylvain+ocaml at le-gall.net> wrote:
>>>> 
>>>> In the case of Lwt, the solution is pretty straight forward, you just
>>>> use a thread (Thread) to wrap 1. and produce 2.
>>>> 
>>>> I think the overall question is "do we wish to have a common style for
>>>> non-stdlib features", e.g. monads, file descriptor et al. I don't
>>>> think this is mandatory but can be a nice long term goal.
>>>> 
>>>> But maybe I am going too far, and the point of OCaml Platform is just
>>>> to have  a "set of libraries", tested, packaged together and released
>>>> every 6 months (which is already an ambitious goal). If this is the
>>>> case, please ignore my question -- I'll be already extermly happy to
>>>> see an OCaml Platform in 6 months.
>>> 
>>> You raise a good question, but there's no easy answer without basics
>>> in place first. After a discussion with JSC/Citrix folks, the very first
>>> focus of the platform is on filling in the blanks on some basic things:
>>> 
>>> - unified documentation: a cross-referenced documentation set across
>>> multiple libraries, that can be output in different formats. Leo has a
>>> prototype for this based on the new bin-annot files, and will release
>>> it shortly.
>> 
>> Please coordinate with OASIS project, if we need to generate this
>> bin-annot file, this is just a matter of changing 2 options and re-run
>> "oasis setup" before building.
>> 
>>> 
>>> - test harness: something that combines unit tests (oUnit) with coverage
>>> and fuzz tests (QuickCheck), and benchmarks (like Criterion in Haskell).
>>> This is something we hope to work with Xavier Clerc on, as he has made
>>> some excellent contributions here.  The test harness should also encode
>>> dependencies on an environment, so that we can run them automatically
>>> in a virtual environment (absolutely required by Citrix, who have an
>>> incredible test suite for Xen that they want to extend to the OCaml code).
>>> 
>> 
>> Quick note on this point (oUnit):
>> I am currently working on OUnit 2.0, which will feature:
>> - HTML output of tests
>> - JUnit output of tests
>> - parrallel run of tests (up to your number of core)
>> - timing...
>> 
>> One of the goal is to integrate well with Jenkins, so that you can
>> seee the reason why your test is failing directly in the artifact
>> generated by a test run.
>> 
>>> - build: most of the existing build solutions only scale up to small
>>> code bases, so investigating a more unified approach based on compiler-libs
>>> to write an integrated compiler front end that can auto-download
>>> dependencies at the configuration phase, do smart incremental build,
>>> and hook in the test frameworks.
>>> 
>> 
>> I would dare to say on this point, that you should focus on "breaking
>> the longest path". This is an issue we have dealt with since a number
>> of years in Debian. (each new version of OCaml require to recompile
>> all packages on all architecture of debian).
>> 
>> Have a look at that:
>> http://pkg-ocaml-maint.alioth.debian.org/build_order.txt
>> 
>> Each package of a level can be build in parallel. So the compilation
>> time is roughly the sum of the longest package to build in each level.
>> 
>> The first level is "ocaml', which is among one of the longest step. I
>> would say that the full set can be build in ~2h on a recent computer.
>> 
>> It is pretty straightforward to write a makefile that will do all this
>> compilation for you (and it will take you ~30 min to do so). I would
>> recommed you to start by this very simple solution, that will probably
>> scale to ~100 packages with no problem.
>> 
>> Knowing the time it takes to compile Haskell or C++, I would say that
>> we are already pretty good at that -- no need to put it as a top
>> priority.
>> 
>>> This is all tooling, and can be done independently of the process of
>>> selecting libraries.  For that selection, we need to think of this a bit
>>> like a microkernel: pick the absolute smallest set of libraries we need
>>> (including stripping down the OCaml standard library), and then working
>>> our way up from there into a usable set of package.  Along the way, we'll
>>> need to define 'platform tiers' to determine which ones absolutely have
>>> to supported.
>>> 
>>> An important note is that there'll be a reward for adapting your code
>>> to run on the Platform.  The test system will regularly run *your* unit
>>> tests on all the support platforms and send you reports.  Don't have a
>>> FreeBSD/mips box, or a Raspberry Pi?  Opamalot will take care of that
>>> for you and let you know if it comes and passes tests.
>>> 
>>> Amir and Leo are currently writing up the results of the various recent
>>> meetings and will publish it soon.
>>> 
>>> -anil
>> _______________________________________________
>> Platform mailing list
>> Platform at lists.ocaml.org
>> http://lists.ocaml.org/listinfo/platform
>