[opam-devel] mirage, opam and cross-compiling... ?

[Anil Madhavapeddy]

On 18 Sep 2013, at 12:44, Christopher Greenhalgh <Chris.Greenhalgh at nottingham.ac.uk> wrote:

> Is there something that passes for best practice when it comes to cross-compiling, ocaml and opam? 
> 

[I'm CCing opam-devel on this question, as it's pertinent to ongoing work in the trunk of OCaml]

> I've spent the morning looking but ocaml cross-compiling seems to be rather ad hoc at the moment. And the approach I've taken so far following Vouillon with separate target-specific opam packages doesn't seem entirely satisfactory...
> 
> it would seem "better" to be able to switch to a cross-compiler and build standard (or at least cross-compilation-compatible) packages auto-magically. And it would be super-nice if this didn't require any changes to "simple" packages, which in turn implies that cross-compiling tools are the default and the cross-compiler's ocamlbuild is wise to cross-compiling. Other build tools (esp. oasis) could then also be extended to turn out sensible defaults when cross-compiling (e.g. using BuildTools fields).

Right -- the problem with cross-compiling OCaml historically has been a few points:

- the OCaml toolchain itself didn't support building cross-tools
- there's no standard for specifying an equivalent to CFLAGS/CC in OCaml build systems -- everyone defines their own.

The former is being solved in 4.2 trunk now (I believe not yet fully cross-arch, but certainly cross-os).  The latter is why an Android-specific OPAM target is what the existing repository does.

However, I think we're in much better to solve this properly now with compiler switches, but it will require doing two things:

- define an OPAM switch that generates the cross-libraries.  This switch will be incapable of executing the binaries it produces, so it's probably best to generate a host toolchain *and* a separate cross toolchain with a different prefix.

- add patches to existing build systems to use the cross tools (such as OASIS, as you mention).  This still requires an overlay patchset in OPAM, but it's far easier to manage and feed upstream that a set of ad-hoc patches.

- the annoying ones are usually syntax extensions, which need to be built as host libraries and not target libraries.  We have a way to sidestep this as of a few weeks ago.  Leo White has built a specialized compiler driver that bypasses the usual compiler parse tree, and instead statically links in some camlp4 extensions.  This is not only faster, but it avoids dynamic linking in the compiler completely!  Most of our syntax extension requirements are actually pretty minimal (the Lwt one, and perhaps Sexplib and Cow), so we could just link those into one mega-compiler-driver that works on all the source code that we care about.

https://github.com/lpw25/ocaml-with-pp

I'm playing with this at the moment in order to build a js_of_ocaml toplevel for the Real World OCaml website, so let me know how you get along with it.

> I added some notes to the end of my mirage-on-android working notes, https://github.com/cgreenhalgh/opensharingtoolkit/blob/master/docs/mirageonandroid.md
> 
> But I'm wondering if this is already in hand somewhere or better tackled otherwise?

Your notes are incredibly useful -- I think it's time to kickstart some of the cross-compilation work now that the OCaml 4.01 release is out of the way.  This also gives us a useful direction in OPAM development in terms of build system requirements (if you can cross-architecture compile it, then it'll probably work anywhere@)

-anil