[opam-devel] conf- libraries and stub cmas: time for ocaml-discover?

[Anil Madhavapeddy]

I wanted to follow up on this ticket in OPAM-repository with a slightly wider audience, as it affects a lot of the libraries in the tree.
https://github.com/OCamlPro/opam-repository/pull/1008#issuecomment-23250671

Christophe makes the suggestion of having the stub packages for conf-* libraries expose their cflags/lflags in a standard-ish way that can be used by the OCaml package.

We have a similar problem with an upcoming version of the ctypes library that supports linking to C libraries in several ways:

- Dynamically opening the symbols via dlopen(3) and using libffi to call functions.  No C stubs needed.

- C stub generation that outputs C code that is then compiled and linked in as usual.

- Struct "offset probing" that determines precisely which offset a field name maps to by generating enough C code to compile a test. [1]

Ideally, we would be able to query three types of information from a conf package installation:

- an ML file that has values exposing cflags/lflags as OCaml values.  This can be used by discover.ml scripts (as used in Core or Lwt)

- a CMA file that embeds the dynamic linking flags via cclib, including adding -Wl,--no-as-needed on newer systems like Ubuntu.  This lets a Ctypes using dynamic linking (Ctypes.Foreign) just add a CMA to the link line and have the correct gcc/clang libraries work.

- The final thing is some way to pass the right CFLAGS/LDFLAGS to compile up generated C stubs with the right include/library paths to gcc (either from Ctypes, or via the traditional camlidl/swig outputs).  Presumably this would be a raw shell fragment in a file, as usually output by autoconf.

Overall though, it does seem time to come up with a concrete proposal for how to handle system-specific C library configuration.  If we get this right, it should significantly simplify win32 and cross-compilation too.

I'm thinking of an ocaml-discover package that can encapsulate all this stuff and provide a single place for all this stub logic.  Anyone else have any thoughts that I've missed out above?

[1] https://github.com/ocamllabs/ocaml-ctypes/issues/55