[opam-devel] conf- libraries and stub cmas: time for ocaml-discover?

[Jonathan Protzenko]

Hi,

Thanks for looping me in. I never really thought about this, since it's 
pretty far from my usual concerns :-), but that sounds like a fairly 
sensible proposal.

Some questions.
1) Whose responsibility is it to check that the very basic requirements 
are met? (e.g. presence of a gcc-like executable in the PATH). On 
Windows, I'm still making the install of Cygwin as optional, so someone 
would have to check first that we have a toolchain, and provide an 
early, meaningful error message. (I could think of a similar situation 
on Linux, where OPAM is installed via the ppa and somehow ocaml-ffi is 
a binary package or its compilation doesn't need a C compiler.)

I feel like having an ML file that has _values_ exposing cflags/lflags 
precludes an early reporting of errors (“I don't have a compiler in my 
PATH, don't even bother trying to call it!”).

2) Also, beginners on, say, Ubuntu, would have to install the 
build-essential package. Do you have plans for a platform-specific 
advice on how to install proper development packages?

3) I'm not sure about that, but do Lwt / ocaml-ffi use flexdll on 
Windows?

4) I also think that you need to make the gcc name configurable (but 
maybe you mentioned that already). On Windows, it's called 
i686-whatever-gcc, or something.

5) Do you plan to have OCaml distribute this package as part of 
compiler-libs (sounds easier, but possibly a fair bit of discussion 
needed), or rather have the packager distribute it?

Cheers,

~ jonathan

On Mon 26 Aug 2013 12:57:40 PM CEST, Anil Madhavapeddy wrote:
> I wanted to follow up on this ticket in OPAM-repository with a slightly wider audience, as it affects a lot of the libraries in the tree.
> https://github.com/OCamlPro/opam-repository/pull/1008#issuecomment-23250671
>
> Christophe makes the suggestion of having the stub packages for conf-* libraries expose their cflags/lflags in a standard-ish way that can be used by the OCaml package.
>
> We have a similar problem with an upcoming version of the ctypes library that supports linking to C libraries in several ways:
>
> - Dynamically opening the symbols via dlopen(3) and using libffi to call functions.  No C stubs needed.
>
> - C stub generation that outputs C code that is then compiled and linked in as usual.
>
> - Struct "offset probing" that determines precisely which offset a field name maps to by generating enough C code to compile a test. [1]
>
> Ideally, we would be able to query three types of information from a conf package installation:
>
> - an ML file that has values exposing cflags/lflags as OCaml values.  This can be used by discover.ml scripts (as used in Core or Lwt)
>
> - a CMA file that embeds the dynamic linking flags via cclib, including adding -Wl,--no-as-needed on newer systems like Ubuntu.  This lets a Ctypes using dynamic linking (Ctypes.Foreign) just add a CMA to the link line and have the correct gcc/clang libraries work.
>
> - The final thing is some way to pass the right CFLAGS/LDFLAGS to compile up generated C stubs with the right include/library paths to gcc (either from Ctypes, or via the traditional camlidl/swig outputs).  Presumably this would be a raw shell fragment in a file, as usually output by autoconf.
>
> Overall though, it does seem time to come up with a concrete proposal for how to handle system-specific C library configuration.  If we get this right, it should significantly simplify win32 and cross-compilation too.
>
> I'm thinking of an ocaml-discover package that can encapsulate all this stuff and provide a single place for all this stub logic.  Anyone else have any thoughts that I've missed out above?
>
> [1] https://github.com/ocamllabs/ocaml-ctypes/issues/55