[opam-devel] OPAM + Docker

[Sylvain Le Gall]

2013/11/19 Nicolas Braud-Santoni <nicolas at braud-santoni.eu>:
> On 19/11/2013 05:03, ygrek wrote:
>> On Mon, 18 Nov 2013 10:43:59 -0800
>> Anil Madhavapeddy <anil at recoil.org> wrote:
>>
>>> I've made some notes about using OPAM and Docker for easier local bulk builds here:
>>> http://anil.recoil.org/2013/11/15/docker-and-opam.html
>>>
>>> One semi-interesting issue I've already discovered with depexts is that just a "ubuntu" tag isn't enough, since  some depexts are only valid in a specific version of Ubuntu.
>>>
>>> This unfortunately feels like it might be a job for the constraint system and not for flat tags (since I want to say >precise or <saucy).  We could do it with flat tags for now, but I have a bad feeling that it'll get unmanageable quite quickly..
> Two remarks:
> 1. Using the version numbers might be much less confusing than the codenames
> 2. I agree that we need to agree on useable syntax (and semantics) for
> depexts. See https://github.com/ocaml/opam/issues/991
>
>> I personally dislike this depexts field to mention OS packages at all.
> Why ?
> This seems to be the most sane and generic option if we wish to interact
> with the system's package manager [0].

The problem is that each time you want to extend to a new OS
distribution, you may need to do a lot of work. Not to mention that
this work will probably need to be done when upgrading as well,
although it may be less expensive. On the long term you may need to
dedicate a fair amount of time dealing with updating the depext. But
that is my rough estimate, and I do prefer to see people try it than
not do anything about that (but we should be open and consider
dropping this if it turns to be a not that sustainable solution).

>
>> Maybe it's better to specify which external libraries are needed (as .so or as pkg-config files) and then
>> use native OS way (at installation time) to determine which package contains that .so or .pc file (e.g. apt-file on Debian).
> Not all package managers provide these kind of facilities.
> That is someting that, for instance, will not exist in any source-based
> distribution that I am aware of.
>
> Moreover, it seems unwise to fill in information that ought to belong in
> the distribution's packages :
> if I depend on libfoo, major version 1, any minor version, should
> I specify “libfoo-1.so”, “libfoo-1(.\d+)*.so” (dots implicitly
> escaped), or some crazier scheme ?

Not really crazy when you think about it but you should probably not
depends on the .so. You just focus on what you need extracting data
from the package:
- scan .c -> extract #include "X.h", it defines a dependencies
- scan build system -> --libs X or -lX -> you need libX.a

Most of the package manager allow you to do file lookup: e.g. "dpkg
-S" on Debian, example:
$ dpkg -S libpcre.a
libpcre3-dev: /usr/lib/i386-linux-gnu/libpcre.a

Lets take a real example:
$ cd ocaml-zmq
$ grep -r -e '-l.*' .
./_oasis:  CCLib:          -lzmq
./myocamlbuild.ml:          (* When one link an OCaml
library/binary/package, one should use -linkpkg *)
./myocamlbuild.ml:          flag ["ocaml"; "link"; "program"] & A"-linkpkg";
./myocamlbuild.ml:           * options when using this tag. When using
the "-linkpkg" option with
./myocamlbuild.ml:                     (S[A"-dllib"; A("-l"^lib);
A"-cclib"; A("-l"^lib)]);
./myocamlbuild.ml:                     (S[A"-cclib"; A("-l"^lib)]);
./myocamlbuild.ml:            [(OASISExpr.EBool true, S [A "-cclib"; A
"-lzmq"])]);
./myocamlbuild.ml:            [(OASISExpr.EBool true, S [A "-lzmq"])])
./setup.ml:                   don't use the right command-line
arguments, rerun \
./setup.ml:            "-no-log";
./setup.ml:            "-no-links";
./setup.ml:            "-install-lib-dir";
./setup.ml:                      bs_cclib = [(OASISExpr.EBool true, ["-lzmq"])]

$ grep -r -e '#include <.*>' .
./src/caml_zmq_stubs.c:#include <stdlib.h>
./src/caml_zmq_stubs.c:#include <stdint.h>
./src/caml_zmq_stubs.c:#include <string.h>
./src/caml_zmq_stubs.c:#include <stdio.h>
./src/caml_zmq_stubs.c:#include <stdbool.h>
./src/caml_zmq_stubs.c:#include <caml/mlvalues.h>
./src/caml_zmq_stubs.c:#include <caml/alloc.h>
./src/caml_zmq_stubs.c:#include <caml/fail.h>
./src/caml_zmq_stubs.c:#include <caml/callback.h>
./src/caml_zmq_stubs.c:#include <caml/memory.h>
./src/caml_zmq_stubs.c:#include <caml/custom.h>
./src/caml_zmq_stubs.c:#include <caml/intext.h>
./src/caml_zmq_stubs.c:#include <caml/threads.h>
./src/caml_zmq_stubs.c:#include <zmq.h>

$ dpkg -S libzmq.a
libzmq-dev:i386: /usr/lib/i386-linux-gnu/libzmq.a

$ dpkg -S zmq.h
libzmq-dev:i386: /usr/include/zmq.hpp
libzmq-dev:i386: /usr/include/zmq.h

You don't even have to do crazy comlpex thing here (a simple grep in
the source will already provide you good insight). Although, you may
replace dpkg by apt-file, because dpkg only rely on files installed.

The first step (extract information with grep), is distribution
agnostic and the second steps need to be translated once per OS
distribution. If you do the math, 8h spend on refining information
extraction will easily translate to huge savings of time. (and we are
not using crazy complex regexp).

> Where will I look for the .so ?
> (And while using pkg-config is better, it still isn't supported by all
> external dependencies)
>
>> If apt-file is not present it is at least possible to tell (with dpkg -S or ldconfig -p) whether the .so is present and what is
>> precisely needed for the build to succeed (and mention how to find the needed package manually).
> Don't the conf-* packages already achieve this (in an hackish way) ?
>
>> It may be even possible to gater information about .so files automatically - by inspecting generated cmxa and cma
>> files with ocamlobjinfo - so it can be done by ocamlot or something like that.
> How would you get them to build in the first place ?

The .so is not really the official linking target, this is the .a
which is not versionned !

Cheers
Sylvain

>
>
> Regards,
> Nicolas
>
> [0]: Though our definition should include thing such as Homebrew (and
> other equivalents for OSX and/or Windows).
>
>
> _______________________________________________
> opam-devel mailing list
> opam-devel at lists.ocaml.org
> http://lists.ocaml.org/listinfo/opam-devel
>