[opam-devel] OPAM + Docker

Sylvain Le Gall sylvain+ocaml at le-gall.net
Tue Nov 19 10:20:31 GMT 2013 

Hey, just think about an even more "reliable" way to do all that: make
it fails ! ;-)

Not really kidding:
1. try to build
2. it fails because of missing deps
3. parse the output log and spot the error (pretty easy in fact, vim
quickfix as a lot of regex to extract them)
4. you will get the missing file in a standard error format
5. replace the result of grep above by the missing filename
6. fill the depext
7. goto 1.

On the first try it will not be very efficient, but you can send the
result for ObscureOS Y to opam and register it for other ObscureOS Y
users.


2013/11/19 Sylvain Le Gall <sylvain+ocaml at le-gall.net>:
> 2013/11/19 Nicolas Braud-Santoni <nicolas at braud-santoni.eu>:
>> On 19/11/2013 05:03, ygrek wrote:
>>> On Mon, 18 Nov 2013 10:43:59 -0800
>>> Anil Madhavapeddy <anil at recoil.org> wrote:
>>>
>>>> I've made some notes about using OPAM and Docker for easier local bulk builds here:
>>>> http://anil.recoil.org/2013/11/15/docker-and-opam.html
>>>>
>>>> One semi-interesting issue I've already discovered with depexts is that just a "ubuntu" tag isn't enough, since  some depexts are only valid in a specific version of Ubuntu.
>>>>
>>>> This unfortunately feels like it might be a job for the constraint system and not for flat tags (since I want to say >precise or <saucy).  We could do it with flat tags for now, but I have a bad feeling that it'll get unmanageable quite quickly..
>> Two remarks:
>> 1. Using the version numbers might be much less confusing than the codenames
>> 2. I agree that we need to agree on useable syntax (and semantics) for
>> depexts. See https://github.com/ocaml/opam/issues/991
>>
>>> I personally dislike this depexts field to mention OS packages at all.
>> Why ?
>> This seems to be the most sane and generic option if we wish to interact
>> with the system's package manager [0].
>
> The problem is that each time you want to extend to a new OS
> distribution, you may need to do a lot of work. Not to mention that
> this work will probably need to be done when upgrading as well,
> although it may be less expensive. On the long term you may need to
> dedicate a fair amount of time dealing with updating the depext. But
> that is my rough estimate, and I do prefer to see people try it than
> not do anything about that (but we should be open and consider
> dropping this if it turns to be a not that sustainable solution).
>
>>
>>> Maybe it's better to specify which external libraries are needed (as .so or as pkg-config files) and then
>>> use native OS way (at installation time) to determine which package contains that .so or .pc file (e.g. apt-file on Debian).
>> Not all package managers provide these kind of facilities.
>> That is someting that, for instance, will not exist in any source-based
>> distribution that I am aware of.
>>
>> Moreover, it seems unwise to fill in information that ought to belong in
>> the distribution's packages :
>> if I depend on libfoo, major version 1, any minor version, should
>> I specify “libfoo-1.so”, “libfoo-1(.\d+)*.so” (dots implicitly
>> escaped), or some crazier scheme ?
>
> Not really crazy when you think about it but you should probably not
> depends on the .so. You just focus on what you need extracting data
> from the package:
> - scan .c -> extract #include "X.h", it defines a dependencies
> - scan build system -> --libs X or -lX -> you need libX.a
>
> Most of the package manager allow you to do file lookup: e.g. "dpkg
> -S" on Debian, example:
> $ dpkg -S libpcre.a
> libpcre3-dev: /usr/lib/i386-linux-gnu/libpcre.a
>
> Lets take a real example:
> $ cd ocaml-zmq
> $ grep -r -e '-l.*' .
> ./_oasis:  CCLib:          -lzmq
> ./myocamlbuild.ml:          (* When one link an OCaml
> library/binary/package, one should use -linkpkg *)
> ./myocamlbuild.ml:          flag ["ocaml"; "link"; "program"] & A"-linkpkg";
> ./myocamlbuild.ml:           * options when using this tag. When using
> the "-linkpkg" option with
> ./myocamlbuild.ml:                     (S[A"-dllib"; A("-l"^lib);
> A"-cclib"; A("-l"^lib)]);
> ./myocamlbuild.ml:                     (S[A"-cclib"; A("-l"^lib)]);
> ./myocamlbuild.ml:            [(OASISExpr.EBool true, S [A "-cclib"; A
> "-lzmq"])]);
> ./myocamlbuild.ml:            [(OASISExpr.EBool true, S [A "-lzmq"])])
> ./setup.ml:                   don't use the right command-line
> arguments, rerun \
> ./setup.ml:            "-no-log";
> ./setup.ml:            "-no-links";
> ./setup.ml:            "-install-lib-dir";
> ./setup.ml:                      bs_cclib = [(OASISExpr.EBool true, ["-lzmq"])]
>
> $ grep -r -e '#include <.*>' .
> ./src/caml_zmq_stubs.c:#include <stdlib.h>
> ./src/caml_zmq_stubs.c:#include <stdint.h>
> ./src/caml_zmq_stubs.c:#include <string.h>
> ./src/caml_zmq_stubs.c:#include <stdio.h>
> ./src/caml_zmq_stubs.c:#include <stdbool.h>
> ./src/caml_zmq_stubs.c:#include <caml/mlvalues.h>
> ./src/caml_zmq_stubs.c:#include <caml/alloc.h>
> ./src/caml_zmq_stubs.c:#include <caml/fail.h>
> ./src/caml_zmq_stubs.c:#include <caml/callback.h>
> ./src/caml_zmq_stubs.c:#include <caml/memory.h>
> ./src/caml_zmq_stubs.c:#include <caml/custom.h>
> ./src/caml_zmq_stubs.c:#include <caml/intext.h>
> ./src/caml_zmq_stubs.c:#include <caml/threads.h>
> ./src/caml_zmq_stubs.c:#include <zmq.h>
>
> $ dpkg -S libzmq.a
> libzmq-dev:i386: /usr/lib/i386-linux-gnu/libzmq.a
>
> $ dpkg -S zmq.h
> libzmq-dev:i386: /usr/include/zmq.hpp
> libzmq-dev:i386: /usr/include/zmq.h
>
> You don't even have to do crazy comlpex thing here (a simple grep in
> the source will already provide you good insight). Although, you may
> replace dpkg by apt-file, because dpkg only rely on files installed.
>
> The first step (extract information with grep), is distribution
> agnostic and the second steps need to be translated once per OS
> distribution. If you do the math, 8h spend on refining information
> extraction will easily translate to huge savings of time. (and we are
> not using crazy complex regexp).
>
>> Where will I look for the .so ?
>> (And while using pkg-config is better, it still isn't supported by all
>> external dependencies)
>>
>>> If apt-file is not present it is at least possible to tell (with dpkg -S or ldconfig -p) whether the .so is present and what is
>>> precisely needed for the build to succeed (and mention how to find the needed package manually).
>> Don't the conf-* packages already achieve this (in an hackish way) ?
>>
>>> It may be even possible to gater information about .so files automatically - by inspecting generated cmxa and cma
>>> files with ocamlobjinfo - so it can be done by ocamlot or something like that.
>> How would you get them to build in the first place ?
>
> The .so is not really the official linking target, this is the .a
> which is not versionned !
>
> Cheers
> Sylvain
>
>>
>>
>> Regards,
>> Nicolas
>>
>> [0]: Though our definition should include thing such as Homebrew (and
>> other equivalents for OSX and/or Windows).
>>
>>
>> _______________________________________________
>> opam-devel mailing list
>> opam-devel at lists.ocaml.org
>> http://lists.ocaml.org/listinfo/opam-devel
>>

More information about the opam-devel mailing list