[opam-devel] [MirageOS-devel] Managing $PKG_CONFIG_PATH in opam

[Anil Madhavapeddy]

On 13 Sep 2015, at 14:07, Thomas Leonard <talex5 at gmail.com> wrote:
> 
> On 12 September 2015 at 08:49, Tim Cuthbertson <tim at gfxmonk.net> wrote:
>> On the MirageOS mailing list, I submitted some patches[0] to make some
>> mirage libraries build without assuming a strict `opam` destination
>> directory layout. Mainly this was about the build scripts explicitly
>> setting PKG_CONFIG_PATH to `opam config var prefix`/lib/pkgconfig.
>> When building in a nixpkgs environment (using the `opam2nix` tool I'm
>> building[1]), there is no such path, but that's OK - the build
>> environment will have already set $PKG_CONFIG_PATH correctly.
>> 
>> [0] http://lists.xenproject.org/archives/html/mirageos-devel/2015-09/msg00000.html
>> [1] https://github.com/gfxmonk/opam2nix
>> 
>> It was generally agreed that having the build scripts perform this
>> task is not ideal, and Thomas Leonard suggested we could change `opam`
>> itself to export $PKG_CONFIG_PATH, rather than having build scripts
>> assume too much about their environment. That way a build script can
>> assume its pkg-config dependencies are available without caring
>> exactly where they live.
>> 
>> This seems like a good idea to me, and I've written up a fairly simple
>> patch which (I think) will do so:
>> 
>> https://github.com/ocaml/opam/compare/master...gfxmonk:pkg-config
>> 
>> I have only superficially tested it, but in thinking about that it
>> seems like this almost certainly won't be enough. All "pure" opam
>> packages providing pkg-config libraries should work just fine, however
>> there exist a large number of opam packages (conf-*) which exist
>> solely to force installation of system packages, and therefore most of
>> them actually rely on the system $PKG_CONFIG_PATH being used.
>> 
>> We could make `opam` prefix the opam pkgconfig path with the system
>> one, but this could lead to accidental impurity (in the case of these
>> mirage libraries, it would mean that dependencies might accidentally
>> be provided by the system rather than opam deps, which makes builds
>> fragile).
> 
> Since the opam environment is also the user's shell environment, we
> should always add to the path, I think. Using opam shouldn't prevent
> compiling non-OCaml software.

Agreed.  It's ok to substitute a PATH if it was inserted by OPAM, but
the environment should otherwise be unchanged.

Note that OPAM does scrub some variables during the invocation of a
subshell (MAKEFLAGS in particular) to prevent environment parallel
flags to propagate to packages, but nothing like this happens in the
PATH.

>> Any ideas how we can have opam manage PKG_CONFIG_PATH so that build
>> scripts don't have to?
>> 
>> (I've sent this to both the mirage & opam lists - apologies if that
>> causes too much noise, but I figure it affects both projects).

I think we could definitely use an answer to pkg-config management,
but not one that's entwined into the core of OPAM itself. If something
could be figured out that fits in with the compilers-as-packages
feature so that packages could extend the environment, this would
make cross-OS portability much easier.

In general, how important is pkg-config for library tracking?  Is 
our interest in it the consequence of some third-party packages
using it, or do we want it to be the defacto mechanism for handling
system link flags (a bit like ocamlfind for OCaml)?  If the latter,
then quite a bit of work needs to be done for straggling system
libraries that aren't packaged up using pkg-config, which I fear is
a neverending task.  I'm also curious to know how it works in a
Nix-like environment in terms of the composibility of PKG_CONFIG_PATH.

-a