[opam-devel] [MirageOS-devel] Managing $PKG_CONFIG_PATH in opam

[Anil Madhavapeddy]

> On 14 Sep 2015, at 15:05, Thomas Leonard <talex5 at gmail.com> wrote:
> 
> On 14 September 2015 at 14:36, Anil Madhavapeddy <anil at recoil.org> wrote:
>> On 13 Sep 2015, at 14:07, Thomas Leonard <talex5 at gmail.com> wrote:
>>> 
>>> On 12 September 2015 at 08:49, Tim Cuthbertson <tim at gfxmonk.net> wrote:
>>>> On the MirageOS mailing list, I submitted some patches[0] to make some
>>>> mirage libraries build without assuming a strict `opam` destination
>>>> directory layout. Mainly this was about the build scripts explicitly
>>>> setting PKG_CONFIG_PATH to `opam config var prefix`/lib/pkgconfig.
>>>> When building in a nixpkgs environment (using the `opam2nix` tool I'm
>>>> building[1]), there is no such path, but that's OK - the build
>>>> environment will have already set $PKG_CONFIG_PATH correctly.
>>>> 
>>>> [0] http://lists.xenproject.org/archives/html/mirageos-devel/2015-09/msg00000.html
>>>> [1] https://github.com/gfxmonk/opam2nix
>>>> 
>>>> It was generally agreed that having the build scripts perform this
>>>> task is not ideal, and Thomas Leonard suggested we could change `opam`
>>>> itself to export $PKG_CONFIG_PATH, rather than having build scripts
>>>> assume too much about their environment. That way a build script can
>>>> assume its pkg-config dependencies are available without caring
>>>> exactly where they live.
>>>> 
>>>> This seems like a good idea to me, and I've written up a fairly simple
>>>> patch which (I think) will do so:
>>>> 
>>>> https://github.com/ocaml/opam/compare/master...gfxmonk:pkg-config
>>>> 
>>>> I have only superficially tested it, but in thinking about that it
>>>> seems like this almost certainly won't be enough. All "pure" opam
>>>> packages providing pkg-config libraries should work just fine, however
>>>> there exist a large number of opam packages (conf-*) which exist
>>>> solely to force installation of system packages, and therefore most of
>>>> them actually rely on the system $PKG_CONFIG_PATH being used.
>>>> 
>>>> We could make `opam` prefix the opam pkgconfig path with the system
>>>> one, but this could lead to accidental impurity (in the case of these
>>>> mirage libraries, it would mean that dependencies might accidentally
>>>> be provided by the system rather than opam deps, which makes builds
>>>> fragile).
>>> 
>>> Since the opam environment is also the user's shell environment, we
>>> should always add to the path, I think. Using opam shouldn't prevent
>>> compiling non-OCaml software.
>> 
>> Agreed.  It's ok to substitute a PATH if it was inserted by OPAM, but
>> the environment should otherwise be unchanged.
>> 
>> Note that OPAM does scrub some variables during the invocation of a
>> subshell (MAKEFLAGS in particular) to prevent environment parallel
>> flags to propagate to packages, but nothing like this happens in the
>> PATH.
>> 
>>>> Any ideas how we can have opam manage PKG_CONFIG_PATH so that build
>>>> scripts don't have to?
>>>> 
>>>> (I've sent this to both the mirage & opam lists - apologies if that
>>>> causes too much noise, but I figure it affects both projects).
>> 
>> I think we could definitely use an answer to pkg-config management,
>> but not one that's entwined into the core of OPAM itself. If something
>> could be figured out that fits in with the compilers-as-packages
>> feature so that packages could extend the environment, this would
>> make cross-OS portability much easier.
>> 
>> In general, how important is pkg-config for library tracking?  Is
>> our interest in it the consequence of some third-party packages
>> using it, or do we want it to be the defacto mechanism for handling
>> system link flags (a bit like ocamlfind for OCaml)?  If the latter,
>> then quite a bit of work needs to be done for straggling system
>> libraries that aren't packaged up using pkg-config, which I fear is
>> a neverending task.  I'm also curious to know how it works in a
>> Nix-like environment in terms of the composibility of PKG_CONFIG_PATH.
> 
> I'm not sure I understand what you're saying here. pkg-config is like
> ocamlfind for C libraries, and very widely used (try "pkg-config
> --list-all" to see your installed packages). For libraries that use
> it, it is usually the only way to get the flags you need.

Right -- when pkg-config works, it works pretty well.  However, many
libraries don't use pkg-config, or even worse, install broken .pc files
(MacOS X is terrible for this).  In these situations, we will then need
to bypass them and get the compiler flags in some other way.

So when it comes to support for it in OPAM, it would be nice to not
embed pkg-config support directly in the core tool, but to add a slightly
more general mechanism for extending environment variables in such a
way that OS-specific quirks can be dealt with within the repository
rather than by patching OPAM itself.

-anil