[opam-devel] [MirageOS-devel] Managing $PKG_CONFIG_PATH in opam

[Thomas Leonard]

On 14 September 2015 at 14:36, Anil Madhavapeddy <anil at recoil.org> wrote:
> On 13 Sep 2015, at 14:07, Thomas Leonard <talex5 at gmail.com> wrote:
>>
>> On 12 September 2015 at 08:49, Tim Cuthbertson <tim at gfxmonk.net> wrote:
>>> On the MirageOS mailing list, I submitted some patches[0] to make some
>>> mirage libraries build without assuming a strict `opam` destination
>>> directory layout. Mainly this was about the build scripts explicitly
>>> setting PKG_CONFIG_PATH to `opam config var prefix`/lib/pkgconfig.
>>> When building in a nixpkgs environment (using the `opam2nix` tool I'm
>>> building[1]), there is no such path, but that's OK - the build
>>> environment will have already set $PKG_CONFIG_PATH correctly.
>>>
>>> [0] http://lists.xenproject.org/archives/html/mirageos-devel/2015-09/msg00000.html
>>> [1] https://github.com/gfxmonk/opam2nix
>>>
>>> It was generally agreed that having the build scripts perform this
>>> task is not ideal, and Thomas Leonard suggested we could change `opam`
>>> itself to export $PKG_CONFIG_PATH, rather than having build scripts
>>> assume too much about their environment. That way a build script can
>>> assume its pkg-config dependencies are available without caring
>>> exactly where they live.
>>>
>>> This seems like a good idea to me, and I've written up a fairly simple
>>> patch which (I think) will do so:
>>>
>>> https://github.com/ocaml/opam/compare/master...gfxmonk:pkg-config
>>>
>>> I have only superficially tested it, but in thinking about that it
>>> seems like this almost certainly won't be enough. All "pure" opam
>>> packages providing pkg-config libraries should work just fine, however
>>> there exist a large number of opam packages (conf-*) which exist
>>> solely to force installation of system packages, and therefore most of
>>> them actually rely on the system $PKG_CONFIG_PATH being used.
>>>
>>> We could make `opam` prefix the opam pkgconfig path with the system
>>> one, but this could lead to accidental impurity (in the case of these
>>> mirage libraries, it would mean that dependencies might accidentally
>>> be provided by the system rather than opam deps, which makes builds
>>> fragile).
>>
>> Since the opam environment is also the user's shell environment, we
>> should always add to the path, I think. Using opam shouldn't prevent
>> compiling non-OCaml software.
>
> Agreed.  It's ok to substitute a PATH if it was inserted by OPAM, but
> the environment should otherwise be unchanged.
>
> Note that OPAM does scrub some variables during the invocation of a
> subshell (MAKEFLAGS in particular) to prevent environment parallel
> flags to propagate to packages, but nothing like this happens in the
> PATH.
>
>>> Any ideas how we can have opam manage PKG_CONFIG_PATH so that build
>>> scripts don't have to?
>>>
>>> (I've sent this to both the mirage & opam lists - apologies if that
>>> causes too much noise, but I figure it affects both projects).
>
> I think we could definitely use an answer to pkg-config management,
> but not one that's entwined into the core of OPAM itself. If something
> could be figured out that fits in with the compilers-as-packages
> feature so that packages could extend the environment, this would
> make cross-OS portability much easier.
>
> In general, how important is pkg-config for library tracking?  Is
> our interest in it the consequence of some third-party packages
> using it, or do we want it to be the defacto mechanism for handling
> system link flags (a bit like ocamlfind for OCaml)?  If the latter,
> then quite a bit of work needs to be done for straggling system
> libraries that aren't packaged up using pkg-config, which I fear is
> a neverending task.  I'm also curious to know how it works in a
> Nix-like environment in terms of the composibility of PKG_CONFIG_PATH.

I'm not sure I understand what you're saying here. pkg-config is like
ocamlfind for C libraries, and very widely used (try "pkg-config
--list-all" to see your installed packages). For libraries that use
it, it is usually the only way to get the flags you need.

C packages distributed via opam will normally want to install
per-switch .pc files, just as OCaml programs want to install ocamlfind
META files. Either every package extends PKG_CONFIG_PATH with its own
directory in lib, or opam should provide a common directory for them
all (as it already does for binaries, man-pages, etc).

0install takes the approach of adding each library to PKG_CONFIG_PATH,
because it separates the files of different packages completely.
However, it can only do this because it sets up the build environment
for each build itself, with just the libraries needed for that build.
For opam, which likes to put everything into the user's default
environment, PKG_CONFIG_PATH would get very long.


-- 
Dr Thomas Leonard        http://roscidus.com/blog/
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA