[ocaml-platform] Does Core change too often?

Fri Feb 15 22:42:37 GMT 2013

I can't imagine how this would work in OCaml.  Imagine that A has a
function:

    val make_map : unit -> float Core.Std.Map.t

and B has this function:

    val consume_map : float Core.Std.Map.t -> unit

If in your own library you write:

    A.consume_map (B.make_map ())

then if A and B link to different versions of Core that have different
versions of the Map type, then the above code shouldn't compile!

I basically think that the Nix model does not map sanely onto OCaml
modules.  And I don't think this is all that OCaml specific.  The same
bug would show up in Java, I would think.

y

On Fri, Feb 15, 2013 at 5:35 PM, Malcolm Matalka <mmatalka at gmail.com> wrote:
> I don't know how Ocaml linking works, but here is how I believe it works
> in Nix, which is mostly C-based packages, of course:
>
> - All packages are given a unique path
>
> - When you compile a package, if its deps are statically linked then
>   they are just compiled into it, so it doesn't matter what else you're
>   linking against when you use this as dep
>
> - If they are dynamically linked, a post-processing step rewrites the
>   binary so all paths to deps are the unique path, I *think* this means
>   if you have the diamond problem it's ok because the dep will always
>   load the version of the library it was compiled against.
>
> But I could easily be way wrong.
>
> /M
>
> Yaron Minsky <yminsky at janestreet.com> writes:
>
>> I think the diamond dependency problem is a fundamental semantic
>> issue, and can't be worked around by tools.  If I use a Core.Map.t
>> from version 109.06, and the implementation changes in 109.07, then if
>> libraries A and B both use Core.Map.t's in their interface, there's no
>> way they can communicate, unless you propagate the versioning into the
>> library!  You could have a Core_109_07.Map.t, and Core_109_06.Map.t,
>> but I'm pretty sure madness lies that way...
>>
>> y
>>
>> On Fri, Feb 15, 2013 at 7:09 AM, Malcolm Matalka <mmatalka at gmail.com> wrote:
>>> My Nix setup is a huge hack right now, but! here is my current public
>>> fork of nixpkgs with an OLD version of opam + Core 108.07.01:
>>>
>>> https://github.com/orbitz/nixpkgs/commit/fd598f1b55c94b4d466ac5ddffb304136bdc433e
>>>
>>> The only interesting change is I had to patch Async because its build
>>> script requires /bin/bash and /bin/mkdir. And the build failure for
>>> Async is pretty awesome btw :)
>>>
>>> (I'm not 100% sure that the above branch works on Nix since I have some
>>> local changes that have been sitting around for awhile, but it gives the
>>> gist.  If more complete, working versions, are interested to anyone I
>>> can put the effort in to give a more complete demonstration)
>>>
>>> I also have these cheap scripts I use to create my environment when
>>> playing locally.  This should really be done as an eval like opam's
>>> config but right now I do something like 'ocaml_run.sh make'.  Or
>>> ocaml.sh to get the REPL.  This plays fine with opam (for me) since my
>>> script is additive to the environment.
>>>
>>> https://github.com/orbitz/ocaml_wrapper
>>>
>>> @yaron - Correct, AFAIK this does not some diamond dep problem.  Based
>>> on what Anil said I guess ocamlfind needs to be modified for that.  Or
>>> the ability for libraries to be built with their deps linked in already.
>>>
>>> /Malcolm
>>>
>>> Anil Madhavapeddy <anil at recoil.org> writes:
>>>
>>>> Right, but the opportunities for re-using previously compiled libraries
>>>> are limited, because of the exact-version requirements.  Having multiple
>>>> OPAM switches per set of constraints would be almost as good, if much
>>>> more burdensome from a command-line perspective.
>>>>
>>>> Do you have your Nix setup available anywhere? I'd definitely like to
>>>> have a play with it.
>>>>
>>>> -anil
>>>>
>>>> On 15 Feb 2013, at 11:37, Malcolm Matalka <mmatalka at gmail.com> wrote:
>>>>
>>>>> I get around this in Nix with chroot and building OCAMLPATH on the fly.
>>>>>
>>>>> On Feb 15, 2013 12:34 PM, "Anil Madhavapeddy" <anil at recoil.org> wrote:
>>>>> OPAM has a strong dependency on ocamlfind at the moment, which doesn't
>>>>> support multiple library versions.  OPAM could be extended to support
>>>>> multiple installations (since the constraint model expresses it fine),
>>>>> but would need to understand more of the build process too.
>>>>>
>>>>> There are some steps towards this already, such as the `.install` files
>>>>> which automate binary installation/removal.  We were reluctant to put
>>>>> any more in the first version in the interests of getting something out.
>>>>>
>>>>> -anil
>>>>>
>>>>> On 15 Feb 2013, at 11:21, Malcolm Matalka <mmatalka at gmail.com> wrote:
>>>>>
>>>>> > Right now, I think Core not being afraid to change frequently is good.
>>>>> > I'd rather have that than a stale API that kind of sucks.  And I'd hate
>>>>> > to have the public Core get out of synch with what Jane St uses
>>>>> > internally because it increases the overhead for you which decreases the
>>>>> > chance of timely releases.
>>>>> >
>>>>> > Multiple versions has not been a problem for me since I convert opam
>>>>> > packages I'm interested in to Nix packages, and Nix handles multiple
>>>>> > versions of installed packages just fine.  It's probably too late in the
>>>>> > game for opam to have this style as well, I don't know anything about
>>>>> > opam's design, but it's worked pretty well in Nix for me.
>>>>> >
>>>>> > Beyond that, I think good semantic versioning is probably key.  If the
>>>>> > majority of Core changes are backwards compatible (I have no idea if
>>>>> > this is the case) then installing the right package for a lot of the
>>>>> > apps probably isn't so bad.
>>>>> >
>>>>> > /Malcolm
>>>>> >
>>>>> > Yaron Minsky <yminsky at janestreet.com> writes:
>>>>> >
>>>>> >> Right now, the Core suite of libraries changes a lot --- we have a new
>>>>> >> release of everything every week.  The changes on a given week are
>>>>> >> small, but there are always changes.
>>>>> >>
>>>>> >> I can imagine this being something of a problem for OPAM.  If packages
>>>>> >> specify specific revisions of the Core suite, then we're going to have
>>>>> >> a massive version mismatch problem, where no two libraries can agree
>>>>> >> on the version of Core that they need.
>>>>> >>
>>>>> >> I have no obvious ideas as to how to solve this.  Does anyone else
>>>>> >> have ideas?  Should we simply encourage packagers to specify a
>>>>> >> lower-bound constraint on the Core libraries?
>>>>> >>
>>>>> >> y
>>>>> >> _______________________________________________
>>>>> >> Platform mailing list
>>>>> >> Platform at lists.ocaml.org
>>>>> >> http://lists.ocaml.org/listinfo/platform
>>>>> > _______________________________________________
>>>>> > Platform mailing list
>>>>> > Platform at lists.ocaml.org
>>>>> > http://lists.ocaml.org/listinfo/platform
>>>>> >
>>>>>