[opam-devel] How to manage a package manager

Sun Feb 2 18:38:24 GMT 2014

The Nix community is going through a similar discussion about package
ownership.  One suggestion is simply to delete all packages that do not
have an owner.  It's somewhat extreme but does guarantee packages at
least have a contact point that should keep the package up-to-date.

http://lists.science.uu.nl/pipermail/nix-dev/2014-January/012444.html

/M

Yaron Minsky <yminsky at gmail.com> writes:

> I broadly agree with Anil's point about priorities.  opam-in-opam
> seems tricky, and to me relatively low value.  Making packages and
> installation and uninstallation more reliable seems more boring, but
> way more important.
>
> In terms of improving package quality, I wonder if we need better
> tools to highlight to package owners when their packages fail.  One
> approach would be for each package to have a set of owners, and for
> each owner to have a status page that alerts them to problems with
> their packages.  I think there would be a lot of value in alerting
> people when their packages are broken, and it would be even better to
> reject their pull requests automatically if it appears that they're
> breaking the world.
>
> A useful concept here is blame.  When something is broken, it's nice
> if you can (sometimes artificially) choose someone to "blame" for the
> problem, i.e., someone who is in some way involved in what has gone
> wrong, and can be the responsible person to ensure it gets fixed, or
> gets assigned to the right person to fix it.  I wonder if ocamlot (or
> whatever triaging system we end upwith) can have some notion of blame
> built into it.
>
> y
>
> On Sun, Feb 2, 2014 at 10:49 AM, Anil Madhavapeddy <anil at recoil.org> wrote:
>> On 1 Feb 2014, at 19:27, Fabrice Le Fessant <Fabrice.Le_fessant at inria.fr> wrote:
>>
>>> Hi,
>>>
>>>  Anil, I am worried that your workflow is not the typical workflow of
>>> OCaml developers. You are an expert in OPAM, having several OPAM
>>> repositories and so on. The typical user I imagine has little
>>> knowledge of OPAM, and expect upgrades of both the repository and of
>>> OPAM to be as simple as possible. I remember using Cabal to install
>>> some Haskell software: I would do it once per year, at most, and
>>> everytime, I was very annoyed to discover some incompatibility that
>>> would require an update of Cabal itself. Now, I fear to see the same
>>> problem with OPAM.
>>
>> Hi Fabrice,
>>
>> I do agree with you here of course.  I'm just noting from my experience
>> with running quite a few OPAM-based tutorials in the last year (starting
>> with the public beta release in ICFP 2012 with Thomas and Yaron, and
>> subsequently from helping a number of non-OCaml users get started with
>> Mirage and Real World OCaml, both of which require OPAM), that users
>> are primarily confused by two things:
>>
>> - Getting started with OPAM: including utop, their ocamlinit, and all
>> the other bits and bobs (especially editor integration).
>>
>> - Broken packages.  Part of the reason for Cabal's irritation are
>> incorrect package constraints. Now, if you were leave an OPAM install
>> for a few months and upgrade, then you are using a very untested path
>> in the solver.  A lot of things have to work *perfectly* to have a
>> successful package upgrade: the solver has to find a solution, the
>> uninstall targets have to be correct (or else ocamlfind complains
>> on the subsequent installation, and the installation must work including
>> depexts).  If anything goes wrong here, then the user's left to debug
>> the mess themselves.
>>
>> I prioritise improving these above anything else to do with OPAM for
>> my own hacking time.  We've made very significant progress (thanks to
>> Roberto and the aspcud team) in OPAM 1.1.1, but there is always more to
>> improve.
>>
>> For the second point, I must admit I'm a little disappointed by the
>> lack of feedback to my various posts about doing OPAM bulk builds to
>> improve the package repository quality.  After the Docker post [1]
>> (which works on any modern Linux), I also published the logfiles from
>> the build [2] and triaged a number of bugs [3], but not many other
>> people have contributed fixes.  I would very much like to get to a
>> 100% package installation rate, which we can then keep at 100% due to
>> the ongoing package testing for new pull requests.
>>
>> [1] http://anil.recoil.org/2013/11/15/docker-and-opam.html
>> [2] https://github.com/avsm/opam-bulk-logs
>> [3] https://github.com/ocaml/opam-repository/issues/1304
>>
>> The question of upgrading OPAM itself has never come up with anyone,
>> since most users simply used their OS-installed tool to upgrade from
>> OPAM 1.0 to 1.1.  The only major bug with this approach was because
>> the release of OCaml 4.01 coincided with OPAM 1.1, and a lot of Mac
>> users ended up upgrading their system compiler *and* OPAM simultaneously,
>> which led to complete meltdown of their ~/.opam (see [4])
>>
>> [4] https://github.com/Homebrew/homebrew/issues/24315
>>
>> This nasty bug should tell us that upgrades are just very complicated
>> and need a lot of testing (we tested system compiler updates, and OPAM
>> version updates, but never together!). I would expect any auto-update
>> procedure to  at least consider these issues:
>>
>> - What happens during an upgrade if a newly installed OPAM binary
>>   inside a ~/.opam requires an upgrade of the OPAM metadata?
>>
>> - If you do an opam update using the OPAM binary installed within
>>   the ~/.opam space, how do you avoid a 'text file busy' error
>>   due to being unable to replace the currently executing binary?
>>   We see this a lot on Mirage [5] due to the command line tool
>>   and the libraries being mixed up, as mentioned earlier.
>>
>> - If the user invokes OPAM without first calling `opam config env`,
>>   they will use the *old* OPAM with a *new* repository.  This
>>   is currently a non-issue since there is only one OPAM binary,
>>   and all sub-commands spawned by it apply the correct env variables.
>>   The autoupdate scheme proposed will absolutely require that the
>>   user has correctly set up their `opam config env`, which is
>>   something that I've observed many users not do (including myself).
>>   This will thus require support and testing of downgrade.
>>
>> [5] https://travis-ci.org/mirage/mirage-www/jobs/17859382
>>
>>>  Even if it's not easy to achieve, because of the risk of breaking
>>> other OPAM features, I think "transparent update" is definitively
>>> something the typical user of OPAM would like and find very cool with
>>> OPAM, and worth adding the feature to the roadmap !
>>
>> I'm obviously not against adding magical auto-updating, but I hope
>> that any such proposal will consider the issues i raise above.
>> Personally, I'm going to spend my limited time working on what
>> I've observed are the real (but admittedly more boring) problems
>> with day-to-day OPAM use that I listed above, such as upstream
>> packaging and improving the repository quality.
>>
>> More broadly, I do not believe that we should get too dependent on
>> OPAM and ignore upstream binary packaging. In Xen for example,
>> the Debian and Fedora packages require binary OCaml packages to
>> link against, and telling everyone to use OPAM is not an acceptable
>> answer.
>>
>>
>> -anil
>> _______________________________________________
>> opam-devel mailing list
>> opam-devel at lists.ocaml.org
>> http://lists.ocaml.org/listinfo/opam-devel
> _______________________________________________
> opam-devel mailing list
> opam-devel at lists.ocaml.org
> http://lists.ocaml.org/listinfo/opam-devel