[opam-devel] How to manage a package manager (Was: Re: [opam] Build clarification (#1149))

[Richard Mortier]

all;

On 3 February 2014 10:54, Thomas Gazagnaire <thomas at gazagnaire.org> wrote:
>> @samoth : even an approximate brain dump would be very useful here...
>> I have opened an issue for this purpose https://github.com/ocaml/opam/issues/1150
>
> Regarding the upgrade issues discussed there.
>
> ...To summarise, I don't see the upgrade between version of OPAM as a real concern from now on (we have just enough machinery needed in OPAM for our needs I think).
>

coming in a bit late to this thread but fwiw: i agree with thomas
above (and most of the points anil and yaron have made). it does sound
like most of the mechanism needed to handle triggering upgrade of opam
versions is there -- i'd also tend toward thinking that, given the
importance of opam, it would generally be best to inform the user that
there's a new version and (eg) that their packages cannot be upgraded
further until they upgrade opam, but not to force them to do this.

opam-in-opam upgrading is really not a concern for me, as a relatively
simple-minded user of opam, really for two reasons:

1. i install opam via my system package manager (apt or homebrew); i
expect to upgrade it via that too, so that someone else has tested
that it works with the other system library and tool versions i have.
unless opam becomes completely self-contained, it seems better to
distribute to the different distributions dealing with the enormous
test matrix of systems it might be installed on.

2. my impression is that self-updating like this is a tricky thing to
get robust (cf. the very short list of potential issues that anil
raised). i'd probably not trust opam-in-opam upgrading enough to use
it until it had been through at least a few upgrade rounds so someone
else could find the bugs (sorry).

from my point of view, i think there are other aspects of opam that
are more important to fix first. for example, i spent 5-6 hours on
trains on saturday trying to get a fresh install of opam so i could
prep the mirage demos for fosdem. (perhaps a rather harsh network
environment to try something like this, but still...) as i result i
was using a range of intermittent and poorly performing wifi networks
(east midlands trains, st pancras, android hotspot with a giffgaff
SIM) and so i found:

+ i hit timeouts that stopped package installs partway through so had
to be restarted, sometimes after having removed earlier versions of
the packages;

+ i had port blocking prevent some packages being installed which
would then stall a sequence of dependency installations (suggestion:
could we replace all "git://github.com" urls in opam-repository with
"-k git https://github.com" urls?);

+ as i was getting very little feedback on progress (even with
`--verbose --debug`, package installation would frequently stall at
"git remote set-url ...", often for minutes), i found myself assuming
yet another network issue had occurred and killing the process, which
probably didn't help leave things in a consistent state. more
fine-grained levels feedback would also be nice (eg., i often want to
see that packages are downloading ok but i don't usually need to see
all the build output).

+ by about the 3rd go, i found myself removing the default remote
(https://opam.ocaml.org/urls.txt) and adding the direct github url
with -k git as the first thing i did, as the sync `opam update` does
with the https://opam.ocaml.org/urls.txt default seemed extremely slow
in comparison. (i haven't looked, but does it do any checking to see
whether that file changed before trying to pull it?)

+ opam install of a pinned package is a bit brittle if you're in the
middle of editing the contents -- emacs appears to create symlinks
which can end up dangling, which breaks the rsync (possibly the -L
switch would help?) usually with several pages of output to trawl
through to notice what happened.

apologies if that comes across as a rant -- it's not meant to be! if
it would help, i'm happy to add issues to github; i wasn't sure how to
carve them up though.

cheers,

-- 
Richard Mortier
mort at cantab.net