[opam-devel] How to manage a package manager (Was: Re: [opam] Build clarification (#1149))

[Fabrice Le Fessant]

Hi,

  Anil, I am worried that your workflow is not the typical workflow of
OCaml developers. You are an expert in OPAM, having several OPAM
repositories and so on. The typical user I imagine has little
knowledge of OPAM, and expect upgrades of both the repository and of
OPAM to be as simple as possible. I remember using Cabal to install
some Haskell software: I would do it once per year, at most, and
everytime, I was very annoyed to discover some incompatibility that
would require an update of Cabal itself. Now, I fear to see the same
problem with OPAM.

  Let's dream a little: in my ideal world, there would be an OPAM
_bytecode_ binary in the opam-repository, always the lattest stable
version, and able to upgrade whatever version I have of the
opam-repository to the lattest version. Everytime I would do "opam
update", OPAM would first check that binary, download it if needed and
run it instead of itself (using the same trick as done by ocp-build
bytecode to be run on any system), upgrading the repo if needed, and
then do the update. The update would be almost completely transparent
(maybe OPAM would ask the user to authorize the update, since the user
has to trust a binary).

  Even if it's not easy to achieve, because of the risk of breaking
other OPAM features, I think "transparent update" is definitively
something the typical user of OPAM would like and find very cool with
OPAM, and worth adding the feature to the roadmap !
--Fabrice









On Sat, Feb 1, 2014 at 3:32 PM, Anil Madhavapeddy <anil at recoil.org> wrote:
>
> On 1 Feb 2014, at 15:12, Roberto Di Cosmo <roberto at dicosmo.org> wrote:
>
>> On Sat, Feb 01, 2014 at 02:53:06PM +0100, Anil Madhavapeddy wrote:
>>> On 1 Feb 2014, at 14:40, Roberto Di Cosmo <roberto at dicosmo.org> wrote:
>>>>
>>>>> This is a very tricky interaction to get right,
>>>>
>>>> Really? I would like to know more about this; please add the factual elements
>>>> to #1150
>>>
>>> I think the overall problem element is obvious: the opam binary within the
>>> opam installation will be picked up very suddenly during package installation
>>> and upgrades.
>>>
>>> Rather than me listing why every single feature of OPAM would potentially break
>>> (consider the 'repo upgrade' step that OPAM 1.0 -> 1.1 does), I would again
>>> suggest that you provide a strong motivation for why you want the feature in the
>>> first place.
>>
>> Sorry, I really do not understand what you mean. On my side, I do believe I made
>> my point clear enough, and I see no point in continuing this exchange.
>
> Let me go back to your original point, which I agree with:
>
>> That's why one needs a clear, and simple, mechanism to ensure a smooth
>> transition when incompatibilities arrive (and there will be more, and
>> unexpected ones, you can bet on it).
>
> My point is that the user originally obtains their version of OPAM via their
> OS package manager (Homebrew, apt, BSD ports, Yum, etc).  All of these managers
> provide a mechanism to warn the user of any upgrade steps that need to happen
> when upgrading their distribution, and so is an ideal way to update OPAM
> itself in the event of incompatible future changes.
>
> In contrast, `opam update` is part of a *development* workflow, not a system
> upgrade path.  I do that when I want to recompile some part of my working
> trees to a new package set, and often follow it up with an upgrade (via -u).
>
> If you mix these things up, then it violates a lot of expectations from the
> user side, and can lead to the annoying situation where you are forced to deal
> with a distribution upgrade when you were expecting to just alter some local
> code.  Furthermore, it is quite common to destroy and reinitialize ~/.opam
> or set a new OPAMROOT to test for a clean install, and this would subsequently
> downgrade OPAM (or worse, cause an error if your shell path cache becomes
> invalidated).
>
> We've already seen this problem in action with a number of tools such as
> Merlin or ocp-index that really ought to be installed system-wide and not
> within OPAM (since a code update could result in the editor tool being
> recompiled or downgraded).  Another example is the Mirage command-line tool,
> which uses libraries shared with development code.
>
> To summarise, this feature continues to blur the divide between a system tool
> (OPAM, Merlin, ocp-index) from a development tool, and should not be
> encouraged. I would like to see better support for managing system tools
> within OPAM, but I think prioritising upstream integration of them should
> be encouraged first.  OPAM is a developer supplement to make source-based
> development easier, not a complete replacement for an existing OS package
> manager.
>
> I hope that's clearer.  There are many negative reasons for upgrading OPAM
> using OPAM, and I've still not seen a good reason for not focussing on
> better upstream package manager (and hence external dependency) integration
> before anything else.
>
> -anil
> _______________________________________________
> opam-devel mailing list
> opam-devel at lists.ocaml.org
> http://lists.ocaml.org/listinfo/opam-devel



-- 
Fabrice LE FESSANT
Chercheur en Informatique
INRIA Paris Rocquencourt -- OCamlPro
Programming Languages and Distributed Systems