[opam-devel] How to manage a package manager (Was: Re: [opam] Build clarification (#1149))
Roberto Di Cosmo
roberto at dicosmo.org
Fri Jan 31 17:28:11 GMT 2014
[I am moving this discussion away from the BTS and into
the mailing list, as the issues raised here are of
general relevance for the future of Opam, and must
not be lost in the middle of other comments; see
https://github.com/ocaml/opam/pull/1149 for the original
The original issue: what build system should we use to build opam?
Should we cook up an ad-hoc Makefile, use a patched ocamlbuild,
or use OCamlMakefile, or use ocp-build? Is it evil to embark
a build system in the package manager source code?
My quick view of these issues:
1) choice of the build system
we are already plagued with multiple build systems, and one of the reason
of opam's success is that it allows every package author the freedom to use
the one he/she prefers/ is more comfortable with.
Opam is no exception, and opam developers must be able to pick the build system
they prefer, so I would say, it's their decision to make
2) embarking a full build system into the source tree of a package
in general I would find this a very bad idea: one should have a separate
package for the build system, and have it installed before building anything else.
Now, what sets opam apart from other OCaml components is the fact that it is
the *basic* building block for all the other packages in the repository, and
without a running opam, there is no way to install any other package; this
is a bootstrap issue, and that's why, for example, opam embarks in src_ext
the full source of the dose and libcudf libraries, instead of linking to
the packages for dose and libcudf (that do exist in the repository); so
in this particular case *only* I personally would not be shocked to see
some extra stuff in the src_ext directory (especially considering that
all this will need to work on multiple platforms)
3) how do we update opam?
Now, for another important issue, related to the special status of opam
as *the* package manager, and how we should manage it (hence the title
of this message).
I am a bit unhappy with the current state of affairs, where new
versions of opam appear with incompatible changes in the repository format,
hence different versions of the repositories: people using old versions
of opam get lost in a limbo-like repository with non updated packages
until they realise they need to recompile a new version of opam
to see the light again.
We need to find a way to have opam become a package like the others
(with the exception of the embarked code for bootstrapping reasons),
and have it become part of opam-repository... the main difficulty
lies in the fact that its binaries need to go in special places,
and not in the 4.00/3.12 etc. dirs (correct if I'm wrong)
Once this is done, a sensible upgrade path would be:
Time T : I have opam version n installed, accessing repo version n
Time T+delta: some major changes make the old opam unable to use the new
repository format, and the repository migrates to a new place;
a new version n+1 of opam is released, embedding logic to access
the new repository;
opam version n+1 is added also to repo version n
Sometimes later: the user of opam n runs an update, sees that a new
version of opam is available, and upgrades it...
he now has the new opam, that updates the references
to the new repository, and keeps following the evolution
of the repository, with no pain
I know that maybe at some point the repo format will stabilise, but this
simple process will make sure that the lambda-users (private joke: that's
equivalent to "jow of the street" in french, but also hints at functional
programmers :-)) will not get lost, no matter the changes.
Any chances to see this happen soon? A starting point would be to summarize
on this list the difficulties that prevent opam from being a "normal"
More information about the opam-devel