[opam-devel] opam 2.0~alpha2 -- an update on opam's current dev status

[Louis Gesbert]

Note that you may have issues with opam files from the older version that have availability constraints on `ocaml-version`, e.g. the ones found in package sources when pinning. Indeed, since `ocaml-version` is no longer defined as a variable, the packages will become unavailable.

An easy workaround is to define it manually as a switch variable:

    opam config set ocaml-version $(ocamlc -vnum)

Otherwise, what should be done once 2.0 is mainstream is:
- replace availability constraints with `depends: "ocaml" {version-constraint}`
- replace other uses of `ocaml-version` by `ocaml:version` (i.e. the currently installed version of package "ocaml")

(that's basically what `opam-admin upgrade-format` does)

> - Louis Gesbert, 22/07/2016 13:31 -
> This has now been merged, and the rewritten repository at https://opam.ocaml.org/2.0~dev (to which opam 2 is automatically redirected) is updated accordingly.
> 
> This means that anyone testing a 2.0~alpha development version (thanks!) will need to upgrade their opam when they want to do `opam update`. Thanks for your understanding, and enjoy the new version!
> 
> 
> > - Louis Gesbert, 18/07/2016 15:34 -
> > Hi all,
> > 
> > We finally have a solution that I find satisfying to the larger blocker we had 
> > with opam 2.0 — inconsistency of the use of `available:` fields based on a 
> > variable `ocaml-version` that couldn't be known before the `ocaml` package was 
> > installed. See below if you are interested in the details, I have said a few 
> > words about it here and there, but a summary is in order.
> > 
> > With this, and after a little bit more testing, I think we should be done with 
> > the large repository format changes and rewrites, and that alone is a good 
> > reason to release the too-long-awaited successor of 1.2.2. Adding the many, 
> > smaller and larger features you're all waiting for (local switches, per-switch 
> > remotes, etc., etc.) from there should be smoother and can be done 
> > incrementally.
> > 
> > For all users and testers of 2.0~alpha — thank you — the alpha2 changes the 
> > repository layout a bit, so when I merge and put the new repository rewrite in 
> > place on https://opam.ocaml.org/2.0~dev, you will need to update your version 
> > of opam (I don't want to spend time on maintaining repo mirrors at different 
> > dev versions, not counting the issues that might arise on the older versions).
> > 
> > Also, I think we should release opam 2.0 while the default repository is still 
> > an automatic rewrite of the 1.2 repository; then, take the time to put a 
> > reverse rewrite in place (the 1.2→2.0 rewrite is far from trivial, changing 
> > .comp to opam files, rewriting variables, changing formulas on variables into 
> > dependencies on packages, so this won't be easy). Once this is done, automatic 
> > testing follows and enough people have made the move, we can independently 
> > switch the format of the official opam-repository to 2.0.
> > 
> > 
> > ---
> > 
> > Now for the technical details. The upcoming dev version creates 4 distinct 
> > `ocaml` packages:
> > - `ocaml-system`: an OCaml compiler from the system, outside of opam. There is 
> > an instance of the package for every OCaml release.
> > - `ocaml-base-compiler`: the "official" releases of OCaml, compiled from 
> > source by opam, with the corresponding version
> > - `ocaml-variants`: a single package for all "other" versions of OCaml, built 
> > by opam. The package versions have the form `OCAML_VERSION+VARIANT`, e.g. 
> > `4.02.3+fp`
> > - `ocaml`: a wrapper package that depends on one of the first three, for each 
> > OCaml release, mapping the versions in the case of `ocaml-variants`. For 
> > example, `ocaml.4.03.3` depends on `ocaml-system{=4.02.3} | ocaml-base-
> > compiler{=4.02.3} | ocaml-variants{>=4.02.3 & <4.02.4~}`. Additionally, the 
> > package runs an (ocaml) script that detects the specifics of the OCaml 
> > version, initialising package variables such as `ocaml:native-dynlink`¹.
> > 
> > All of the first three are mutually conflicting, which is the reason for 
> > keeping all variants in a single package². Any constraints on the OCaml 
> > version should now be expressed through a dependency to the `ocaml` package 
> > instead of the `available:` field.
> > 
> > To this point, this leaves us with the matter of the "system" compilers, which 
> > need to be selected manually. To this end, the following provides a nice 
> > solution, that doesn't rely on any OCaml-specific mechanism:
> > - support for "global" (i.e. not per-switch) opam variables, defined in 
> > ~/.opam/config, is added
> > - ~/.opam/config has a field that allows to initialise those (lazily) from the 
> > output of a command (if the command fails or doesn't exist, the variable is 
> > simply left uninitialised)
> > 
> > Based on this, we define, in the configuration file, an `ocaml-sys-version` 
> > global variable bound to the output of `ocamlc -vnum`. Then we add a 
> > constraint `available: ocaml-sys-version = _:version` to the `ocaml-system` 
> > package. This way, only the installable ocaml-system package is visible, and 
> > additionally, if it changes, existing opam mechanisms will trigger an up/down-
> > grade to the correct version. Also, still using existing mechanisms, `opam 
> > switch ocaml-system` will pick the right version, while `opam switch 4.02.3` 
> > will either pick `ocaml-base-compiler` (if `ocaml-system.4.02.3` is not 
> > available) or state that the choice is ambiguous and ask to choose between 
> > `ocaml-system` and `ocaml-base-compiler` otherwise. In case the variable is 
> > not properly defined, the `ocaml-system` package won't be available, but it 
> > won't break anything besides.
> > 
> > We also extend the way in which such a variable can be initially defined in 
> > opam, by supporting an `init` file (e.g. ~/.opamrc or /etc/opamrc) that allows 
> > to choose, at `opam init` time, many ~/.opam/config options and e.g. the 
> > initial repositories to use. Opam has a default, built-in init file that binds 
> > to https://opam.ocaml.org and defines `ocaml-sys-version`, but this is better 
> > separated from the code, and easier to override, than before. 
> > 
> > 
> > Hope this all makes sense, comments welcome. All of this is already working, 
> > with the proper automatic rewrites, and just needs a little more polish before 
> > merging into master.
> > 
> > Best,
> > Louis
> > 
> > 
> > ¹ Contrary to before, this never becomes a global variable and can be used in 
> > commands but never in the `available:` field.
> > ² The `provides:` field would be a more elegant solution to this, but causes 
> > more problem than it solves; design work to have it in opam has advanced — 
> > it's yet desirable for many reasons — but let's keep matters separate.
> _______________________________________________
> opam-devel mailing list
> opam-devel at lists.ocaml.org
> http://lists.ocaml.org/listinfo/opam-devel
>