[opam-devel] [RFC] OPAM files as *the* OCaml package metadata format

[Louis Gesbert]

Le jeudi 14 avril 2016, 14:30:19 Daniel Bünzli a écrit :
> Le jeudi, 14 avril 2016 à 10:04, Louis Gesbert a écrit :
> > Yup, 2.0~alpha is getting rolled out. I'll be grateful if you are ready to
> > take the risk, as you have been most helpful with your reports on previous
> > releases.
> 
> What kind of time frame to envision ? Should I write topkg directly against
> 2.0 ? For example the existence of x-fields may have an impact on the
> actual package description API (I'd like to specify the location of the
> readme, change log and license files there rather than in the topkg package
> description as this may be useful to other consumers, e.g. doc indexers)
> and topkg's description API should not be changed too often.

This sounds nice.
I would release the alpha just now, but since I'm taking a few days off 
(sunday-thursday), I'll wait until I am back. As for the final version, it's 
hard to predict, but I'd say it should be ready by the summer.

> Also depending on what 2.0 the tool provides some aspects of topkg may be
> simplified. Namely I need #2446 (opam query) to avoid an ipc call from the
> package description to topkg the tool on pinned topkg packages (see note
> [1] if you are interested why).

The work on 2.0 so far has been focused on the heavy-lifting, API rehauls, and 
feature-enabling design changes; the alpha is _not_  a feature freeze, and I 
expect to spend most of my time adding the smaller, but most useful features 
in the coming weeks (besides the inevitable bug fixing of course).

`opam query` is not here yet but it's nothing difficult and will be added.

> I think it would be nice to have an independent library with simple deps
> that handles the data formats (e.g. opam-installer should use this). Anil
> suggested somewhere else that opam-lib as it stands is not so easy to
> install due to the dependency cone.

Yes, I have had requests for this already. For now the dependencies for 
building `opam-lib.format` are:
- unix
- re
- ocamlgraph
- jsonm

Jsonm is not used per se, but we bind it in our set and map functors. 
ocamlgraph is only needed by the parallel processing engine, which could be 
moved to a different lib (it's bound in OpamPackage.Graph, but that can 
probably be easily changed as well).
The main problem may actually be that there is no separate opam package ?

> > - opam now puts all its metadata into a .opam-switch/ subdir below the
> > switch prefix. This is cleaner.
> 
> Why is the directory hidden ?

No strong opinion on this, it's just how is commonly done for metadata dirs, 
and makes the prefix look like a standard one. It could also actually be 
outside the prefix. Side-note: switch prefixes are still by default directly 
below ~/.opam, which means you can't e.g. name a switch "repo" since that name 
is used by opam...

> If I'm not mistaken these two things do not help me at all with the request
> which is to be able to handle non opam-managed scenarios. Basically the
> workflow I see for for OS package managers and topkg packages is the
> following:
> 
> 1) Download the distribution archive of the package.
> 2) Build the package according to the build instructions of the OPAM file
> present in the archive 3) Process the generated OPAM install file for
> installing the package (e.g. using opam-installer).
> 
> Now 3) should install the OPAM file present in the archive aswell so that
> tooling external to opam can easily access package metadata stored in OPAM
> files.

I see. There is not much required from opam then, maybe just automatic 
installation (or linking) of the opam files to a standard location.

> > Unless you use overlay files (`files/xxx` that get added to the archive),
> > which should really only be used for hot-patching in the repo and not on
> > development packages, you can now put everything within a single opam
> > file:
> > 
> > - For the description, use a `descr:` field, and a simple string. The new
> > `"""`-delimited syntax for strings makes this more convenient.
> 
> Would it please be possible to split `descr` into `synopsis` (the one-line
> tagline) and `descr` (the long description) ?

That might be more explicit indeed, but makes descr-file and opam-included 
descr further apart (both are still supported at the moment, with a logged 
warning if both are present). 

> > > Build tool link helpers, documentation indexer, finder and viewers, etc.
> > 
> > The repository maintainers will probably have to put some guidelines on
> > what should or should not be within these files, when they are part of
> > the official repository ?
> 
> Which files are you talking about ?

Sorry, I was speaking about the `x-*` fields in the package definition 
("opam") files.

Best,
Louis