[opam-devel] [RFC] OPAM files as *the* OCaml package metadata format

[Daniel Bünzli]

Le jeudi, 14 avril 2016 à 10:04, Louis Gesbert a écrit :
> Yup, 2.0~alpha is getting rolled out. I'll be grateful if you are ready to
> take the risk, as you have been most helpful with your reports on previous  
> releases.

What kind of time frame to envision ? Should I write topkg directly against 2.0 ? For example the existence of x-fields may have an impact on the actual package description API (I'd like to specify the location of the readme, change log and license files there rather than in the topkg package description as this may be useful to other consumers, e.g. doc indexers) and topkg's description API should not be changed too often.

Also depending on what 2.0 the tool provides some aspects of topkg may be simplified. Namely I need #2446 (opam query) to avoid an ipc call from the package description to topkg the tool on pinned topkg packages (see note [1] if you are interested why).  

> The granularity of opam libs for manipulating these files has been much
> improved, but it could probably still be improved. Feel free to suggest such  
> changes (OpamFile should probably be split; it would be better to do these  
> _before_ everyone migrates to the new APIs). Also, the data handling is done  
> through the new lens-like OpamFormat module (in lib opam-format).

I think it would be nice to have an independent library with simple deps that handles the data formats (e.g. opam-installer should use this). Anil suggested somewhere else that opam-lib as it stands is not so easy to install due to the dependency cone.

  
> > 1. Where should those be installed in `lib:` (like METAs) or `etc:` (seems
> > more logical) or … ?
>  
> Two things:
> - opam now puts all its metadata into a .opam-switch/ subdir below the switch  
> prefix. This is cleaner.

Why is the directory hidden ?  
  
If I'm not mistaken these two things do not help me at all with the request which is to be able to handle non opam-managed scenarios. Basically the workflow I see for for OS package managers and topkg packages is the following:

1) Download the distribution archive of the package.
2) Build the package according to the build instructions of the OPAM file present in the archive
3) Process the generated OPAM install file for installing the package (e.g. using opam-installer).

Now 3) should install the OPAM file present in the archive aswell so that tooling external to opam can easily access package metadata stored in OPAM files.
  
> Unless you use overlay files (`files/xxx` that get added to the archive),
> which should really only be used for hot-patching in the repo and not on  
> development packages, you can now put everything within a single opam file:
>  
> - For the description, use a `descr:` field, and a simple string. The new  
> `"""`-delimited syntax for strings makes this more convenient.

Would it please be possible to split `descr` into `synopsis` (the one-line tagline) and `descr` (the long description) ?  
  
> > Daniel:
> > 1) There's no support at the moment for such directories in OPAM install
> > file
>  
> This can easily be added if we get a consensus

FWIW I'm not interested in arguing where this should live, I have already too many other things to decide on. Just tell me where these OPAM files should get installed and provide me with support for the location in OPAM install files...

> > Build tool link helpers, documentation indexer, finder and viewers, etc.
> The repository maintainers will probably have to put some guidelines on what
> should or should not be within these files, when they are part of the official  
> repository ?

Which files are you talking about ?

Best,  

Daniel

[1] The reason for this is that topkg the lib, that packages depend upon, has no dependency, hence no opam-lib.  

On pins I need to be able to access the fields of the opam file to perform the source watermaking process that is performed on releases (which handles e.g. version substitution, but also allow substitution of variables with the value of arbitrary opam field).  

The idea is that if you have topkg-care installed when you pin, which has the topkg tool and depends on opam lib, the watermarking process will be performed correctly: the package description calls topkg the tool to get the field (otherwise we just substitute with undefined and warn). Now of course if #2446 exists I can just call opam and always be correct.