[opam-devel] opam2{rpm,deb}

[Török Edwin]

On 05/03/2015 10:14 PM, Anil Madhavapeddy wrote:
> On 20 Mar 2015, at 23:22, Török Edwin <edwin+ml-ocaml at etorok.net>
> wrote:
>> 
>> Hi,
>> 
>> [On Anil's suggestion I'm moving the discussion to opam-devel. 
>> TL;DR: I am about to start writing opam2{rpm,deb} for my package
>> and its dependencies. Does anyone else need such a tool or want to
>> help?]
> 
> Apologies for the delay replying -- I continue to see significant
> interest in a tool like this for generating standalone installable
> packages easily for end-user tools.
> 
> There seems to be a basic tension in upstream OS packaging these
> days: to add all the libraries for every language as a package, or to
> just "vendor" third party libraries and build a standalone binary
> with fewer dependencies.  In OCaml's case the latter works great,
> just as with Go, due to the static linking.

I think opam2debian might be useful for this, but such packages won't be/aren't meant to be
accepted in upstream distributions.


> Out of interest, do you also need Ubuntu PPA (or the Fedora
> equivalent) support?  The issue with upstreaming libraries is the
> difficulty of managing updates, and so I'd also like to be able to
> build a complete PPA archive with the latest OPAM libraries.  This
> shouldn't be too difficult with a mechanical OPAM transformation, but
> I suspect there will be lots of versioning issues.

*Iff* Debian also had a PPA yes, otherwise its just easier to point people to just one 
place (either the OpenSUSE build service like you do for opam, or repositories hosted on your own domain).

I use the latter for my company, the tools to build repos (reprepro, createrepo) are fairly easy to use,
it is just that whole cycle of building packages / repositories / QA takes a full day's of work to reach
the desired result for just the 4 packages we have, and I can't see how that could scale up to the hundreds that OPAM has.
For that somehow the whole build process should be more incremental and more parallelizable, i.e. more like a Makefile, but I'm not sure
how well the current tools would support such a workflow. 

> 
> Yeah, the `findlib` file in OPAM was a first step to close the gap
> between OASIS and OPAM, and we can continue to add automated scripts
> to keep this information in sync.  It would be good to only depend on
> one package database for this translation, and OPAM is the outermost
> layer and so most appropriate.
> 
> 
> Yep, I think the last thing is key, since the OPAM version numbers
> also don't map perfectly onto the upstream packaging ones.  For
> Debian or Ubuntu, we'd need to bump the extra versions for any small
> change to the package, whereas OPAM doesn't track these small
> changes.
> 
> 
> I do agree with this all, and that it's the next important stage to
> make it easier to distribute OCaml applications upstream.  It's quite
> a bit of work, so perhaps we should transfer this design to the OPAM
> wiki to make tracking it easier.

Wiki page posted here: https://github.com/ocaml/opam/wiki/opam2%7Brpm,deb%7D

I'm known for overcomplicating/overengineering designs, so take this with a grain of salt and consider it more a wishlist
than a definite plan.

I think the short version is:
 * there is no canonical source of metadata, we should have tools to compare the existing sources though (opam, _oasis, debian/control and .spec) to let package maintainer
decide the correct metadata / to let upstream sync the metadata
 * we should have tools that track package (and it dependencies') lifecycle in various major distributions from review -> actual release to avoid dupplication of effort
or see where help is needed (e.g. such a tool would say that in fedora: opam -> ocaml-re APPROVED -> FE-NEEDSPONSOR jonludlam, or aspcud -> gringo -> clasp REVIEW PENDING)

>  Once there is some stability, we can start ticking off various bits of low hanging fruit like the lifecycle query tools, and figure out who can work on what component.

Yes I think small tools with less ambitious goals would be a good start. I'm mostly interesting in version constraints and C dependencies, so I put some ideas concerning that on the wiki too.

> 
> I'm personally keen on seeing this so that we can have binary
> distributions of Core/Async more easily, as well as distribute the
> MirageOS command-line more sensibly on Ubuntu and Fedora.
> 

Best regards,
--Edwin