[opam-devel] OPAM 1.3 roadmap

[Anil Madhavapeddy]

On 25 Feb 2015, at 10:57, Peter Zotov <whitequark at whitequark.org> wrote:
> 
> On 2015-02-25 13:48, Roberto Di Cosmo wrote:
>>> I'm less sure about the viability of recording installed files
>>> strictly -- I view thatas an advisory rather than enforcement-based
>>> mechanism.  The reason I like the "make ~/.opam a git store" is that
>>> its possible for applications to write directly into the store as they
>>> do right now, but still let us track changes precisely.  In fact, if
>>> we forbid subshells from writing into `~/.opam/.git`, this would be
>>> a production grade solution that also offers instant-rollback in case
>>> of compilation errors (no more waiting for a full recompilation of
>>> the original dependencies!).
> 
> Please don't make .opam a git store. Even with one snapshot, it means
> that my .opam would be 11G instead of 5.5G it currently is. What's worse
> is that every recompilation or upgrade would balloon the store even more.
> I can easily anticipate it filling the entirety of my 500GB drive,
> for example. That's absurd.
> 
> Also, git is pretty slow at handling many large binary files (which is
> the single worst workload you can use it for). Delta compression
> takes noticeable time on my blog, which has a mere 500MB of photos.

This needs some experimentation, but it would not be as naive as simply
shelling out to Git, as we have a complete implementation in pure OCaml.
It would also be optional, and history rebased regularly so as not to
grow too far (just as `git gc` does).  Binary compression can also be
sorted out asynchronously.

Adding this feature has a number of upsides in the future, particularly
for sharing OPAM installations in classroom environments, which has been
a very common request.

I'm almost certain it could be implemented outside of OPAM by adding
the right build-time hooks, which may be shareable with the sandboxing
infrastructure.  This discussion for a plugin API has come up several
times, and this the right time to add it now that we have several concrete
usecases (depext, sandbox, vcs)

-anil