[opam-devel] docker hub update?

[Török Edwin]

On 04/11/2016 14:44, Anil Madhavapeddy wrote:
> The Docker packages up on ocaml/opam are still somewhat experimental,
> so please do bear with me as I rearrange things around to make it
> work more efficiently.

Hi Anil,

They're already quite useful, thanks a lot for creating them!

>  On the other hand, this is a great time to
> bring up your usecases for them
> as well so I can make breaking
> changes in the Hub namespace before announcing them properly.
> 
> There are a few goals for the Docker Hub containers:
> 
> - make it easy to publish an OCaml-based application (such as your
> Skylable LibreS3, or Mirage, Irmin, Flowtype, Hack, Infer, etc etc)

I'd love to be able to do this, but I had to jump through some hoops to get my image size down,
e.g. by installing, compiling and removing the development tools in a single RUN line.
This got my image down from ~1GB to 85 MB, but I can't use an ocaml/opam base image for this (since removing files doesn't shrink the image download size).

Don't know if it'd be possible to squash some layers on the Docker Hub, or build in 2 phases: 
one container to build a .deb/.apk/.rpm, another container to install that on a minimal base image
(using a package to make sure all system gets installed, like libev, etc.).

For this use-case it would be useful to have a base image that doesn't change every day, but only when ocaml/opam/opam depext itself changes.

> -> continuous integration with a variety of distros and compiler version
> variants

My current use case fits roughly here. I want to test that I got all opam deps and system deps properly
specified, i.e. I want to build in a 'clean' environment.
Docker's cache comes in handy to avoid rebuilding all the deps when the opam file doesn't change.
Testing on multiple distros is meant to be as simple as changing the FROM line (and fixing opam files where needed):
https://gist.github.com/edwintorok/0149412fb55f58c0b7c5eb605adaa8fa

Also for testing a local opam remote, although SSH agent forwarding is a bit tricky:
https://gist.github.com/edwintorok/b8bf92acc6ba9aecadcb25f0f492e8e2

> - development environments for OCaml

I thought about this, but I'd personally wait until Docker gains the ability to run containers as non-root.

> - ocaml/opam-base -
> provides an installed OPAM binary, but _not_ initialised, and has the
> capability to volume mount an OPAM repository from which it can
> initialise the local base without any data being held in the
> container.

Yes! This would be similar to Debian containers, where you must run apt-get update:
otherwise 'apt-get install' finds no packages.

> - ocaml/opam - embeds an OPAM repository checkout and
> works as-is right now

Not sure if rebuilding the image or using ONBUILD would be better here.
For local usage ONBUILD might get you something fresher without having to pull everyday from Docker.

On Docker Hub the shared checkout would probably save time in containers that depend on this.

>> E.g. 'ocaml/opam:latest' and 'ocaml/opam:debian' was last updated 9
>> days ago according to docker hub, but then some images were updated
>> a few hours ago. That looks confusing, would it be possible to
>> prioritize building 'ocaml/opam:latest'?
> 
> It's difficult to prioritise unfortunately -- Docker (my employer) is
> sponsoring the builds so that we have up to 100 in parallel, and they
> are pushed when they were built.

Nice!

> 
> I was on vacation last week and it was an errant cron job that did
> the push that you saw, but the eventual plan is to have a cron job
> that rebuilds every day.

Ok, in case some limits are hit would it be possible to pick a few images (like :latest) that are triggered first,
and the rest are triggered later?

Best regards,
-- 
Edwin Török | Co-founder and Lead Developer

Skylable open-source object storage: reliable, fast, secure
http://www.skylable.com