[ocaml-infra] Staging Build and Deployment of ocaml.org

[David Sheets]

On Mon, Mar 17, 2014 at 11:55 AM, Anil Madhavapeddy <anil at recoil.org> wrote:
> On 17 Mar 2014, at 11:49, David Sheets <sheets at alum.mit.edu> wrote:
>
>> Our present deployment workflow unnecessarily relies on a single build
>> environment running on the same (?) machine as the serving system.
>>
>> ocaml.org is entirely static with a refresh on the order of minutes. I
>> propose we take advantage of this feature to move to a fully staged
>> build and deployment solution.
>>
>> Specifically, I'd like to see:
>>
>> 1. Generated assets and build logs publicly accessible
>> 2. Build environment specification (e.g. "this stock Ubuntu image").
>> 3. History
>>
>> I'd like to suggest the following design from back-to-front:
>>
>> 1. Travis CI builds ocaml.org on commit hook to source repository
>> 2. Travis CI pushes successful builds to static asset repository
>> 3. Server checks out static asset repository on commit hook
>
> This seems like a fairly standard workflow for Travis, as long as the
> run can always finish within a 50 minute time limit.  I'd be in favour
> of splitting the static HTML staging from the generation of the static
> HTML for the reasons you list above.
>
>> If we would like to stage development and production, the following may satisfy:
>>
>> 1. Travis CI builds ocaml.org on commit hook to source repository
>> 2. Travis CI pushes successful builds to static assets
>> 3. beta.ocaml.org checks out static asset repository on commit hook to
>> static assets
>> 4. ocaml.org checks out static asset repository on special commit hook
>> to static assets
>
> The only potential issue with this are hardcoded references to http://ocaml.org
> within the staged site, which would prevent a beta.ocaml.org site from
> working correctly.  If domain names are not hardcoded, then it should be
> fine.

In my opinion, all such references are bugs and beta.ocaml.org would
be operating correctly if these references are misdirected there. One
of the nice things about having an image of the static assets deployed
is that, if we stick to polyglot (X)HTML in our generated pages, we
can trivially check, at any time, for broken links and violations of
link policies like this one.

>>
>> Where "special commit hook" is a commit by a non-Travis user which
>> contains as its commit message a human/machine-readable description of
>> the deployment. These commits can themselves be consumed to generate a
>> site change log which is exposed on the site.
>>
>> This approach should help us achieve faster iteration on the
>> mechanical components of the site, ease build debugging, ease build
>> reproduction, and offer a pre-built image of the site's assets.
>>
>> What do you think?
>
> Great idea. I'd very much like to minimise the amount of non-publically
> triage-able code on the ocaml.org VMs, in order to lock them down more
> and more as time goes on.
>
> -anil