[ocaml-platform] Is it taking too long for OCaml software to become 4.03-compatible? Would release process changes help?

Thu Jun 30 16:41:29 BST 2016

Fabrice: indeed.

I thought of proposing a process for using opam-builder against trunk (eg.
swipe the state every two weeks and update the compiler used to the HEAD of
trunk at this moment only; the incremental capabilities are still useful to
monitor the effect of proposed fixes in opam packages), but I think that
for trunk fixing there are more delicate questions that arise (than at the
beta stage). For example, deciding when the trunk state is "good" for
testing, and deciding what to do with breakage report, as reporting them to
maintainers "too soon" is not always the right choice -- we may want to
revert the change in trunk instead, or improve its compatibility status.

I have a clear, simple proposal for the end of the release process: remain
in beta stage as long as there are OPAM packages that are considered
installable but fail to build with the beta. (Provided we have tools to
automate the addition of ocaml version bounds, the release manager would
have the avaibility to declare it done and simply mark all remaining
packages un-installable whenever they want to get the release out.)

As a first step, I would get all installable OPAM packages to build on
4.03.0, and I will try to push fixes and bounds, and motivate people to
participate to that effect in the following weeks, using the opam-builder
reports.

I'm interested in working to get more software to build on trunk, but I
don't have a clear proposal or process in mind yet, so I won't make any
specific proposal for now.

P.S.: I'm a bit worried by the "uses a fork of OPAM" part of opam-builder,
isn't there a risk of increased divergence over time and eventual
unusability? Could we integrate the salient features in opam proper?

On Thu, Jun 30, 2016 at 11:09 AM, Fabrice Le Fessant <
Fabrice.Le_fessant at inria.fr> wrote:

> The reason why OPAM-builder cannot monitor trunk is by design: to achieve
> almost real time on a 4-core computer for 5 versions of OCaml, opam-builder
> makes an extensive use of caching binary installations of dependencies.
> This cannot be done on a moving target like trunk, where you would have to
> recompile all packages (all versions, sometimes with different depends
> because of depopts) at every commit in the trunk, after clearing the cache.
> That's the reason why I only do it for official beta releases, and
> opam-builder takes two or three days to compile the repo.
>
> Le jeu. 30 juin 2016 à 16:48, Gabriel Scherer <gabriel.scherer at gmail.com>
> a écrit :
>
>> Let's clarify that I am not trying to blame maintainers of packages for
>> the issues that I highlighted. I believe that we are all doing our best
>> given the current processes and expectations, and I would like to fix the
>> process to improve the results. (In particular, I think that the beta
>> period for 4.03 was insufficient for maintainers to update their packages.)
>>
>> Yotam: I don't think a clear separation of responsibility (package
>> authors vs. "the entire community") is possible. The free software that
>> powers the OCaml ecosystem should be maintained collectively at all times.
>> It is particularly true in the context of adapting to a new OCaml version
>> that changed some things; it is often the case that the package authors are
>> not the most qualified to understand the required changes -- they know
>> their codebase well, but don't necessarily follow OCaml evolutions closely.
>> Early-adopters and enthusiasts, such as you and I, have a large role to
>> play in helping the ecosystem evolve gracefully by sending patches to fix
>> the bugs.
>>
>> Fabrice: again, it's not about who maintains the software (and I don't
>> think that the ocaml/ umbrella automatically guarantees more maintenance).
>> External contributors can easily fix software to work on trunk or new OCaml
>> versions -- I've done it for atdgen and Coqide and it was not much work.
>>
>> Fabrice: Indeed, I think that opam-builder is a great tool to put at the
>> forefront of such an effort. The current version
>>
>>   http://opam.ocamlpro.com/builder/html/report-last.html
>>
>> monitors the state of the OPAM repository for released OCaml versions.
>> Everything that is orange (except maybe for the "lint" column) is a bug
>> that we must collectively fix.
>>
>> I will try to experiment more with opam-builder (understand its error
>> explanations better and try to fix a part of it to see what the experiment
>> is) before suggesting any specific process.
>>
>> On Thu, Jun 30, 2016 at 9:32 AM, Yotam Barnoy <yotambarnoy at gmail.com>
>> wrote:
>>
>>> I've also been bitten by this issue with 4.03. Specifically, Merlin
>>> took about 2 weeks post-release to be ready, making it much harder to
>>> program in 4.03, and I had a dependency chain that needed OASIS to
>>> build.
>>>
>>> I don't think we can avoid having a specialized pre-release period for
>>> getting the packages up and running. I noticed many of these breakages
>>> before 4.03 was released, but package authors aren't responsive to a
>>> pre-release breakage.
>>>
>>> It's important to prioritize though -- those packages that have the
>>> most reverse-dependencies and the ones that are most popular, should
>>> be the highest priorities to fix. OPAM has this info, and just as the
>>> https://opam.ocaml.org/packages/index-popularity.html page shows the
>>> most popular packages, I think it should pull their build-state for
>>> the latest OCAML release version and highlight them if they're broken.
>>> Speaking of which, it would be nice if the opam webpage also had a
>>> 'sort by reverse dependencies' list.
>>>
>>> One schedule that perhaps could work would be to have a
>>> pre-platform-release month. Within this month, package authors would
>>> have the first 2 weeks to fix their packages. After these 2 weeks,
>>> there would be a rallying cry to fix packages sorted by priority
>>> (highest reverse-dependencies & highest popularity), and the entire
>>> community would be mobilized to help in the effort. Once a
>>> "sufficient" number of the highest priority packages are fixed, the
>>> full release can take place, but the work can be ongoing.
>>>
>>> -Yotam
>>>
>>> On Thu, Jun 30, 2016 at 9:17 AM, Daniel Bünzli
>>> <daniel.buenzli at erratique.ch> wrote:
>>> > Le jeudi, 30 juin 2016 à 13:58, Fabrice Le Fessant a écrit :
>>> >> You can ask this kind of diligence from sub-contractors, not from
>>> open-source contributors, especially when a package is maintained only by
>>> one developer on his/her sparetime.
>>> >
>>> > I'm not asking anything. I'm only commenting on the fact that Sylvain
>>> said that the time response was not long. It wasn't indeed but release time
>>> is not the right time point to consider. The right time point to consider
>>> is the rc betas.
>>> >
>>> >> Or if you think these components are key to the infrastructure, why
>>> not move their development to github.com/ocaml/ (
>>> http://github.com/ocaml/) with a larger team of maintainers, as it was
>>> done for OPAM ?
>>> >
>>> > I personally don't care as I'm not interested in oasis. But many other
>>> programmer seem to use it so it's a pre-requesite for them to be able to
>>> work in a given release and if this doesn't happen during the betas then we
>>> get less testing. Regarding oasis' development model I'll let users of
>>> oasis decide on what is best them.
>>> >
>>> > Best,
>>> >
>>> > Daniel
>>> > _______________________________________________
>>> > Platform mailing list
>>> > Platform at lists.ocaml.org
>>> > http://lists.ocaml.org/listinfo/platform
>>> _______________________________________________
>>> Platform mailing list
>>> Platform at lists.ocaml.org
>>> http://lists.ocaml.org/listinfo/platform
>>>
>>
>> _______________________________________________
>> Platform mailing list
>> Platform at lists.ocaml.org
>> http://lists.ocaml.org/listinfo/platform
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ocaml.org/pipermail/platform/attachments/20160630/a0ceeb6b/attachment.html>