[wg-parallel] About Lwt and Async

[Yaron Minsky]

On Wed, May 1, 2013 at 9:00 AM, David Scott <dave.scott at eu.citrix.com> wrote:
>
> Hi,
>
> In the XenServer part of Citrix we're starting to use Core for some projects
> (and no doubt we'll incorporate Async into those soon) and we're also using
> Lwt, mainly via mirage. We haven't invested so much in either that it would
> be impossible to switch, although it will take a while: shipping code will
> often be maintained for around 5 years.
>
> I had a quick look at our development opam repo[1] and, at the package level
> at least, our stuff breaks down roughly into
>
>   * 25% C bindings for bits of Unix; traditional style; potentially blocking
> operations.
>   * 25% protocol implementations / format converters. These are a mix of
> traditional style, with some lwt creeping in (often kept separate using
> ocamlfind subpackages)
>   * 50% apps / services / tools. I'm expecting both Async and Lwt to start
> creeping in here.
>
> Our apps usually talk to each other via RPC so we can live with a mixture of
> Async apps and Lwt apps but it would obviously be simpler if we were able to
> standardise (eventually) on one API.
>
> The simple Lwt shim on top of Async Jeremie described would potentially be
> useful for us: in some of our less performance-critical apps we could
> clearly live with a bit of extra boxing. We're also not using very much of
> the Lwt library so even a very basic integration would still be useful.
>
> Our performance critical apps will use mirage and its ecosystem of
> libraries, all based on Lwt. If it were possible to port the kernel of Async
> to mirage then we could use the Async engine everywhere, and
> slowly port everything over to it.
>
> Cheers,
> Dave
>
> [1] https://github.com/xen-org/opam-repo-dev

I believe Anil is interested in moving Mirage to Async (Anil, can you
confirm?), and Jeremie and Stephen are going to do the reorganization
of Core so that there is a Core_kernel that's more portable than the
full Core, and Async's core (which we're going to rename to
Async_kernel) is going to depend only on Core_kernel.

As Jeremie said, the Lwt shim seems doable, and we'll look into that
as an intern project this summer.

I should say: I very much endorse Daniel's suggestion that codecs
should when possible be written in such a way as to be adaptible
between multiple engines.

As a side note, it looks like we're not the only community with this
problem.  There are some interesting discussions in the Scala
community about the mismatch between Akka futures and Finagle
futures.  It's all very reminisicent.

https://groups.google.com/forum/#!topic/akka-user/eXiBV5V7ZzE

We're also contemplating having an intern look as to whether Libuv
would be a reasonable way of getting Async to be more portable to
other places, including Windows.

y