[wg-parallel] About Lwt and Async

[Daniel Bünzli]

Le mercredi, 1 mai 2013 à 01:42, Yaron Minsky a écrit :
> That said, my objection to your proposal is essentially technical.
> Writing a layer that abstracts over Lwt and Async seems highly
> problematic.  

We agree on that, I don't like the functor approach. My take on that is to turn the problem upside down and instead of abstracting over the asynchronous primitives, provide libraries that are asynchronous in their own mode of operation independently of monadic concurrency libraries, and hence usable by any of them (or whatever asynchronous mechanism that may become prevalent in the future).   

For me the idea is that we can afford *library* programmers to suffer that pain --- basically hand made cps progamming, it's fun bun can be odd… However *application* (or end-user) programmers should not have to suffer and use the monadic programming library of their choice.  

Now the point to which that idea can be brought is unknown to me. One thing that is certain to me is that at least for codecs (which is not a small amount of libraries in my opinion), this can be done and should be done.
  
> I think it's hard to build such an abstraction that is performant,

Regarding codecs, the pattern I mention allows you to use a unique, fixed size, buffer for input and output -- perfect for Unix.{read,write}. There may however be a cost for cps that I'm not really able to assert -- but I guess you also pay that price if you work directly with a monadic library.

  
> reliable, easy to reason about,  

In fact the whole point of my wip on Fut [1] was to attach a clear denotational semantics to its values and primitives. But since some combinators have side effects (cancellation) it hinders equational reasoning. Still I think it's nice to have a clear semantics attached your combinators, something I find really lacking in both async and lwt. Maybe I'm special but I like to understand my abstractions, especially having a clear concurrency model of the runtime is important to me (and a simple one aswell, hence Fut's fifo queues).

> I don't understand what you mean by "application independent".  
By application *dependent* I mean related to the software architecture, design and constraints you choose to try to solve your specific problem -- maybe that's just non obviously reusable code.  
  
> There
> are of course lots of application-independent libraries written in
> Async (and Lwt too), from communication mechanisms like Async-RPC to
> code for setting up TCP servers and clients in the Tcp module, to
> logging facilities like Async's Log library. But these libraries all
> depend on the error handling, concurrency and performance semantics of
> Async.

These would be for me in the "concurrency patterns" I mentioned. So the question is how far can we go here by just plugin asynchronous agnostic libraries, and how painful it is (or not) for the end-user. I was actually intending to work on these questions beyond codecs (which is a solved problem for me) in my work on webs [2], but I'm on other things right now.

Best,

Daniel  

[1]  
http://erratique.ch/tmp/fut/Fut.html
https://github.com/dbuenzli/fut/blob/master/src/fut.mli

[2]
https://github.com/dbuenzli/webs