[wg-parallel] About Lwt and Async

[Stephen Weeks]

Async's approach shouldn't cause more select/epoll calls.  It is only
a different order in which the closures run.  It does require more
allocation, because one has to create a "job" to run later rather than
immediately run a closure.  But that's very small.  Plus, we have a
patch coming soon that will eliminate even that allocation, by reusing
the memory for jobs.

> Got it. I don't have a feel for the performance impact of such deferred
> scheduling, except that the difference between busy-spinning if there are
> outstanding requests, vs dropping into select/kqueue/epoll more frequently
> is very significant.
>
> For example, the Arakoon folks anecdotally reported a 20x performance loss
> between a direct Unix implementation of their database layer vs an Lwt
> one.  A loss that large can only be explained by context switching or
> pathological scheduling somewhere (given we know that Lwt doesn't result
> in a lot more allocation on the major heap).
> http://www.slideshare.net/eikke/arakoon (slide 14/15)
> (I'm CCing Romain, who might have details).
>
> Either way, this seems like a good time to establish a few simple
> microbenchmarks such as a TCP iperf or HTTPbench, just to have a few
> baseline numbers to evaluate such design decisions against.  I do believe
> that the Async 'drop to scheduler' behaviour is far easier to understand
> than Lwt's, but not if it costs an order of magnitude in I/O throughput.
>
> -anil
>
> _______________________________________________
> wg-parallel mailing list
> wg-parallel at lists.ocaml.org
> http://lists.ocaml.org/listinfo/wg-parallel