[wg-camlp4] benchmarks

[Gerd Stolpmann]

Am 07.02.2013 21:39:39 schrieb(en) Thomas Gazagnaire:
> Hi all,
> 
> In my opinion as a *user* of camlp4, the main pitfall that I can see  
> (and which has not yet been addressed) is that it is really slowing  
> down the compilation process. To improve that, the usual methods are:
> 1. to use native syntax extensions
> 2. to statically link the syntax extensions into one preprocessor  
> binary

Well, Alain started with the suggestion to run all preprocessors as  
separate binaries. Of course, if you have many preprocessors or many  
files to preprocess, this will become slow. Under the most common  
assumption, though, namely that only a few files need preprocessing at  
all, this is no problem, and certainly the most flexible option.

Regarding dynloading. First, it's not only that native dynloading does  
not work on all platforms. Worse, not even bytecode dynloading works  
everywhere when it comes to loading external C libraries (which may be  
required here and there). Because of this, we should design without  
dynloading in mind, and at most consider this as an accelarator that is  
available on certain platforms only.

In my opinion, there should be ppx drivers (= something that turns a  
preprocessor into a program suited for -ppx). The simple default driver  
just pipes the parsetree through the preprocessors, and these are  
running as subprocesses. But it should also be possible to link a  
custom ppx driver for the not-so-frequent cases that the preprocessors  
are needed often. And finally, there could also be a ppx driver that  
dynloads.

I guess the details are not so complicated to figure out (e.g. define a  
module type a ppx preprocessor must satisfy, and have some utilities to  
"make this a program", "make this a dynloadable extension", and "bind  
several ppx processors together").

> Currently, 1. is very broken  (natdynlink doesn't work on all  
> platforms and most of the syntax packages do not install their native  
> libraries anyway) and 2. is not very flexible because of lack of  
> tooling, and so very rarely used.
> 
> So, if we decide to switch to a new preprocessing tool, I guess it's  
> important to get these hings right:
> * it should be easy enough for external tools to statically build  
> pre-processors.
> * the new preprocessor should be benchmarked on realistic examples to  
> see how much time we gain vs. camlp4. A good example might be mirage  
> which is using only pa_lwt and cstruct (or xen-api which is using  
> only ocaml-rpc, or core but that maybe too much syntax in one go).

Fully agreed to both points.

Gerd

> 
> Best,
> Thomas
> PS: I'm not speaking about the bootstraping time of camlp4, which is  
> indeed very painful, but which doesn't really affect the end-user  
> once the compiler is installed.
> _______________________________________________
> wg-camlp4 mailing list
> wg-camlp4 at lists.ocaml.org
> http://lists.ocaml.org/listinfo/wg-camlp4
> 
> 



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd at gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------