[wg-camlp4] benchmarks

[Hongbo Zhang]

On 2/7/13 4:22 PM, Gerd Stolpmann wrote:
> Am 07.02.2013 21:39:39 schrieb(en) Thomas Gazagnaire:
>> Hi all,
>>
>> In my opinion as a *user* of camlp4, the main pitfall that I can see 
>> (and which has not yet been addressed) is that it is really slowing 
>> down the compilation process. To improve that, the usual methods are:
>> 1. to use native syntax extensions
>> 2. to statically link the syntax extensions into one preprocessor binary
>
Fan encourages static linking to native code, that's to say "linking all 
your plugins as much as possible", unlike p4 the linking does not have 
any side effects, it registers the features it could provide,  each file 
itself tells what features it want to use.
> Well, Alain started with the suggestion to run all preprocessors as 
> separate binaries. Of course, if you have many preprocessors or many 
> files to preprocess, this will become slow. Under the most common 
> assumption, though, namely that only a few files need preprocessing at 
> all, this is no problem, and certainly the most flexible option.
>
> Regarding dynloading. First, it's not only that native dynloading does 
> not work on all platforms. Worse, not even bytecode dynloading works 
> everywhere when it comes to loading external C libraries (which may be 
> required here and there). Because of this, we should design without 
> dynloading in mind, and at most consider this as an accelarator that 
> is available on certain platforms only.
>
> In my opinion, there should be ppx drivers (= something that turns a 
> preprocessor into a program suited for -ppx). The simple default 
> driver just pipes the parsetree through the preprocessors, and these 
> are running as subprocesses. But it should also be possible to link a 
> custom ppx driver for the not-so-frequent cases that the preprocessors 
> are needed often. And finally, there could also be a ppx driver that 
> dynloads.
>
> I guess the details are not so complicated to figure out (e.g. define 
> a module type a ppx preprocessor must satisfy, and have some utilities 
> to "make this a program", "make this a dynloadable extension", and 
> "bind several ppx processors together").
>
>> Currently, 1. is very broken  (natdynlink doesn't work on all 
>> platforms and most of the syntax packages do not install their native 
>> libraries anyway) and 2. is not very flexible because of lack of 
>> tooling, and so very rarely used.
>>
>> So, if we decide to switch to a new preprocessing tool, I guess it's 
>> important to get these hings right:
>> * it should be easy enough for external tools to statically build 
>> pre-processors.
>> * the new preprocessor should be benchmarked on realistic examples to 
>> see how much time we gain vs. camlp4. A good example might be mirage 
>> which is using only pa_lwt and cstruct (or xen-api which is using 
>> only ocaml-rpc, or core but that maybe too much syntax in one go).
>
> Fully agreed to both points.
>
> Gerd
>
>>
>> Best,
>> Thomas
>> PS: I'm not speaking about the bootstraping time of camlp4, which is 
>> indeed very painful, but which doesn't really affect the end-user 
>> once the compiler is installed.
>> _______________________________________________
>> wg-camlp4 mailing list
>> wg-camlp4 at lists.ocaml.org
>> http://lists.ocaml.org/listinfo/wg-camlp4
>>
>>
>
>
>