[wg-camlp4] benchmarks

[Alain Frisch]

On 2/7/2013 9:39 PM, Thomas Gazagnaire wrote:
> In my opinion as a *user* of camlp4, the main pitfall that I can see
> (and which has not yet been addressed) is that it is really slowing
> down the compilation process.
 >
> So, if we decide to switch to a new preprocessing tool, I guess it's important to get these hings right:
> * it should be easy enough for external tools to statically build pre-processors.
> * the new preprocessor should be benchmarked on realistic examples to see how much time we gain vs. camlp4. A good example might be mirage which is using only pa_lwt and cstruct (or xen-api which is using only ocaml-rpc, or core but that maybe too much syntax in one go).

It is certainly a good idea to do some benchmarks with the -ppx, I'll 
try to find some time to do so.

*If* performance is not satisfactory,  one should then identify the 
respective contributions of:

  1. Marshaling / demarshaling AST on the process boundaries.
  2. Spawning new processes.
  3. Traversing the AST with the ast_mapper class (overhead of method 
dispatch) and rebuilding a new AST in memory.

I doubt 3 would be a problem even with a lot of ppx rewriters, 
especially because they can be native code programs on platforms which 
support ocamlopt (even if native dynlink is not supported). *If* 1 or 2 
become a bottleneck,  one should indeed consider ways to avoid them.

The current version of ast_mapper:

http://caml.inria.fr/cgi-bin/viewvc.cgi/ocaml/trunk/parsing/ast_mapper.mli?view=markup

provides a very simple "registration" API:

=========================================================================
val register_function: (string -> (string list -> mapper) -> unit) ref

val register: string -> (string list -> #mapper) -> unit

     (** Apply the [register_function].  The default behavior is to run
         the mapper immediately, taking arguments from the process
         command line.  This is to support a scenario where a mapper is
         linked as a stand-alone executable.

         It is possible to overwrite the [register_function] to define
         "-ppx drivers", which combine several mappers in a single
         process.  Typically, a driver starts by defining
         [register_function] to a custom implementation, then lets ppx
         rewriters (linked statically or dynamically) register
         themselves, and then run all or some of them.  It is also
         possible to have -ppx drivers apply rewriters to only specific
         parts of an AST.  *)
=========================================================================

The main program of a typical ppx tool would look like:


let mapper _args =
    object
       inherit Ast_mapper.mapper
       method! expr = ...
    end

let () = Ast_mapper.register "foo" mapper


When linked as a stand-alone program, this produces something which can 
be readily used as a -ppx rewriter.  But it is also possible to create 
"ppx drivers" which link code as above statically or dynamically (and 
then decide to run it or not).  It would even be possible to provide a 
hook in the compiler so as to make it possible (through compiler-libs) 
to link statically custom versions of ocamldep.opt / ocamlc.opt / 
ocamlopt.opt with statically linked "ppx rewriters" (avoiding any 
marshaling of the AST or any extra process).  I can imagine that for 
huge code base, this could be considered.


Alain