[wg-camlp4] benchmarks

[Gerd Stolpmann]

Am 11.02.2013 11:45:57 schrieb(en) Gabriel Scherer:
> On Mon, Feb 11, 2013 at 11:31 AM, Anil Madhavapeddy <anil at recoil.org>  
> wrote:
> > It should work just fine with ppx, and has the advantage of  
> reducing the number of ocamlfind calls per-file (which is a  
> considerable overhead on large projects, since it has to scan the  
> META directories on every invocation).
> 
> That's of course rather orthogonal to the present discussion, but have
> you been in contact with Gerd to see if some caching solution would be
> possible? Intuitively it feels like ocamlfind could easily maintain a
> cache in /var/run/blah and only clean it on install/remove commands.

As a system engineer I think this is overkill. ocamlfind only loads the  
META files actually needed (it does not "scan"), and loading a file  
already in the page cache is a matter of microseconds. Even when you  
have 100 packages this should be hardly noticeable. Such a cache can  
even make things worse, because loading once a large cache can be more  
costly than a few META files (simply because of the data volume).

What is probably more a problem when you have many packages: This  
generates many -I paths, and the compiler has to search for modules. I  
don't know how this is implemented, but if it is the naive way, this is  
an O(n*m) algorithm (n=number of -I directories, m=number of modules to  
look up).

Gerd


> (An external process invocation would still make some overhead so in
> you're case you'd still have to go all the way, but that could improve
> the threshold at which such ~hacks become interesting.)
> 
> On Mon, Feb 11, 2013 at 11:31 AM, Anil Madhavapeddy <anil at recoil.org>  
> wrote:
> > On 11 Feb 2013, at 10:06, Alain Frisch <alain.frisch at lexifi.com>  
> wrote:
> >>
> >> For a code base like Jane Street's where many "extensions" have to  
> be used everywhere, I suspect that the "independent" ppx processes  
> might become an issue (or maybe not, if compared to camlp4), but the  
> following solutions would work:
> >>
> >> - Statically linking a "big rewriter" called with -ppx.
> >
> > This worked well in the case of Mirage; in the monolithic  
> repository, we link together all the camlp4 extensions into a single  
> big one, and invoke that across all files.
> >
> > It should work just fine with ppx, and has the advantage of  
> reducing the number of ocamlfind calls per-file (which is a  
> considerable overhead on large projects, since it has to scan the  
> META directories on every invocation).
> >
> > There's also the advantage in camlp4 that this checks that syntax  
> extensions do not conflict. This is less of an issue with ppx,  
> thankfully.
> >
> > -anil
> > _______________________________________________
> > wg-camlp4 mailing list
> > wg-camlp4 at lists.ocaml.org
> > http://lists.ocaml.org/listinfo/wg-camlp4
> 
> 



-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd at gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------