[wg-camlp4] Feedback to Leo's post from Jake

[Anil Madhavapeddy]

Camlp4 slows down my projects, makes them more complex to build, and more prone to hard-to-track failures (due to buggy extensions generating invalid AST fragments occasionally).  It also makes it hard to build reliable IDE integration when you have to deal with an extensible grammar.

I love using it as an extension mechanism, and I hate using it day-to-day for every single bit of code that I write.

Alain and Leo are proposing separating extensible grammars from AST transformations, which seems sensible to me.  Can you be more specific about what you envisage a simple API would look like?

-anil

On 25 Jan 2013, at 16:48, Hongbo Zhang <hongboz at seas.upenn.edu> wrote:

> Hi Anil,
>   I am just wondering why not expose a simple API on top of  Camlp4
> instead of rolling a new mechanism? The advanced users can still have
> advanced features.
> 
> 
> On Fri, Jan 25, 2013 at 11:34 AM, Anil Madhavapeddy <anil at recoil.org> wrote:
>> I summarised some of the Twitter conversation at this URL, which has the thread:
>> http://twitter.theinfo.org/294459288762601472
>> 
>> Jake's last point is perhaps most important: "surveying what exists then supporting only common cases is philosophically backward for an extension mechanism".
>> 
>> While camlp4 is a powerful extension mechanism, it's perhaps been too successful.  Some of the common extensions have been used for years now without being integrated back into the core compiler suite, to the point that it's almost impossible to write a real application without using camlp4.
>> 
>> I view the current effort as a selection process for the successful extensions from camlp4, and building lightweight alternatives that reduce the reliance on the larger mechanism.  This makes the process of integrating other tools such as build, IDE integration and documentation much easier.  For example, we're currently working on a new ocamldoc that generates unified cross-references, and would like to separately render type_conv-generated functions to avoid cluttering the output. This is easy with annotations+ppx (assuming that type_conv's `with` is added to the default grammar).
>> 
>> There's no reason why camlp4 cannot continue to co-exist with some of these alternative mechanisms, but our addiction to it really ought to be reduced.
>> 
>> Another point he raises is that Leo's blog post "misses important camlp4 use cases. e.g. nested quot/antiquot, exts to pattern matching".  Some extensions (such as bitstring) are very naturally packaged as camlp4 syntax extensions, and not entirely solved by the ppx scheme.  It would be good to understand if this is outside of the scope of ppx, or on the list to solve.
>> 
>> (And thanks for the comments Jake; as I noted on Twitter, it was his fantastic blog series [1] and ICFP tutorial [2] that really unlocked the mysteries of camlp4 for me).
>> 
>> [1] http://ambassadortothecomputers.blogspot.co.uk/search/label/camlp4
>> [2] http://nicolaspouillard.fr/talks/cufp-metaprogramming-tutorial-slides.pdf
>> 
>> -anil
>> _______________________________________________
>> wg-camlp4 mailing list
>> wg-camlp4 at lists.ocaml.org
>> http://lists.ocaml.org/listinfo/wg-camlp4
> 
> 
> 
> -- 
> -- Regards, Hongbo
>