[wg-camlp4] Feedback to Leo's post from Jake

[Anil Madhavapeddy]

I summarised some of the Twitter conversation at this URL, which has the thread:
http://twitter.theinfo.org/294459288762601472

Jake's last point is perhaps most important: "surveying what exists then supporting only common cases is philosophically backward for an extension mechanism".

While camlp4 is a powerful extension mechanism, it's perhaps been too successful.  Some of the common extensions have been used for years now without being integrated back into the core compiler suite, to the point that it's almost impossible to write a real application without using camlp4. 

I view the current effort as a selection process for the successful extensions from camlp4, and building lightweight alternatives that reduce the reliance on the larger mechanism.  This makes the process of integrating other tools such as build, IDE integration and documentation much easier.  For example, we're currently working on a new ocamldoc that generates unified cross-references, and would like to separately render type_conv-generated functions to avoid cluttering the output. This is easy with annotations+ppx (assuming that type_conv's `with` is added to the default grammar).

There's no reason why camlp4 cannot continue to co-exist with some of these alternative mechanisms, but our addiction to it really ought to be reduced.

Another point he raises is that Leo's blog post "misses important camlp4 use cases. e.g. nested quot/antiquot, exts to pattern matching".  Some extensions (such as bitstring) are very naturally packaged as camlp4 syntax extensions, and not entirely solved by the ppx scheme.  It would be good to understand if this is outside of the scope of ppx, or on the list to solve.

(And thanks for the comments Jake; as I noted on Twitter, it was his fantastic blog series [1] and ICFP tutorial [2] that really unlocked the mysteries of camlp4 for me).

[1] http://ambassadortothecomputers.blogspot.co.uk/search/label/camlp4
[2] http://nicolaspouillard.fr/talks/cufp-metaprogramming-tutorial-slides.pdf

-anil