[wg-camlp4] Working Group: the future of syntax extensions in OCaml, after camlp4

[Alain Frisch]

Dear caml-list,

There is a growing opinion that camlp4 is overly complex considering the 
benefits it brings to OCaml developers.  I would personally go as far as 
to say that the future of OCaml and the OCaml community would be 
brighter if camlp4 could be removed from our "basic ecosystem".  In 
particular, most of the current uses of camlp4 to create syntax 
extensions could probably be replaced by the new "-ppx" technology (see 
below) and small extensions to the compilers.

A lot needs to happens for this camlp4-free OCaml ecosystem to become a 
reality and we have to come up with a solid transition plan.  A new 
community-driven working group, chaired by Leo White and me, is being 
set up today to elaborate this plan.  If this topic is of interest to 
you, please join our mailing list:

   http://lists.ocaml.org/listinfo/wg-camlp4


Some of the most important tasks for our new working group will be:

  - Gather information from the community about the use of camlp4.
    (Please consider sharing information about your use of camlp4
    and camlp4-based extensions, even if you don't plan to participate
    to the working group discussions!)

  - Finalize support for -ppx and make technical proposals for
    extensions of the OCaml compilers in order to enable a transition of
    camlp4-based extensions to -ppx (in particular, we need to come up
    with a concrete syntax for generic extension points in the grammar).

  - Write some "canonical" examples of extensions based on -ppx and
    provide information and support to developers of extensions for
    switching from camlp4 to -ppx.

  - Discuss integration of -ppx with existing tools (findlib, build
    systems, etc).

  - Find a solution in the community for the future of camlp4
    (in particular, discuss how / how long / by who it will be
    maintained).

  - Discuss longer-terms plans beyond -ppx, including extra language
    support, to facilitate light syntactic meta-programming for OCaml
    (Leo has some clever ideas!).



The discussion on the mailing list will start in a few days, to give 
some time for interested people to join.  In the meanwhile, Leo has 
written a blog post to get the discussion started:

  http://www.lpw25.net/2013/01/23/camlp4-alternative-part-1.html

You can also read about -ppx:

  http://www.lexifi.com/blog/syntax-extensions-without-camlp4
  http://www.lexifi.com/blog/syntax-extensions-without-camlp4-lets-do-it

Some projects have already started to replace camlp4 by -ppx:

   bisect (supports both camlp4 and ppx since version 1.3)
   sedlex (unicode-friendly lexer generator, successor of ulex)
   omonad (syntax for monadic code, similar to pa_monad)


Many thanks to Anil Madhavapeddy and to OCamlLabs for setting up the 
working group and its mailing list!


Alain