[wg-camlp4] Time for a summary?

[Yaron Minsky]

Thanks Leo!  This is very helpful.  Maybe we should throw this up on a
wiki page somewhere, and keep it up to date as proposals show up.

A few notes of my own below.

On Wed, Feb 6, 2013 at 1:11 PM, Leo White <lpw25 at cam.ac.uk> wrote:
> c) Sebastien Mondet suggested that run-time types or dyntype might be
> sufficient without ppx for many extensions.
>
> Markus Mottl pointed out that there are runtime performance
> implications for using those solutions.

I strongly agree with Markus on this.  Run-time types are lovely, but
we really need something more efficient for the vast majority of our
camlp4 needs.  Using a comparison function constructed with run-time
types, for example, would be a disaster performance wise.  We've done
experiments internally, and the performance hit is absolutely
crushing.

> d) Jeremy suggested that rather than implementing extensions as AST
> transformers that operate over the whole AST they might be implemented    as
> transformers only for the part of the AST that they were to operate    on.
> So an extension used like this:
>
>      (:perform
>          x <-- m;
>          y <-- n;
>          return (x y))
>
>    would be implemented as a function with type:
>
>      val perform : Parsetree.expression -> Parsetree.expression
>
>    He suggests that this may help with scoping, safety and
> compositionality.
>
>    Alain pointed out that some legitimate uses of ppx don't work nicely
> with the constraint that they can operate only on a marked fragment. He
> also pointed out that handling extensions as those extensions are found
> within the source forces a top-down expansion order, which is not
> necessarily desirable.

I wonder if this is a place where we can have it both ways: have an
underlying ppx mechanism that is a whole-AST transformation; and
expose libraries that support more disciplined, scoped
transformations.  We could strongly encourage the latter, but still
allow the former when necessary.

> e) I suggested an extension to Jeremy's proposal. This involves giving the
> expansion functions in their own kind of file (".mlq" - compiled to
> ".cmq"), referring to them using a namespace mechanism, and then having
> the compiler itself perform the expansion of the extensions.
>
>    There are more details in my blog post:
>
>      http://www.lpw25.net/2013/02/05/camlp4-alternative-part-2.html
>
>    Since such a solution would take a while to implement, and since moving
> an extension from ppx to it would be trivial, I proposed using ppx for
> the short/medium term.
>
>    I suggested that such a mechanism might improving tooling and make it
> easier for average users to use extensions.
>
>    Both Gabriel and Alain suggested that the use of namespaces might be
> unnecessary and too heavyweight. Alain also questioned the need for
> special ".mlq" files.

I do like the idea of eventually getting to a name-spaced solution.
In our world, there are really two cases: one, where we really just
want to extend the language, and we're happy to specify which language
we're using fairly broadly at the build-system level.  For such
things, namespaces do not seem helpful.

The other case is where you want to invoke a particular,
special-purpose syntax extension for just one file.  There, having
something that would allow the specification of the extension in the
file, ideally with some integration with namespaces, seems lovely.
That said, I think it's a less important use-case in the current
world.

I wonder, does anyone on the list of have much experience with
Scheme's approach to macros?  There's been a lot of work there,
covering everything from hygeine to integration with module systems.
It would be great to leverage off of that experience.

y