[wg-camlp4] Matching on concrete syntax (was: Re: Camlp4 uses)

[Alain Frisch]

On 3/29/2013 4:12 PM, Gabriel Scherer wrote:
> That is better indeed. I can only encourage you to add these kind of
> conveniences in the E submodule (or maybe somewhere else) as usage
> suggests that they are useful.

I believe that a good design for a Convenience submodule will follow the 
adaptation of existing syntax extension into -ppx style.  A first list 
of convenience functions can be found in:

http://caml.inria.fr/cgi-bin/viewvc.cgi/ocaml/branches/extension_points/experimental/frisch/print_gen.ml?revision=HEAD&view=markup

(a quick attempt to generate "variantizer" functions from type 
definitions, here taken from .cmi files)

> I'm still not sure that quasi-quotations are not a better approach,
> because the problem here is that the user has to learn a new interface
> to describe code fragments instead of using the syntax he's already
> familiar with.

I think it is very similar with quotations and antiquotations.  There 
are fewer names to know, but this can become more confusing.

For instance, in:

   <:expr<  $xxx:e$ >>

you need different antiquotations names to indicate at least that x is 
(i) an expression, or (ii) a string to be interpreted as a lower-case 
identifier, or (iii) a longident, or (iv) a string to be interpreted as 
a constructor name, or (v) a string to be interpreted as a string 
literal.  Each anti-quotation can be used only in specific contexts. 
Basically, one recreates a dedicated sub-language, with its own syntax 
and types, whereas OCaml is not *that* bad at manipulating ASTs.


> I still have the intuition that those arguments are more
> relevant to the expert extension writer, and that for a large set of use
> cases that concern *simple* extensions and beginner extension writers,
> quasiquotations are still noticeably easier to use.

I wouldn't trust a syntactic tool written by someone who is not quite 
familiar with the definition of the Parsetree.  That said, I've nothing 
against trying to make writing extensions as simple as possible. I'm 
personally more interested in providing tools and techniques to support 
writing robust "non trivial" extensions, but it's a nice observation 
that the current proposal with extension nodes supports quotation, and I 
encourage people interested in supporting "concrete syntax" ppx 
rewriters to push this idea further.


> The question of whether the extension uses classical or revised syntax
> is largely orthogonal to the design of the extension itself (except
> occasionally with quotation ambiguities concerns), and I could have used
> the classic syntax to write the Camlp4 extension just as well.

Including within quotations?  At some time, this was not supported; 
later it was supported but discouraged, because the standard syntax was 
not "regular" enough to support non-ambiguous quotations (I don't know 
the details).  But maybe that the status of quotations in regular syntax 
has changed.

> I found that
> there was no such re-learning curve with quasiquotations, they just work
> out of the box -- once you've been rebrained, once and forall, to see
> those <:stuff< >> as structured code rather than ASCII noise.

Can you tell without looking at the documentation or existing code how 
to write a function which maps a string s to an AST fragment 
representing a constructor of name s applied to a single argument, the 
string literal represented by s.  I.e. mapping "Foo" to the AST fragment 
corresponding to the expression

   Foo "Foo"

?

>     Implementing this "quote" expander is not very difficult, just a
>     little bit tedious (and this can be automated by parsing the
>     definition of the Parsetree).
>
>
> Isn't that essentially the same thing as the tool you implemented for
> Xavier above? (Can you reuse code between both?)

This tool relies on the toplevel value printer.  I'm not sure it will be 
easy to suport anti-quotations with this approach.

 > If I understand correctly, this is also the "Meta" operation of
 > Camlp4,
 > turning the AST for the expression <foo> into the AST for the OCaml
 > expression representing the AST for <foo>. When you say "automated by
 > parsing the definition of the Parsetree", do you have a realistic
 > design
 > in mind for such boilerplate code generators, or do you plan in
 > practice
 > to implement them by hand?

Something similar to 
branches/extension_points/experimental/frisch/print_gen.ml could be used 
to generate automatically an AST lifter.  The difficult part is to 
design anti-quotations, though, and since I'm not convinced by this 
approach, I'd rather put energy myself in other projects.

> Same old battle-horse: I dislike the idea that [%quote ] would change
> the meaning of syntactically valid OCaml code such as __patvar or
> !!patvar.

Point taken.  But since we are already under an "extension", I think 
this is less bad.

>     Quasi-quotations would be useful if the expanders had to generate
>     big fragments of mostly static code, with only a few "dynamic"
>     placeholders.  In my experience, this is rarely the case: you
>     assemble the resulting OCaml code by combining many small fragments
>     generated programmatically.  For these cases, a nice library of "AST
>     constructors" seems better to me.
>
>
> Maybe we need both, but if we eventually get nice constructor names for
> the AST definitions (I know that doesn't depend on you) I think if we
> only had time/energy/maintenance for two among (1) AST definitions (2)
> AST combinator library and (3) quasiquotation mechanism, I would suggest
> we keep (1) and (3) rather than (1) and (2).

My preference, as you understood, would be (1) and (2) rather than (1) 
and (3).

An interesting project would be to review existing syntax extension and 
see how much "almost static fragments" (where quasi-quotations shine) 
they have compared to code assembling tiny fragments (where combinators 
are betters).  I suspect that most interesting tools based on Camlp4 
don't have a lot of static fragments, but I might be wrong.


-- Alain