[wg-camlp4] Structured comments, shallow embeddings and deep quasiquotations

[Gabriel Scherer]

> In which category would you put code generators based on type declaration (a
> la type-conv) also fall in this category?

I think it depends on the "phase" in which the code generation results
are needed.

You could shoehorn type-conv into structured comments by saying that
the type definitions present in foo.ml, annotated with structured
comments, result in the production of a derived foo_with.ml file
(containing the generated stuff) -- that could later be used in your
project along with foo.ml

(Note: this seems somehow related to a change Hongbo made to the
Camlp4 map/fold/filter generators in Fan, that I'm not sure I
completely understand or like, and was discussed in
http://old.nabble.com/Some-utilities-about-camlp4-td33360537.html )

However, this mode of use puts constraints on the use of the
extension: you cannot in a same file have a type-conv-using
declaration, and refer to the (morally, not yet produced) result of
the boilerplate generation. I believe most people would rather see
type-conv in the second category of "shallow embeddings", under a
syntax in the spirit of

  (@deriving Eq Hash Print)
  type foo =
    ...
  ;;

I would have liked to choose a postfix syntax, but there is in general
no end delimiter to attach to unambiguously: in (type foo = Bar of baz
(+ blah)), the annotation may attach either to the constructor
argument type expression, or to the whole phrase. The real syntax
extension in fact adds an additional optional terminal, "with". The
only "end-closed" type declarations are record declarations, and this
is only because record types are not first-class type expressions.

Thanks you for your detailed syntax examples in the other topic. I
have taken the liberty to put them on the wiki:
  https://github.com/gasche/ocaml-syntax-extension-discussion/wiki/Use-Cases-using-light-annotations

Regarding concrete syntax, I would be tempted to suggest that (prefix
or postfix) annotations be attached to non-terminals (including
parenthesis). I believe there is an important difference between
"modulating this AST node" and "modulating the whole enclosed
expression" that would relevant for reasoning on the locality and
composability of syntactic extensions. I'm not sure yet what's the
best way to go (I'll try to see what Leo and you suggest).

Finally, I think -ppx + arbitrary annotations, without any further
restriction, is too free-form for robust syntax extensions: one
important problem with Camlp4 is that it allowed syntax extension
writers to modify the syntax in very bad way that hurt robustness. As
already discussed, I have a strong distaste for extensions that only
piggyback on existing syntax (without adding any explicit marker); I
would feel safer if the extension *mechanism* disallowed such
unstructured extensions, or at least made them less rewarding to write
than the composable ones. (For example by only passing to the
extension writer the part(s) of the AST that have been annotated).
Unfortunately, I don't see how Bisect would fit any such restriction.
Maybe that's a problem best solved by socialization (writing a
documentation on good practices, and yelling on people), but I sort of
doubt it -- I don't know how many time I've had to argue for *not*
globally changing the associativity of infix operators through Camlp4
in Batteries.

On Fri, Feb 1, 2013 at 8:50 AM, Alain Frisch <alain at frisch.fr> wrote:
> On 1/31/2013 10:13 PM, Gabriel Scherer wrote:
>>
>> I like Leo's idea of distinguishing three different forms of syntax
>> extensions (I'm not discussing concrete syntax):
>>
>> 1. structured comments
>> 2. shallow embeddings
>> 3. deep embeddings
>
>
> In which category would you put code generators based on type declaration (a
> la type-conv) also fall in this category?  For me, this is one of the most
> important use case of -ppx (the other being instrumentation of code, covered
> by 1 I guess).  Basically, they only require the ability to add annotations
> on type declaration and type expressions (and one needs to be able to
> include OCaml expressions in the annotations).  If/when runtime types are
> added to OCaml, they will hopefully also benefit from those annotations.
>
> I guess that 1 is excluded because those annotations are required to compile
> to program and 3 is excluded, because this has nothing to do with syntax.
> But 2 does not seem to apply either.
>
> -- Alain