[wg-camlp4] On domain-specific foreign syntaxes

[Alain Frisch]

On 01/31/2013 02:57 PM, Hongbo Zhang wrote:
>
>        let x = foo << " >> bar
>
> This is unfair, how do you overcome this problem in ppx. The same thing,
> except that you don't use it

The discussion here is not about ppx vs camlp4, but whether it is a good 
idea to use quotations with "ocaml code in them".  My argument is that 
it is a very bad idea, because the only decent way to support quotation 
in editor modes is to ignore them (which is not currently the case).

I hope, for instance, that the ocaml emacs mode will be adapted to 
whatever syntax for quotation we choose to support natively in OCaml. 
Once this is done, emacs will no longer be confused with examples as the 
one above (good!), but it will no longer provide any assistance if OCaml 
code had to be written inside the quotation (bad! but only if we 
implement sedlex as you suggest, with a big quotation around the whole 
lexer definition, including actions).  If quotations are purely used to 
inject code in foreign syntax (with very simple anti-quotations which 
don't really benefit from editor support), this correct behavior of 
editors is no longer a problem.

Let me clarify my position about quotations:

The whole point of quotations, for me, is to escape from OCaml lexical 
convention.  If we have to embed fragments of foreign language syntax 
which works fine with OCaml rules (e.g. fragments which don't need 
double quotes or backslashes), strings are just fine:

fun x -> (:sql "select * from table where id = $x")

So the most important criterion of the syntax for quotations is that 
they actually allow to write an arbitrary piece of foreign syntax, and 
the simplest way to achieve that is to avoid putting any constraint on 
the closing delimiter.  That's why I suggest that the opening delimiter 
defines what the closing delimiter is explicitly, without too much 
restriction.

  {!{ ... }!}
  {x{ ... }x}
  {{ ... }}     assuming the content does not use }}


I'm not against quotations per se, but I argue:

  - Against their use such as the one suggested by Honbgo for sedlex 
(including the whole lexer definition, including potentially large OCaml 
expressions, inside a big quotation).

  - Against the idea that the syntactic aspect of quotations has 
anything to do with the notion of extension node to be expanded to AST 
fragments by some code.  They are just independent notions:

     * if the extension node can benefit from the OCaml syntax, this is 
really good (no need to write any parsing code in the "expander", and 
good editor support);

    begin(:sedlex) match lexbuf with
    | ... -> e1
    end

(I've used a syntax which make it clearer what the scope of the 
extension node is, and which dissociates the extension marker e1 from 
its content e2:  begin(:e1) e2 end)

     * quotations can be useful independently from any "expander", 
simply because they provide a different convenient style of writing 
strings which does not require escaping of double quotes and backslashes:

     failwith {{I "like" double quotes.}}

     * both notions can be combined if needed (an extension node whose 
content really has to be written in foreign syntax with a lot of double 
quotes or backslashes)

    begin(:sedlex) begin match lexbuf with
    | {{ '"' }} -> QUOTE
    | {{ '\' }} -> BACKSLASH
    | {{ [a-z]+ }} -> IDENT
    end

    or:

    ((:html) {{ <div class="$x"> }})

    (a lighter syntax with parentheses)

-- Alain