[wg-camlp4] [Caml-list] Working Group: the future of syntax extensions in OCaml, after camlp4

[Alain Frisch]

On 1/28/2013 10:12 PM, Gerd Stolpmann wrote:
> But I'd also like to fight for stream parsers a bit. In general, they
> are a nice way to write top-down parsers. Like for all top-down parsers,
> the class of acceptable grammars is not very powerful (less than
> LALR(1)), but you can easily add things that are nearly impossible with
> bottom-up parsers, like stores or contextuality, and you can locally
> escape from the parsing discipline entirely. Sometimes it is a perfect
> match.

In my experience with top-down parsers, a manually written recursive 
descent with pattern matching on (normal) lists of tokens, plus a few 
"combinators", works also fine.

>  From an old OCaml manual:
>
>...
 >
>
> stream-component ::= ’ expr
>                     | expr
> stream-matching ::= stream-pattern [pattern] -> expr
>                      { | stream-pattern [pattern] -> expr}
> stream-pattern ::= [< >]
>                    | [< stream-pat-comp {; stream-pat-comp [?? expr]} >]
> stream-pat-comp ::= ’ pattern [when expr]
>                    | pattern = expr
>                    | ident

Let me throw a crazy idea: maybe we could merge the grammar for patterns 
and expressions, and let the type checker reject bad patterns and bad 
expressions from this more admissible class.  This would give plenty of 
space for extensions.

A quick inspection of Parsetree shows that the following syntactic forms 
are available for patterns but not expressions:

  - Alias pattern:  p as x
  - "open record" flag: { l1=p1; _ }
  - or pattern: p1 | p2
  - type pattern: #t
  - range patterns: c1..c2 (currently desugared by the Parser)

It seems possible to enrich the class of parsed expressions with those 
cases without introducing conflicts (to be checked...).

Assuming it is indeed technically possible, do you think it would be a 
good idea to merge the syntactic categories of patterns and expressions?


Alain


>
> Doing this with -ppx is certainly a challenge.
>
> Some ideas:
>
>   - [< >]: Maybe we should simply add several types of brackets to
>     the grammar. Syntax as for list, but reserved for extensions.
>
>   - 'expr: This is like a tag. Maybe use I(expr) instead, where "I"
>     means "item".
>
>   - pattern = expr: I guess this is what Alain sees as difficulty.
>     As quotation it would look strange. What we need here is something
>     in the Parsetree that allows us to include expressions into patterns
>     (which would look like a "computed pattern"). Maybe `(expr) inside
>     patterns. Or another notation...
>
> Gerd
>
>
>>
>>
>> Alain
>>
>>
>
>