[wg-camlp4] A new branch to experiment with extension points

[Gabriel Scherer]

I'm sorry if my remarks set you in this way, but I'm not sure the
recent changes go in the right direction.

1. I have bad feelings about the idea to split "type foo = bar and baz
= foobar" in two structure items. I'm not sure how that affect the
parsing rules (is "type foo = bar;; and baz = foobar" legal now?), and
more importantly it doesn't fit my mental model of structure items as
being self-contained unit of meaning that can, say, be sent separately
to a toplevel (and this is actually a very important use case). If
"foo" and "baz" are mutually recursive type in my example about, it
makes no sense to consider "type foo = bar" in isolation.

This remark is in direct conflict with my previous remark that "with"
could be replaced by the generic annotation on structure_items. This
previous remark was a mistake, as I had not thought of mutually
recursive definitions, and I now believe that you indeed need to
attach attributes to the grammar element named "typedef" in the manual
grammar description (
http://caml.inria.fr/pub/docs/manual-ocaml/manual016.html ).

(Orthogonal anecdote: Damien Doligez has been looking closely at the
BNF described in the manual recently, and committed some fixes to it.
Contrary to what intuition suggests, it is reasonably safe to assume
that it is correctly describes the grammar of the language as we
understand it.)

2. I have mixed feelings about the idea that detaching struct_item
attributes from the corresponding struct_item is the right way to
handle the OCamldoc setting. I'd rather keep a more semantic model
that makes it explicit what an annotation scopes over, and more
convenient to write consumers/extensions to handle attributes (no need
to peek at the next structure items to collect all annotations for the
current one). Maybe ocamldoc sections could simply be modeled as
annotations on "();;", if we don't discover that we really want
ignored-by-default annotations in the general case (I wouldn't be
surprised).


On Fri, Mar 1, 2013 at 7:51 PM, Alain Frisch <alain.frisch at lexifi.com> wrote:
> On 03/01/2013 06:52 PM, Leo White wrote:
>>
>> The standard way to define a delimiter like "(:", in keeping with how
>> the rest of OCaml is parsed, is to make it a single token from the lexer.
>
>
> Agreed!
>
>
>>
>> 1. Extensions:
>>
>>      (: ident expr_opt )
>>    Used as a structure item, signature item, expression or type
>> expression.
>>
>> 2. Attributes:
>>
>>      [: ident expr_opt ]
>>
>>    Attached postfix to structure items, signature items, expressions or
>>    type expressions.
>>
>> 3. With Attributes
>>
>>      with ident ( expr), ident, ident (expr)
>>    Attached postfix to structure items or signature items.
>
>
> This looks good to me.
>
> We also need attributes on individual types in a type declaration.
>
> Also, I think it make sense for structure/signature items, to consider that
> attributes are "between" items, to support things like ocamldoc section
> headers.
>
> I've started to work on a change to the Parsetree (unfinished, not committed
> yet):
>
> http://alain.frisch.fr/split_type_decls.txt
>
> where each declared type in a type declaration become a full
> structure/signature item with a marker telling if this is the first type in
> the declaration or not (similar to the representation of signatures in the
> Types module).  This puts some extra invariants in the Parsetree (a
> Pstr_type Prec_nextmust follow another Pstr_type ..., maybe with attributes
> in between), but then we don't need to distinguish syntactically attributes
> on a declared type and attributes on the whole declarations, which would be
> hard to explain to users.
>
> The goal is to be able to write (keeping my original syntax for attributes
> in structure items):
>
> ===============================================
> ..(:doc 2 "My section header")
>
> type t = {...} ..(:id expr) ..(:id expr)
> and  s = {...} ..(:id expr)
>
> ..(:doc 2 "Another section header")
> ===============================================
>
> Here we have the following sequence of structure items:
>
>  Pstr_attribute "doc"
>  Pstr_type (Prec_first, ...)
>  Pstr_attributes "id"
>  Pstr_attributes "id"
>  Pstr_type (Prec_next, ...)
>  Pstr_attributes "id"
>  Pstr_attributes "doc"
>
>
>
> Independently of the syntax, I'd be interested to get feedback on these two
> proposals:
>
>  -  splitting declarations of mutually recursive types into single
> declarations in the Parsetree, and
>
>  -  having attributes as standalone items within signatures and structures
> (so also between types in a single recursive declaration) instead of having
> them attached to other items.
>
>
> Alain
>
> _______________________________________________
> wg-camlp4 mailing list
> wg-camlp4 at lists.ocaml.org
> http://lists.ocaml.org/listinfo/wg-camlp4