Hi, Jeremy,<div> I like your ideas, ;-). Fan adopt the same approach as you proposed.</div><div>Instead of </div><div> (@deriving ["sexp"; "json"])<br> type t = F of int | G of s<br> and s = H of (t * t)<br>
<br>I used the notation</div><div>{:ocaml|</div><div>type t = F of int | G of s</div><div>and s = H of (t * t)</div><div>|}</div><div>{:derive| (sexp,json)|}</div><div>But the semantics are essentially the same, </div><div>
should we take a serious look at Fan? As a much more advanced tool, Fan, compared with ppx, while it does not require any change to the compiler, and porting other camlp4 based library is much easier, I see a lot of benefits here.</div>
<div><br></div><div>If we change the compiler a lot for little benefit in the internal compiler, that makes life too hard for more advanced tools, are we a bit short-sighted here? I really like some other features like run-time types in the compiler instead, my 2 cents</div>
<div><br></div><div><br><div class="gmail_quote">On Sat, Feb 2, 2013 at 11:17 AM, Jeremy Yallop <span dir="ltr"><<a href="mailto:yallop@gmail.com" target="_blank">yallop@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On 1 February 2013 15:38, Gabriel Scherer <<a href="mailto:gabriel.scherer@gmail.com">gabriel.scherer@gmail.com</a>> wrote:<br>
> Finally, I think -ppx + arbitrary annotations, without any further<br>
> restriction, is too free-form for robust syntax extensions: one<br>
> important problem with Camlp4 is that it allowed syntax extension<br>
> writers to modify the syntax in very bad way that hurt robustness. As<br>
> already discussed, I have a strong distaste for extensions that only<br>
> piggyback on existing syntax (without adding any explicit marker); I<br>
> would feel safer if the extension *mechanism* disallowed such<br>
> unstructured extensions, or at least made them less rewarding to write<br>
> than the composable ones. (For example by only passing to the<br>
> extension writer the part(s) of the AST that have been annotated).<br>
> Unfortunately, I don't see how Bisect would fit any such restriction.<br>
> Maybe that's a problem best solved by socialization (writing a<br>
> documentation on good practices, and yelling on people), but I sort of<br>
> doubt it -- I don't know how many time I've had to argue for *not*<br>
> globally changing the associativity of infix operators through Camlp4<br>
> in Batteries.<br>
<br>
</div>I agree with Gabriel. Actually, I think that a small tweak to the<br>
design of -ppx could address both this and a number of problems that<br>
others have raised during the discussion here.<br>
<br>
The -ppx approach applies one or more global transformations to the<br>
ASTs of OCaml source files; these transformations can be parameterized<br>
by attributes attached at particular points in the syntax tree. This<br>
is a significant improvement over the Camlp4 approach, largely because<br>
it exchanges the (unnecessary) ability to change the concrete syntax<br>
for a number of valuable guarantees, which make extensions easier to<br>
write and code that uses extensions easier to understand.<br>
<br>
We can go further in this direction, and give up more (unnecessary)<br>
power in return for further guarantees. As Gabriel says, since -ppx<br>
extensions can arbitrarily transform the AST, it's not possible to<br>
understand any part of a program that uses extensions without<br>
understanding every aspect of the behaviour of every extension. We<br>
could, of course, seek to solve this by convention and social<br>
pressure, but there seems to be an emerging consensus that this isn't<br>
really satisfactory. One of the nice things about functional<br>
programming is that you have strong guarantees (via parametricity,<br>
immutability, and so on) about the effects of calling a particular<br>
function. We should strive to make it possible to reason in the same<br>
manner about the effects of syntax extensions.<br>
<br>
There are other legitimate concerns with the current proposal. As<br>
Xavier Clerc and others point out, attributes are apparently<br>
undeclared (i.e. global) and untyped. Alain rightly notes that it<br>
seems to be difficult to introduce declarations and types for<br>
attributes without significant complexity. Still, as OCaml<br>
programmers we're used to the benefits of precisely-scoped names and<br>
strongly-typed data, and it seems a shame to give these benefits up if<br>
we can find a way to keep them.<br>
<br>
Hongbo Zhang raises a further concern: when syntax extensions are<br>
global transformations on the whole file, the order in which<br>
extensions are applied becomes significant. This is a fairly serious<br>
matter, I think: the semantics of code that uses syntax extensions is<br>
now dependent on external factors, since we need to look for the flags<br>
passed to OCaml in the build configuration in order to understand the<br>
source.<br>
<br>
I think we can address all these concerns with a small adjustment in<br>
perspective. Instead of globally-scoped, untyped attributes processed<br>
by file-level externally-specified transformations, we might add a<br>
single node to the OCaml grammar for statically-executed AST<br>
rewriters. Using the same syntax already proposed for attributes, we<br>
might write, for example:<br>
<br>
(@deriving ["sexp"; "json"])<br>
type t = F of int | G of s<br>
and s = H of (t * t)<br>
<br>
or<br>
<br>
(@perform)<br>
(x <-- m;<br>
y <-- n;<br>
return (x y))<br>
<br>
In order for this to be valid code, 'deriving' and 'perform' should<br>
resolve to functions of appropriate types:<br>
<br>
val deriving : string list -> Parsetree.structure_item -><br>
Parsetree.structure_item<br>
<br>
val perform : Parsetree.expression -> Parsetree.expression<br>
<br>
Either during parsing or in a post parsing phase, the ASTs following<br>
'@deriving ["sexp"; "json"]' and '@perform' are passed to those<br>
functions and the results are inserted in place into the AST.<br>
Gabriel's concern is addressed, because there's no way for @perform<br>
(say) to access other parts of the AST: its effects are purely local.<br>
Xavier's concern is addressed, since AST rewriters, unlike attributes<br>
are declared and typed (and hence scoped). Hongbo's concern is<br>
addressed, since composition is explicit:<br>
<br>
(@deriving ["sexp"])<br>
(@nonrec)<br>
type t = C of t<br>
<br>
(Here '@deriving ["sexp"]' is applied to the result of applying '@nonrec'.)<br>
<br>
It should be possible to write almost all extensions in this manner.<br>
A variant of the stream parser syntax fits easily:<br>
<br>
(@parser)<br>
([ `If; x = expr; `Then; y = expr; `Else; z = expr ] => "if";<br>
[ `Let; `Ident x; `Equal; x = expr; In; y = expr ] => "let")<br>
<br>
as does Anil's cstruct extension:<br>
<br>
(@cstruct ~endianness:little)<br>
type pcap_header = {<br>
uint32_t magic_number; (* magic number *)<br>
uint16_t version_major; (* major version number *)<br>
...<br>
}<br>
<br>
Other extensions such as ifdef, js_of_ocaml, and pgsql could be<br>
handled in the same sort of way.<br>
<br>
Jeremy.<br>
<br>
[I'm deliberately avoiding the interesting but orthogonal questions of<br>
custom lexical syntax, and benign annotations for tools here.]<br>
<div class="HOEnZb"><div class="h5">_______________________________________________<br>
wg-camlp4 mailing list<br>
<a href="mailto:wg-camlp4@lists.ocaml.org">wg-camlp4@lists.ocaml.org</a><br>
<a href="http://lists.ocaml.org/listinfo/wg-camlp4" target="_blank">http://lists.ocaml.org/listinfo/wg-camlp4</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>-- Regards, Hongbo
</div>