<br><br>On Thu, Feb 7, 2013 at 5:09 AM, Alain Frisch <<a href="mailto:alain.frisch@lexifi.com">alain.frisch@lexifi.com</a>> wrote:<br>><br>><br>> "-ppx" on its own is even faster to build, since there is nothing to build (it is just a command-line flag, in the same way as "-pp" on which Fan relies, I guess).<br>
<br>Not always, as I pointed out, as long as you combine several global Ast Rewriter, I am optimistic that Fan has a great chance to outperform ppx.(currently with respect to the compiling performance, Fan is on par with ocamlc.opt directly, slightly slower)<br>
><br>><br>><br>>> 2. Fan does not require any compiler change, easy to distribute, on<br>>> the contrary the pervasive change to compiler is close to kill P4, Fan<br>>> or any other advanced external tools<br>
><br>><br>> ppx is already part of the development version and required minimal changes. We are also discussing the addition of a few syntactic constructs which will only impact the definition of Parsetree and the official parser. This cannot really be considered as a "pervasive change" to the compiler.<br>
><br>> Can you elaborate on why you think this would kill Camlp4 or Fan? I know from experience that Camlp4 is quite tedious to update when the concrete syntax of OCaml changes, but I'm sure someone will manage to update Camlp4/Fan definition of OCaml's AST and the associate parsers.<br>
><br>> -ppx is compatible with pre-processors implemented with -pp (Camlp4/Fan), as long as those pre-processors can understand the new syntactic constructs and pass them back to OCaml.<br>><br>Well, things are always possible, but just too difficult in some cases, remember that (Camlp4/Camlp5/Fan) preprocess the intermediate Ast, if we clutter Parsetree too much, like adding some attributes node per syntax category and ignored by the compiler later, which may require all other tools to mix Intermediate Ast with Parsetree, it's always possible, just too hard. I am perfectly fine with ppx so far, and I am happy to see bisect switch to ppx, because global Ast Rewriter happen to be very fit there, but if we try to make everything possible in ppx, there's a large chance that it will result in a ugly design in the compiler.<br>
><br>><br>> > unlike P4, it will<br>> > not inhibit OCaml's compiler's progress.<br>><br>> Your point above illustrates that it is not that simple: a simple addition to the OCaml syntax, which would normally require to adapt only<br>
> parser.mly, parsetree.mli, <a href="http://ast_mapper.ml/mli">ast_mapper.ml/mli</a> and a few other modules in the compilers, also requires to port the same changes to Fan's definition of the OCaml AST and to its parsers.<br>
> ]<br><br>I think the amount of work is similar, the only difference is that we maintain the change <a href="http://ast2pt.ml">ast2pt.ml</a>(dump intermediate ast into parsetree), it's just a piece of cake, and we don't need to maintain ast_mapper, since it is generated automatically in Fan. For the fan ast, we have tools<br>
{|derive(Map2<br> Fold2 OIter MetaExpr MetaPatt Map Fold Print OPrint OEq<br> GenLoc Strip ) |}<br>for FanAst, suppose in one day you have those tools for parsetree, the maintenance will<br>be a nightmare. <br>
<br>>> 4. It's easy to port P4's code base to Fan, it only takes me 2 hours<br>>> to port Alain's ulex to Fan<br>><br>><br>> It did not take that much longer to port ulex from Camlp4 to -ppx, and as an extra added bonus, it really gave me the feeling that I could finally *breathe* and understand exactly what I'm writing. Frankly, I'm more comfortable writing:<br>
><br>> E.let_ Recursive states<br>> (E.sequence<br>> (appfun "Sedlexing.start" [eid lexbuf])<br>> (E.match_ (appfun (state_fun 0) [eid lexbuf])<br>> (cases @ [P.any (), error])<br>
> )<br>> )<br>><br>> than:<br>><br>> <:expr< fun lexbuf -><br>> let rec $list:Array.to_list states$ in<br>> do { Ulexing.start lexbuf;<br>> match __ulex_state_0 lexbuf with<br>
> [ $list:Array.to_list cases$ | _ -> raise Ulexing.Error ] } >><br>><br>> which requires me to learn two new "sub-languages" (the revised syntax and the notion of quotation / antiquotation). Imagine that the "rec" flag above should be set only according to some condition to be checked; I know directly how to write that in regular OCaml but I would need to dig into Camlp4 documentation (or not) to see how to introduce an "anti-quotation on the rec flag".<br>
<br>Here, this happens to hit the sweet spot of quosi-quotation, first, I agree that the revised syntax should be adapted to original syntax, and that's part of my on-going work.<br>Let me explain a bit why quosi-quotation is superb here:<br>
1. It quickly goes into my muscle memory, because I write ocaml everyday (not parsetree), it's just my <b>intuition</b> that the Ast should be constructed that way, and I don't need to remember those alien function names, to be honest, I don't know what's the module E here. If I want to make a piece of code into data, I just need to quote them, add "{:expr||}" surrounding them, that's all. How would that happen in your "E" module?<br>
2. The E module seems to be a bit magic here, but Fan's quosiquotation works very nice with Emacs or any decent IDE, just like slime for lisp, I type one key in my Emacs, it just expand the code explicitly(I attached the result), but it's never going to happen with your "E" module, since it requires <b>explicit multiple staging</b>.<br>
3. For the parsetree, if you want to do syntactic grep over bigarray, you will get into much trouble:<br> ({pexp_desc=Pexp_ident<br> {txt= Ldot (Ldot (Lident "Bigarray", array), ("get"|"set" as gs)) ;_};_}<br>
This does not solve the problem at all, since it can not discriminate "a.{b,c}" from "Bigarray.get a b c " in Fan, you only need to do say<br> "`Bigarray (a,b,c)"<br> There are so many syntax desugaring in the parsetree that I don't know where to stop.<br>
<br>>> 5. Global Ast Rewriter is available but discouraged<br>>> 6. Local Ast Rewriter is provided(deriving and type_conv conflicts<br>>> will never happen in Fan)<br>>> {:ocaml| type u = A of int |}<br>
>> {:derive| (sexp,json) |}<br>><br>><br>> Can you describe in more detail your position concerning the following points?<br>><br>If it can be implemented by ppx, it can be straightforwardly translated into Fan, since ppx is simply a tiny proper subset of Fan(the part of global Ast Rewriter). There are so many great ideas proposed here that I would shamelessly steal into Fan even without bothering any change to the compiler. Fan provides first class parser/lexer(or in-line parser lexer), which is good, and there are people who appreciate it(<a href="https://sympa.inria.fr/sympa/arc/caml-list/2013-02/msg00020.html">https://sympa.inria.fr/sympa/arc/caml-list/2013-02/msg00020.html</a>), john harrop even claims that "<span style="color:rgb(51,51,51);font-family:'Trebuchet MS',Myriad,'Gill Sans','Century Gothic','Bitstream Vera Sans',verdana,lucida,arial,helvetica,sans-serif;font-size:11.199999809265137px;background-color:rgb(255,255,255)">Parsers written </span><span style="color:rgb(51,51,51);font-family:'Trebuchet MS',Myriad,'Gill Sans','Century Gothic','Bitstream Vera Sans',verdana,lucida,arial,helvetica,sans-serif;font-size:11.199999809265137px;background-color:rgb(255,255,255)">using Camlp4 are nicer than with any other tool I have ever used"</span>, but users are not locked in specific parsing technology, it's perfectly fine if you want to use either menhir or ocamlyacc.<br>
<br><br>--<br>-- Regards, Hongbo