[wg-camlp4] Time for a summary?

[Hongbo Zhang]

On Thu, Feb 7, 2013 at 5:09 AM, Alain Frisch <alain.frisch at lexifi.com>
wrote:
>
>
> "-ppx" on its own is even faster to build, since there is nothing to
build (it is just a command-line flag, in the same way as "-pp" on which
Fan relies, I guess).

Not always, as I pointed out, as long as you combine several global Ast
Rewriter, I am  optimistic that Fan has a great chance to outperform
ppx.(currently with respect to the compiling performance, Fan is on par
with ocamlc.opt directly, slightly slower)
>
>
>
>>     2. Fan does not require any compiler change, easy to distribute, on
>> the contrary the pervasive change to compiler is close to kill P4, Fan
>> or any other advanced external tools
>
>
> ppx is already part of the development version and required minimal
changes.  We are also discussing the addition of a few syntactic constructs
which will only impact the definition of Parsetree and the official parser.
 This cannot really be considered as a "pervasive change" to the compiler.
>
> Can you elaborate on why you think this would kill Camlp4 or Fan?  I know
from experience that Camlp4 is quite tedious to update when the concrete
syntax of OCaml changes, but I'm sure someone will manage to update
Camlp4/Fan definition of OCaml's AST and the associate parsers.
>
> -ppx is compatible with pre-processors implemented with -pp (Camlp4/Fan),
as long as those pre-processors can understand the new syntactic constructs
and pass them back to OCaml.
>
Well, things are always possible, but just too difficult in some cases,
remember that (Camlp4/Camlp5/Fan) preprocess the intermediate Ast, if we
clutter Parsetree too much, like adding some attributes node per syntax
category and ignored by the compiler later, which may require all other
tools to mix Intermediate Ast with Parsetree, it's always possible, just
too hard. I am perfectly fine with ppx so far, and I am happy to see bisect
switch to ppx, because global Ast Rewriter happen to be very fit there, but
 if we try to make everything possible in ppx, there's a large chance that
it will result in a ugly design in the compiler.
>
>
> > unlike P4, it will
> > not inhibit OCaml's compiler's progress.
>
> Your point above illustrates that it is not that simple: a simple
addition to the OCaml syntax, which would normally require to adapt only
> parser.mly, parsetree.mli, ast_mapper.ml/mli and a few other modules in
the compilers, also requires to port the same changes to Fan's definition
of the OCaml AST and to its parsers.
> ]

I think the amount of work is similar, the only difference is that we
maintain the change ast2pt.ml(dump intermediate ast into parsetree), it's
just a piece of cake, and we don't need to maintain ast_mapper, since it is
generated automatically in Fan. For the fan ast, we have tools
   {|derive(Map2
      Fold2 OIter MetaExpr MetaPatt Map Fold Print OPrint OEq
      GenLoc  Strip ) |}
for FanAst, suppose in one day you have those tools for parsetree, the
maintenance will
be a nightmare.

>>     4. It's easy to port P4's code base to Fan, it only takes me 2 hours
>> to port Alain's ulex to Fan
>
>
> It did not take that much longer to port ulex from Camlp4 to -ppx, and as
an extra added bonus, it really gave me the feeling that I could finally
*breathe* and understand exactly what I'm writing. Frankly, I'm more
comfortable writing:
>
>   E.let_ Recursive states
>     (E.sequence
>        (appfun "Sedlexing.start" [eid lexbuf])
>        (E.match_ (appfun (state_fun 0) [eid lexbuf])
>           (cases @ [P.any (), error])
>        )
>     )
>
> than:
>
>   <:expr< fun lexbuf ->
>     let rec $list:Array.to_list states$ in
>     do { Ulexing.start lexbuf;
>          match __ulex_state_0 lexbuf with
>          [ $list:Array.to_list cases$ | _ -> raise Ulexing.Error ] } >>
>
> which requires me to learn two new "sub-languages" (the revised syntax
and the notion of quotation / antiquotation).  Imagine that the "rec" flag
above should be set only according to some condition to be checked; I know
directly how to write that in regular OCaml but I would need to dig into
Camlp4 documentation (or not) to see how to introduce an "anti-quotation on
the rec flag".

Here, this happens to hit the sweet spot of quosi-quotation, first, I agree
that the revised syntax should be adapted to original syntax, and that's
part of my on-going work.
Let me explain a bit why quosi-quotation is superb here:
1. It quickly goes into my muscle memory, because I write ocaml everyday
(not parsetree), it's just my *intuition* that the Ast should be
constructed that way, and I don't need to remember those alien function
names, to be honest, I don't know what's the module E here. If I want to
make a piece of code into data, I just need to quote them, add "{:expr||}"
surrounding them, that's all. How would that happen in your "E" module?
2. The E module seems to be a bit magic here, but Fan's quosiquotation
works very nice with Emacs or any decent IDE, just like slime for lisp, I
type one key in my Emacs, it just expand the code explicitly(I attached the
result), but it's never going to happen with your "E" module, since it
requires *explicit multiple staging*.
3. For the parsetree, if you want to do syntactic grep over bigarray, you
will get into much trouble:
   ({pexp_desc=Pexp_ident
                    {txt= Ldot (Ldot (Lident "Bigarray", array),
("get"|"set" as gs)) ;_};_}
   This does not solve the problem at all, since it can not discriminate
"a.{b,c}" from "Bigarray.get a b c " in Fan, you only need to do say
   "`Bigarray (a,b,c)"
   There are so many syntax desugaring in the parsetree that I don't know
where to stop.

>>     5. Global Ast Rewriter is available but discouraged
>>     6. Local Ast Rewriter is provided(deriving and type_conv conflicts
>> will never happen in Fan)
>>         {:ocaml| type u = A of int |}
>>         {:derive| (sexp,json) |}
>
>
> Can you describe in more detail your position concerning the following
points?
>
If it can be implemented by ppx, it can be straightforwardly translated
into Fan, since ppx is simply a tiny proper subset of Fan(the part of
global Ast Rewriter). There are so many great ideas proposed here that I
would shamelessly steal into Fan even without bothering any change to the
compiler. Fan provides first class parser/lexer(or in-line parser lexer),
which is good, and there are people who appreciate it(
https://sympa.inria.fr/sympa/arc/caml-list/2013-02/msg00020.html), john
harrop even claims that "Parsers written using Camlp4 are nicer than with
any other tool I have ever used", but users are not locked in specific
parsing technology, it's perfectly fine if you want to use either menhir or
ocamlyacc.


--
-- Regards, Hongbo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ocaml.org/pipermail/wg-camlp4/attachments/20130207/67c154b5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tmp.ml
Type: application/octet-stream
Size: 3179 bytes
Desc: not available
URL: <http://lists.ocaml.org/pipermail/wg-camlp4/attachments/20130207/67c154b5/attachment.obj>