[wg-camlp4] Matching on concrete syntax (was: Re: Camlp4 uses)

[Hongbo Zhang]

Writing metaprogramming either in quasiquotation or combinator is fine for
me, the problem is the
latter approach does not work well with IDE and it's hard to read (we read
code more than write)
About implementation, my experience is that quotation is very tedious but
not difficult, antiquotation is not easy to get correct. about map_handler,
it can be improved in Fan
let rec map_handler =
  let patvar = "__exn" in with match_case (* set the default language*)
   fun
   [ {| $m1 $ $m2 |} ->
     {| $(map_hander m1) | $(map_handler m2)  |}
   | {| $p when $w -> $e |} ->
     {| ($p as $lid:patvar) when $w -> $(add_debug_expr _loc patvar e)  |}
   | m -> m ];;

On Fri, Mar 29, 2013 at 11:12 AM, Gabriel Scherer <gabriel.scherer at gmail.com
> wrote:

>
>
> On Fri, Mar 29, 2013 at 3:35 PM, Alain Frisch <alain.frisch at lexifi.com>wrote:
>
>> I'd write it as:
>>
>> let add_register e body =
>>   let_in [pany, app (evar "ExceptionHandling.register") [e]] body
>>
>> which does not look so bad.  This relies on the following definitions,
>> which could go e.g. in a Ast_helper.Convenience module:
>>
>> let evar s = E.ident (mknoloc (Longident.parse s))
>> let let_in l body = E.let_ Nonrecursive l b
>>    (* maybe with an optional argument for the recursive case *)
>> let pany = P.any ()
>> let app f args = apply f (List.map (fun a -> "", a) args)
>>
>
> That is better indeed. I can only encourage you to add these kind of
> conveniences in the E submodule (or maybe somewhere else) as usage suggests
> that they are useful. I originally planned to report on that through proper
> (?) channels, but that got lost in the noise of things to do, sorry.
>
> I'm still not sure that quasi-quotations are not a better approach,
> because the problem here is that the user has to learn a new interface to
> describe code fragments instead of using the syntax he's already familiar
> with. You have convincingly argued that the OCaml syntax cannot always be
> used to describe AST fragments (even in Camlp4, if you use the classic
> syntax in quotations some ambiguities force you to revert to plain AST
> constructors from time to time), and I hear the argument about fitting lots
> of small pieces together instead of inserting large chunks of code. I still
> have the intuition that those arguments are more relevant to the expert
> extension writer, and that for a large set of use cases that concern
> *simple* extensions and beginner extension writers, quasiquotations are
> still noticeably easier to use. That this feeling appears to be reflected
> by Xavier Clerc, which is one of the other early-triers of -ppx, gives me a
> hint that it may have some objective qualities.
>
>
>>
>> The rest of the code is interesting as well:
>>
>> Camlp4:
>>
>>   value rec map_handler =
>>     let patvar = "__exn" in
>>     fun
>>     [ <:match_case at _loc< $m1$ | $m2$ >> ->
>>         <:match_case< $map_handler m1$ | $map_handler m2$ >>
>>     | <:match_case at _loc< $p$ when $w$ -> $e$ >> ->
>>         <:match_case at _loc<
>>           ($p$ as $lid:patvar$) when $w$ -> $add_debug_expr _loc patvar
>> e$ >>
>>     | m -> m ];
>>
>>   value filter = object
>>     inherit Ast.map as super;
>>     method expr = fun
>>     [ <:expr at _loc< try $e$ with [ $h$ ] >> ->
>>       <:expr< try $e$ with [ $map_handler h$ ] >>
>>     | x -> super#expr x ];
>>   end;
>>
>>
>> PPX:
>>
>>   method expr e =
>>     let e = super#expr e in
>>     { e with pexp_desc =
>>         match e.pexp_desc with
>>           | Pexp_try (body, handler) ->
>>             let instrument_case (pat, body) =
>>               let patvar_str =  "__exn" in
>>               let patvar = Location.mknoloc (Longident.parse patvar_str)
>> in
>>               let pat = { pat with ppat_desc =
>>                 Ppat_alias (pat, Location.mknoloc patvar_str) } in
>>               (pat, add_register patvar body) in
>>             Pexp_try (body, List.map instrument_case handler)
>>           | other -> other
>>     }
>>
>>
>> This might be a matter of taste, but I prefer the PPX version, which I
>> can read only by knowing about the Parsetree (which is required anyway to
>> write any such code), a normal OCaml library.  The quotation and
>> anti-quotations in the Camlp4 version look very noisy, and it relies on a
>> syntax I'm not familiar with (revised syntax) and syntactic extensions
>> (quotations/antiquotations), with their own conventions ("<:match_case<",
>> $lid:$).  Since I'm not writing extensions every day, I really prefer
>> having to learn how to use simple OCaml data types and libraries
>> (Parsetree, Ast_helper) rather than to learn new syntax and new concepts.
>>  Also, I'd write the code above as:
>>
>> (* --> in Ast_helper.Convenience *)
>> let palias p x = P.alias p (mknoloc x)
>> let evar s = E.ident (mknoloc (Longident.parse s))
>> ...
>>
>>   method expr e =
>>     let e = super#expr e in
>>     match e.pexp_desc with
>>     | Pexp_try (body, handler) ->
>>         let instrument_case (pat, body) =
>>           (palias pat "__exn", add_register (evar "__exn") body)
>>         in
>>         {e with pexp_desc = Pexp_try (body, List.map instrument_case
>> handler)}
>>     | e -> e
>
>
>
> I used this example specifically to discuss the quasiquotation feature,
> not as a general comparison of -ppx and Camlp4's extension writing
> facilities. I do agree that, in this example, the AST traversal framework
> of -ppx is in fact better than Camlp4's. This is related to the fact that
> the AST structure is simpler and more closely reflect the way I logically
> think about pieces of OCaml code: match only takes a (pattern * expr) list
> instead of being a recursive expression of nested branches. That's a plus
> for ppx's design (and I like your approach of improving the upstream AST
> description to make it even better, at least while we don't have extension
> writers with code that breaks when we change it).
>
> That said, I think you're also a bit quick to jump to conclusions here.
> The question of whether the extension uses classical or revised syntax is
> largely orthogonal to the design of the extension itself (except
> occasionally with quotation ambiguities concerns), and I could have used
> the classic syntax to write the Camlp4 extension just as well. What
> happened in practice is that I looked at the set of old Camlp4 extensions I
> had lying around, copy-pasted the code of the one that looked most closely
> like what I was looking for (traversing the "match" structure), and spent
> the rest of the time wondering which output code to produce (the
> "add_register" function I quoted, plus the pattern-alias stuff).
>
> So the bottleneck in practice was in the code production part. It was just
> as true for the -ppx version (the traversal part was easy to write), but my
> experience producing code in the -ppx version was much less gratifying. Of
> course, it was my first use of your Ast_mapper.E interface, so there was
> some learning cost to take into account. But then I had type errors, I had
> to look at the documentation again, and it was a bit painful. Finally, I
> only handle Camlp4 extensions about once a year, so I have time to forget
> most details about how they work, in particular I *always* look at the
> documentation for the concrete AST constructor names in the (rare) cases
> where I need them. I found that there was no such re-learning curve with
> quasiquotations, they just work out of the box -- once you've been
> rebrained, once and forall, to see those <:stuff< >> as structured code
> rather than ASCII noise.
>
> I think the small tool you just introduced (translating OCaml code into
> AST-building ocaml code) may have made this "code production" part easier,
> but I'm not fond of the idea of pasting auto-generated code in my own code.
> In any case, it would be better if it produced code using the nice
> high-level interface, rather than the hard-to-read AST definitions, but
> that may be much more painful to implement so I'm not really asking for
> that.
>
>
>
>>  But maybe we
>>> can have quasiquotations with the current extension mechanism?
>>>
>>> let add_register patvar body =
>>>    [%quote
>>>      let _ = ExceptionHandling.register [%anti patvar] in [%anti body]
>>>    ]
>>>
>>
>> Implementing this "quote" expander is not very difficult, just a little
>> bit tedious (and this can be automated by parsing the definition of the
>> Parsetree).
>
>
> Isn't that essentially the same thing as the tool you implemented for
> Xavier above? (Can you reuse code between both?)
> If I understand correctly, this is also the "Meta" operation of Camlp4,
> turning the AST for the expression <foo> into the AST for the OCaml
> expression representing the AST for <foo>. When you say "automated by
> parsing the definition of the Parsetree", do you have a realistic design in
> mind for such boilerplate code generators, or do you plan in practice to
> implement them by hand?
>
> Note that nothing forces to use [%anti x] for antiquotations.  We could
>> very well decide on a more lightweight convention, like prefixing
>> identifiers, or using a dedicated operator:
>
>
>>     [%quote
>>       let _ = ExceptionHandling.register __patvar in __body
>>     ]
>>
>>     [%quote
>>       let _ = ExceptionHandling.register !!patvar in !!body
>>     ]
>>
>
> Same old battle-horse: I dislike the idea that [%quote ] would change the
> meaning of syntactically valid OCaml code such as __patvar or !!patvar. I
> think I would with [%anti ] for now, or maybe just [%a ] if need be -- in
> any case this is not part of the eventually-crowded extension namespace as
> they really are only markers to be rewritten by the implementation of
> %quote.
>
>
>> But this approach would work nicely only for writing expressions or
>> patterns on OCaml expressions, not on other syntactic categories (because
>> the content of an extension node is an expression).  The problem is that
>> AST-manipulating code tend to require to work a lot with many different
>> categories (like "match cases") even to build expressions.
>>
>
> That's a good point.
>
>
>> Quasi-quotations would be useful if the expanders had to generate big
>> fragments of mostly static code, with only a few "dynamic" placeholders.
>>  In my experience, this is rarely the case: you assemble the resulting
>> OCaml code by combining many small fragments generated programmatically.
>>  For these cases, a nice library of "AST constructors" seems better to me.
>>
>
> Maybe we need both, but if we eventually get nice constructor names for
> the AST definitions (I know that doesn't depend on you) I think if we only
> had time/energy/maintenance for two among (1) AST definitions (2) AST
> combinator library and (3) quasiquotation mechanism, I would suggest we
> keep (1) and (3) rather than (1) and (2).
>
>
>>
>>
>>
>> -- Alain
>>
>
>
> _______________________________________________
> wg-camlp4 mailing list
> wg-camlp4 at lists.ocaml.org
> http://lists.ocaml.org/listinfo/wg-camlp4
>
>


-- 
-- Regards, Hongbo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ocaml.org/pipermail/wg-camlp4/attachments/20130329/35302e52/attachment.html>