[wg-camlp4] Reconciling Merlin and ppx

[Yaron Minsky]

I've been playing using Merlin more and more for personal projects,
and have started to wonder about how we're going to make Merlin work
effectively in a ppx world.  In some ways, ppx simplifies Merlin's
job, since there's now just one concrete syntax to parse.

But there are still problems to solve.  In particular, you will still
need to be able to apply AST-transformers to the partial ASTs that
Merlin generates in order to run it through the type-inference stage,
or to determine the environment.

Obviously, Merlin could just reimplement a bunch of common AST
transformers, as it does now for some common camlp4 extensions.  But
this seems like a situation that is too painful to maintain in the
long term.

A better idea, perhaps, would be to have some libraries or standards
about how to present an AST transformer so that it's suitable for
running against a partial AST.  I don't really know the details of how
Merlin works, but I understand it does a first pass to parse
expressions into top-level units, the parsing of which is done
independently.  AST transformers that operate on those units would
naturally gain some level of incrementality.  Another, perhaps more
intrustive approach would be to write PPX transformers against
Merlin's version of the AST.

I don't quite know what the right solution is, but when 4.02 comes out
we're all going to be writing a bunch of new AST transformers, and if
we really want Merlin to be a success, we should have a plan for how
those transformers will integrate with it.

y