[wg-camlp4] Reconciling Merlin and ppx

[Alain Frisch]

On 11/04/2013 04:15 PM, Gabriel Scherer wrote:
> I think we should consider making statefulness explicit in the
> rewriters API, that is asking stateful rewriter to use a marshallable
> state type and express their transform in the state monad (state * AST
> -> state * result), instead of silently initializing internal state.

We probably need to turn "state" into "state option" for the first 
invocation of the rewriter.  This is a nice idea, but it would require 
to change the current format of AST files (shared with Camlp4), and 
introduce some unsafe behavior (the state will be untyped in the 
external representation).

Moreover, it would not cover all possible cases.  The problem is so much 
that a rewriter can have an internal state, but rather than it can be 
completely non compositional.  For instance, sedlex collects all lexer 
specifications and generate shared transition tables/functions at the 
beginning of the unit.  Admittedly, this is just an optimization (and it 
works fine to process phrase by phrase), but other examples of ppx 
rewriters could really do non-local processing which cannot be simply 
captured by a state threaded sequentially.


Alain