[wg-camlp4] Reconciling Merlin and ppx

[Alain Frisch]

On 11/04/2013 05:58 PM, Frédéric Bour wrote:
> Correct me if I am wrong, but the current code already relies on
> marshalling and this is where the unsafety lies − the external state can
> just be seen as an abstract type ?

Currently, the marshaled data has always the same type (and we are 
supposed to change its magic number if the format changes), so all the 
I/O logic for PPX transformers is wrapped in a type-safe API.

Here, we are talking about allowing a specific PPX transformer to return 
an extra piece of arbitrary data (unless we want to specify the type of 
the state).  The data only makes sense for a specific (version of a) PPX 
transformer, and it cannot be protected by a magic number.

> Right. The point is that some extensions (like eliom) don't fit in the
> framework that merlin or the toplevel could expect for working
> incrementally.

Did you do some benchmarks?  I really don't know, but it could be the 
case that parsing + preprocessing the whole unit is fast enough.  For 
type-checking, I'm less sure, but a mechanism to reuse the result from 
the previous run on the common prefix should not be too difficult to 
support.  With such an approach, you could deal with a wider range of 
PPX rewriters without any custom support.


> I think that we should focus on providing a simple API for the most
> common cases (stateless transformers, then state monad, then
> backtracking stack, etc.). As long as the plugin informs its host of the
> way it is intended to be used, it is possible to provide graceful
> degradation.

The "state monad" API would work in the toplevel, but I believe you 
would need something more fine-grained for Merlin, in order to inform 
the rewriter about entering/leaving structures decomposed by Merlin 
itself.  Consider a rewriter which maintains a current "path" 
(collecting the M in "module M = struct ... end" for the current scope).


-- Alain