[wg-camlp4] Reconciling Merlin and ppx

[Frédéric Bour]

On 04/11/2013 15:06, Alain Frisch wrote:
> On 11/04/2013 02:46 PM, Frédéric Bour wrote:
>> Merlin is used in an interactive context (were we expect a response in
>> less than ~ 60ms, on average).
>>
>> If we retransform the whole AST during each invocation, we also need to
>> retype the whole AST after.
>
> I assume there is already some logic to detect from which point the 
> code has changed, and to resume type-checking from the start of the 
> phrase, using some kind of snapshot.  Couldn't this logic be applied 
> after the preprocessing?

The process is roughly the following:
− Each chunk has an ending position.
− The editor tells the coordinates of the first character that changed.
− Merlin drops everything ending after that position then notify the 
editor of the ending position of the last chunk on merlin side.
− The editor is expected to tell the rest of the buffer (up to the 
cursor) starting from this last chunk.

So the "detection logic" has more to do with the editor than with the AST.
> Still, calling the PPX transformers as external processes might 
> already be too slow (there is always the option to link them in).
Constantly invoking external processes should be fast enough in merlin's 
case, as long as the work to do is not proportional to the size of the file.

>
> > Then an "open Core.Std" alone is enough to
>> blow up the time budget. (And this existimation is done on a desktop
>> computer, some users have EEE).
>
> For this specific example, this patch might help:
> http://caml.inria.fr/mantis/view.php?id=5916
> (I'm still waiting for a reproducible case before accepting the patch...)

Merlin's actually use a patch to make substitution lazy, guaranteeing 
"open" time proportional to the number of identifier introduced, not to 
the number of path substituted.
So "open Core.Std" is instantaneous (except on cold start, where the 
vast majority of the time is spent unmarshalling data on C-side).

I can't tell if this is the same approach as in the patch you linked, 
but it works well for Merlin.

>
>> However, I wonder how this state threading would be handled in the 
>> REPL ?
>
> Currently, it's not.  See:
>
> http://caml.inria.fr/mantis/view.php?id=5904#c8797
>
>
> Even though some rewriters might need to maintain state (or even have 
> access to the whole unit), many of them (and in particular those 
> corresponding to the extensions currently supported by Merlin) won't 
> probably need it.

Ok. So if some work had to be done to ensure proper compatibility 
between merlin and ppx, it would be for supporting stateful rewriters.

>
>
> Alain