[wg-camlp4] Editor support for syntax extensions

[Gerd Stolpmann]

Am 30.01.2013 15:21:20 schrieb(en) Alain Frisch:
> On 01/30/2013 12:19 PM, Török Edwin wrote:
>> I still get syntax highlighting and matching parens in both vim and  
>> emacs for the above (although maybe antiquotations would confuse it):
> 
> Highlighting gets completely wrong as soon as you start e.g. using  
> double quotes. And indentation is just impossible if the editor isn't  
> aware of the grammar implemented by the extensions.  One could  
> certainly try to fix this and introduce more tools, complexity and  
> border cases. But I still fail to see the real benefits of allowing  
> very customized syntax which would justify such efforts.  The whole  
> point of -ppx versus camlp4 is precisely to stop messing up with the  
> concrete syntax: define some generic enough extension points, and  
> just work with a fixed concrete syntax.  I'd prefer to see dead  
> simple solutions adopted for -ppx, and more advanced stuff (based on  
> concrete syntax) left to camlp4/fan.
> 
>> 4. What syntax extensions to use => define in the source code
>> 
>> This has already been mentioned that the source file could define  
>> what syntax extensions it uses, so it works for all build systems  
>> (ocamlbuild, makefiles, omake), etc.
>> It may not even need special support from the compiler: there could  
>> be a -ppx that finds these pragmas and loads those other -ppx on its  
>> own!
> 
> I've started something along these lines:
> 
> https://github.com/alainfrisch/ppx_drivers
> 
> Exactly how this works with ocamlfind needs to be discussed.  I know  
> Gerd has some ideas!

Well, I had some private discussions with Alain before this list  
started.

A central point is that the ppx driver and ocamlfind remain separate  
software projects. You need the ability to use ppx preprocessors  
without ocamlfind, and the role of ocamlfind is "only" to make it more  
user-friendly. This should be similar to normal library lookup - where  
ocamlfind finally generates only a command where many details are  
filled in.

Alain had the idea to include the names of the preprocessors directly  
in the source text. He chose the syntax

include PPX(First_ppx_processor)(Second_ppx_processor)

Well, we probably can now select something better, not reusing the  
"include" notation for something that is not "include". Maybe let's  
prefer now

(:PPX "first_ppx_preprocessor" "second_ppx_preprocessor")

just to pick the current discussion up. The task of the ppx driver is  
to interpret these directives, and to run the preprocessors in the  
right order as subcommands (or, if the platform supports it, as  
plugins).

Ocamlfind integration: The problem here is that such a directive needs  
to be forwarded to ocamlfind in order to figure the details out. This  
would need a relatively tight coupling between the ppx driver and  
ocamlfind. In order to avoid that the ppx driver calls ocamlfind for  
every activated preprocessor, the idea is to make the ppx driver a  
library, maybe with a single function

preprocess : (string -> string) -> in_channel -> out_channel

The string->string function is the lookup function. It takes a string  
like "first_ppx_preprocessor" and returns the command to run.

So, that enables us to provide two utilities:

  - ocamlppx: This would be part of the OCaml distribution, and would be
    limited in functionality. In particular, it interprets the ppx  
strings
    directly as commands (identity as lookup function):

    (:PPX "/path/to/my/ppx/executable maybe arguments")

  - ocamlfind ppx: This version provides extended lookup features.
    You would just use

    (:PPX "package-name")

    and it is ocamlfind's task to look the executable up (or,  
optionally,
    load a plugin, but the details of the mechanism are hidden from the
    simple user).

Gerd


> 
> Alain
> _______________________________________________
> wg-camlp4 mailing list
> wg-camlp4 at lists.ocaml.org
> http://lists.ocaml.org/listinfo/wg-camlp4
> 


-- 
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany    gerd at gerd-stolpmann.de
Creator of GODI and camlcity.org.
Contact details:        http://www.camlcity.org/contact.html
Company homepage:       http://www.gerd-stolpmann.de
------------------------------------------------------------