[wg-camlp4] My no-use of camlp4 syntax extensions for Otags

[Alain Frisch]

On 02/22/2013 01:32 PM, Hendrik Tews wrote:
> For generating TAGS one needs exact location information on the
> identifiers in the abstract syntax tree. This has always been a
> problem with camlp4, see the various open bug reports on this
> topic. It would be good, if, together with -ppx, suitable
> functions would be provided for getting the location info right
> for those syntax tree nodes that get added/modified in -ppx
> preprocessors.

Defining the notion of the "right location" for syntax tree nodes which 
are added is not easy.  For instance, which location should we use for 
nodes obtained after macro expansion?  The location of the macro 
definition, or the location of the "call site"?  Similarly, which 
location should we use for code generated from type definitions?  The 
location of the type definition?


> Another point that was always problematic with camlp4 was
> documentation.

Point taken.  The nice thing is that we will mostly have to document the 
parse tree and the calling convention for ppx, which is rather simple. 
The amount of information to be aware of will be much smaller than with 
Camlp4.


> For these reasons, it would probably be simpler, if otags has its
> own driver that invokes the parser and the preprocessors as
> needed. For this it would be nice, if the compilerlibs could
> provide a function parse_with_ppx, or everything that is needed
> for writing it. parse_with_ppx takes a filename and a list of
> preprocessor names, parses the file, runs all preprocessors and
> returns the ast.

This is already the case, you can do something like:

   let inputfile = Pparse.preprocess sourcefile in
   let ast =
    Pparse.file ppf inputfile Parse.implementation ast_impl_magic_number
   in
   ...
   Pparse.remove_preprocessed inputfile


The -pp and -ppx preprocessors are taken from Clflags 
(Clflags.preprocessor, Clflags.ppx).

Maybe we should encapsulate the code above as a single function (with 
proper support for errors, i.e. removing the temporary file produced by 
the -pp preprocessor in case of error).


Alain