[wg-camlp4] My no-use of camlp4 syntax extensions for Otags

[Hendrik Tews]

Alain Frisch writes:
   Date: Mon, 25 Feb 2013 10:30:20 +0100
   Subject: Re: [wg-camlp4] My no-use of camlp4 syntax extensions for Otags
   
   On 02/22/2013 01:32 PM, Hendrik Tews wrote:
   > For generating TAGS one needs exact location information on the
   > identifiers in the abstract syntax tree. 
   
   Defining the notion of the "right location" for syntax tree nodes which 
   are added is not easy.  For instance, which location should we use for 
   nodes obtained after macro expansion?  The location of the macro 
   definition, or the location of the "call site"?  Similarly, which 
   location should we use for code generated from type definitions?  The 
   location of the type definition?

It is not only about the synthesized or added ast nodes. Camlp4
suffers from the problem that sometimes location info for ast
nodes that follow a camlp4 macro are wrong (eg. bug #5127). I
hope that with -ppx we get for free, that location info of nodes
that are not touched stays correct.

Regarding synthesized nodes, my suggestion would be to have a
clear definition of an undefined location, which should be the
default for synthesized ast nodes. There should also be a clear
definition of synthesized location info, for the case where the
author of the preprocessor decides to provide some better
location info. 

For instance, from otags point of view, the identifiers for
printers and xml parsers that are generated for a type definition
should have the location of that type definition. But only the
identifiers, the generated function bodies can have the undefined
location. For otags I need to distinguish the locations for these
generated functions from those that really appear in the code.
This is because TAGS files contain small fragments from the
sources to make the TAGS file more robust when line numbers
change. For TAGS entries of synthesized functions and values it
is probably better to not copy any part of the source into the
TAGS file.


   > Another point that was always problematic with camlp4 was
   > documentation.
   
   Point taken.  

Thanks a lot! If you choose a form where user contribution is
easy (eg a wiki) I'll probably contribute, similarly to the
camlp4 wiki.

   > For this it would be nice, if the compilerlibs could
   > provide a function parse_with_ppx
   
   This is already the case, you can do something like:
   
Great!

   Maybe we should encapsulate the code above as a single function (with 
   proper support for errors, i.e. removing the temporary file produced by 
   the -pp preprocessor in case of error).
   
I would prefer something that automatically deals with temporary
files but raises errors as exceptions together with a function
that converts these exceptions into proper error messages. 

There are people who run otags automatically on whole directory
trees to index all files in there. For them, files with parse
errors should be simply skipped without even producing an error
message.


Bye,

Hendrik