[wg-camlp4] My no-use of camlp4 syntax extensions for Otags

[Hendrik Tews]

Hi,

I just noticed the wg-camlp4 initiative, here is my no-use of
camlp4 syntax extensions.

I use camlp4 for otags, the OCaml tag generator for emacs and vi.
I don't use any syntax extensions, I only use the parsing engine
and the ability to access the abstract syntax tree. After
generating the TAGS, the syntax tree is thrown away.

For generating TAGS one needs exact location information on the
identifiers in the abstract syntax tree. This has always been a
problem with camlp4, see the various open bug reports on this
topic. It would be good, if, together with -ppx, suitable
functions would be provided for getting the location info right
for those syntax tree nodes that get added/modified in -ppx
preprocessors. 

Another point that was always problematic with camlp4 was
documentation. I believe that the attitude present in
http://caml.inria.fr/pub/ml-archives/caml-list/2007/04/65b9d8de785307fe310d1179cea0729c.fr.html
and in
http://mirror.ocamlcore.org/caml.inria.fr/pub/ml-archives/caml-list/2007/05/9c2dd8a116e04931c0ba88337c7ced69.en.html
was largely responsible for the failure of the new camlp4. 
I would therefore strongly suggest to provide complete
documentation for the ast that preprocessors must process with
the _beta_ release of the next version.


One could think, that otags could work perfectly as a -ppx
preprocessor. There are, however, some issues:

- The otags ast preprocessor would produce no output, because no
  compilation should be happen when generating TAGS.

- For simplicity I would need to pass some state from one
  preprocessor invocation to the next. 

  For Emacs TAGS this would just be the open file descriptor of
  the TAGS file.

  vi TAGS should be sorted, so for vi I would need to pass the
  set of accumulated TAGS. Only the last preprocessor would
  write them to the file.

- For simplicity otags permits tagging whole projects or
  directory trees at once, where different files need possibly be
  preprocessed with different syntax extensions. Therefore I
  would need a version of the -ppx flag that applies to one file
  or a flag that clears the list of accumulated ppx'es.

For these reasons, it would probably be simpler, if otags has its
own driver that invokes the parser and the preprocessors as
needed. For this it would be nice, if the compilerlibs could
provide a function parse_with_ppx, or everything that is needed
for writing it. parse_with_ppx takes a filename and a list of
preprocessor names, parses the file, runs all preprocessors and
returns the ast.

Bye,

Hendrik