[ocaml-platform] An alternative proposal for namespaces

[Alain Frisch]

Dear all,

I'd like to propose a variant of Leo's proposal ( 
http://www.lpw25.net/2013/03/10/ocaml-namespaces.html ).


1. Getting rid of "simple namespace through filenames"

Leo proposes that a file foo-bar.ml automatically creates a module Bar 
in namespace Foo.  With this approach, collecting all modules available 
in the namespace Foo requires to look for all foo-*.cmi files in the 
search path.  This is known to create problems with dependency analysis 
and generated files.  Moreover, since it is also suggested that the 
source file can be named differently from the target .cmi files, it is 
even more difficult for ocamldep to produce a good result.   Another 
problem which I raised in a previous mail is that loading the whole 
directory might prevent different kind of build systems based which 
infer dependencies by capturing system calls.

I propose to keep search path files as the only way to define 
namespaces.  For the typical user (who relies on ocamlfind), this does 
not change anything.  The overhead for library developers is minimal. 
And this could drastically improve the quality of ocamldep.  It would 
work as follow:

  - A namespace-qualified module name is looked up in search path files 
passed to ocamldep.  For those references, ocamldep can produce a 
dependency to concrete files.  This is the best of both worlds: (i) 
contrary to "ocamldep -modules", the build system does not need to 
recreate the lookup logic (in Leo's proposal, this even requires calling 
ocamldep many times in this process, which might be quite slow);  (ii) 
contrary to "ocamldep" (without -modules), this work well even with 
files to be generated (and even when the source file is named differently).

  - An unqualified module name is resolved by taking local module 
declarations and "open namespace" statement into account.  If the name 
can potentially refer to a namespaced module defined in on the search 
path files passed to ocamldep, ocamldep returns a dependency to concrete 
files as in the case above.  Otherwise, it behaves as of today 
(returning either module names in "-modules" mode or concrete files 
otherwise, by looking at the file system).  Since only non-namespaced 
module names are potentially returned as module names, the build system 
can behave as today (simply looking buildable files in -I directories).


Also, "open namespace Foo" would fail if there is no definition of 
modules under namespace "Foo" in the search path files passed to the 
compiler.  This is more robust than looking for foo-*.cmi files on the 
file system: these files might not exist yet, either because (i) 
dependency analysis is bogus or (ii) there is no actual dependency to 
any module in Foo in the current unit; or, on the contrary, there could 
be foo-*.cmi left from a previous compilation even though we have 
decided e.g. to change the name of the namespace.  It's much better, in 
my opinion, to be able to work with a "closed world" assumption w.r.t. 
namespaces while compiling a single unit.


2. Simplifying search path files

I propose to get:

   (i) rid of aliases;

   (ii) define as an error the situation where the same 
namespace-qualified name is defined twice (with different target files) 
in the set of all search path files used together.

(ii) will strongly encourage libraries to avoid stepping on each other 
feet and thus make the life of the user better.  Conflicts resolved by, 
say, a first-match policy are likely to fail at some point anyway (with 
bad error messages).  Namespaces are introduced to avoid such problems.

Complex namespace manipulation, which could benefit from aliases, are 
likely to be useful only to advanced users.  I don't see it as a 
problem that they have to resolve aliases manually or with very simple 
tools.

A nice consequence is that the ordering of search path files does not 
matter for the resolution of namespaced names to filenames, so the 
findlib package descriptions don't have to deal with it.  Also search 
path files and -I directories become independent concepts (-I 
directories are only used to resolve non-namespaced names).


3. Using search path files for reverse lookup

The information contained in search path files allows the compiler to 
map back from "compilation unit names" to namespaced names.  This 
information can be used by the compiler the produce nicer error messages 
involving namespaced names.  (This might be implicit in Leo's proposal.) 
   Here, we can define a first-match policy for this reverse lookup (if 
the same compilation unit is given several namespaced names).

The same applies for ocamldoc.


4. Replacing "-name"

Leo proposes to allow specifying the target compilation unit name 
(*.cmi/*.cmo/*.cmx/*.o) with a new compiler command-line argument.

The use case is to simplify the life of library developers so that they 
don't need to use long names for all their source files.  This should 
greatly simplify the migration of existing libraries to namespaces.
In practice, however, most the source files in such a library will need 
to include a new "open namespace" directive to make other modules from 
the same library accessible.  Also, we will need to inform the build 
system about the relation between source and target filenames.  For 
ocamlbuild, for instance, this will probably mean the creation of a new 
kind of text file to describe this mapping.

Since libraries will need to ship search path files anyway, I think we 
can reuse them to simplify further the migration to namespaces.  I 
propose to support calling the compiler with a "-namespace" argument:

   ocamlc -c mylib.mlpath -namespace Mylib bar.mli

This will do three things:

  - Lookup in mylib.mlpath for the compilation unit name corresponding 
to Mylib..Bar and use it as the target filename (if mylib.mlpath 
contains a line "Mylib..Bar:mylib_xxx", then the compiler will compile 
bar.mli to mylib_xxx.cmi).  We hard-code in particular that in this 
mode, the source filename is equal to the module name as seen by 
"clients" of the library.

  - Automatically "open namespace Mylib" for compiling bar.mli

  - Keep a reference to the "-namespace" argument in the generated .cmi 
file.

Tools could then benefit from the fact that the source file (bar.ml) can 
be retrieved mechanically from either the compilation unit name 
(mylib_xxx) or the qualified name (Mylib..Bar).  And ocamlbuild and 
other build systems don't need to introduce another way to map form 
source file names to compilation unit names.



-- Alain