[ocaml-platform] on the need and design of OCaml namespaces

[Alain Frisch]

On 2/22/2013 3:06 PM, Leo White wrote:
> There are a number of possible solutions to spurious opens. The simplest
> of which is to only look for "Core.Std.Mutex" in directories which
> contain the special "Core.Std" .cmi file (mentioned previously as a
> solution to typos in open statements). These special .cmi files could
> also be extended to include a list of modules that have that namespace
> within the current directory which would prevent spurious reads entirely.

This seems a little bit hackish to me, and likely to require more 
over-engineering (do we need a tool to create those .cmi files; if they 
are plain text file, it's ugly to use the .cmi extension).

>>  - It prevents from putting .cmi files from many libraries in the same
>> directory, which is sometimes useful (to simplify deployment; to
>> control precisely the set of .cmi available for a given file; to
>> improve performance by avoiding repeated lookups in many directories).
>
> I think that this ability is of dubious value and not really a big loss.

We use this quite intensively on LexiFi's code base.  This really speeds 
up compilation time under Windows, and we also use it to simplify 
deployment (our application is shipped with some .cmi files from many 
libraries and automatically compiles user-provided addins against them; 
it would be tedious -- and useless -- to reproduce a complete hierarchy 
of libraries on the installation side).

We would hate to have the third-party libraries we use adopt a new 
feature (namespaces) which solves a problem we don't have but forces us 
to change in non trivial ways how we organize our code base and deployment.

> This is basically an error, so dependencies
> should not be expected to be correct.

I don't agree.  Having wrong dependencies is a nightmare to debug and 
during development having errors in the code is not an exceptional 
situation.  A robust build system should handle nicely things like 
moving files around, renaming them, etc.

> There is possibly the need for some kind of partial dependency for
> parallel builds. This is more like a lock than a dependency, so there
> should be no question of circular dependencies. I'm not really familiar
> with how parallel file accesses work on different file systems, but
> perhaps the compiler could lock ".cmi" files before reading and writing.
> This might be a good idea more generally for cases where dependencies
> have not been correctly calculated.

These locks do not really solve the problem.  Imagine you have a big 
project with two modules

   foo/a.ml in namespace Foo
   bar/a.ml in namespace Bar

Now you compile x.ml which refers to Foo # A. "ocamldep -modules" 
reports that the dependencies for it include mode "A", which must be 
mapped to all buildable a.cmi/a.cmx in your tree, i.e. both foo/a and 
bar/a.  (Things are even worse if you use the same syntax as for 
modules, because then any reference like Foo.A must be interpreted as a 
potential dependency to foo.cmi/cmx or to a.cmi/cmx.)  But maybe 
bar/a.ml refers to x.ml, and then you have a circular dependency.

I'd like any proposal about namespaces to come with a description of (i) 
how ocamldep is supposed to behave; (ii) how build systems (based on, 
say, make, omake and ocamlbuild) are supposed to be adapted.

So here it is for my proposal of using "short names" declared in 
external files:

   - Doing "open namespace Core_std" is strictly equivalent to doing 
"module Mutex == Core_std_mutex;; module Thread == Core_std_thread;; 
module Date == Core_std_date" assuming a new module aliasing feature 
(available in structures and signatures).

   - ocamldep would read the core_std.ns file (meaning that it must 
exist and be up-to-date when ocamldep runs; I expect those files to be 
quite static so this shouldn't be a big problem -- otherwise, we would 
need to have a first pass where ocamldep would returns the list of .ns 
files to be opened, and the build system would arrange to build them).

  - when ocamldep encounters a module reference "Mutex" in a scope where 
an alias "module Mutex == Core_std_mutex" has been defined (manually or 
by loading core_std.ns), it reports a dependency to module 
Core_std_mutex instead of Mutex.

  - the build systems do not have to be adapted.



>
>> That's why I've proposed to allow specifying mapping between
>> references to external modules in dedicated files.  We could have a
>> file core_std.ns (probably shipped with Core) with this content:
>>
>> Mutex = Core_std_mutex
>> Thread = Core_std_thread
>> Date = Core_std_date
>>
>> and just a reference in the source code (or on the command-line):
>>
>>   open namespace Core_std
>>
>> which would load core_std.ns and use the corresponding module renaming
>> in the rest of the module.
>
> There is very little difference between that suggestion and having a
> core_std.ns file containing:
>
>   Mutex;
>   Thread;
>   Date;
>
> and using that as a (partial) declaration of a Core_std namespaces,
> except that you have to give every file a unique long filename. So I
> don't really see the particular benefit of using long filenames.

With my proposal, you don't force users of the library to use the new 
feature (meaning that if for some reason your local build system does 
not work nicely with namespaces, you can always refer to modules using 
their long names).  Moreover, the semantics is very easy to explain, the 
linker does not need to be changed, and we don't change how OCaml 
behaves w.r.t. to the file system (your proposal prevents users from 
using the currently valid technique of copying .cmi files from many 
libraries in the same directory).

> There is also a more general problem with any solution like this, which
> tries to define namespaces (or sets of aliases) in a single file. It is
> difficult to use the namespace from inside the modules that are within
> the namespace. For example, if I use:
>
>   open namespace Core_std
>
> from within mutex.ml then it will attempt to open itself.


I don't understand the problem.  The source file would be 
core_std_mutex.ml and it is fine if it does "open namespace Core_std" as 
long as it doesn't refer to Mutex (which would be a circular dependency).


Alain