[ocaml-platform] on the need and design of OCaml namespaces

[Yaron Minsky]

On Thu, Feb 28, 2013 at 5:17 PM, Didier Remy <Didier.Remy at inria.fr> wrote:
>> I agree with your point about settilng deep semantic issues ahead of
>> syntactic one.  I would however submit that the question of whether
>> you should:
>>
>> - Have opens in the language proper, including local opens
>> - Have renames for namespaces in the language proper
>
> Yes, this is  the minimal, and probably agreed on.

Lovely!

> The main question whether namespaces are hierarchical or flat is still not
> answered, Flat namespaces are not much of an extension to OCaml and do not
> raise many questions about their semantics.  This is the benefit of Alain's
> proposal---but also its weakness to be limited in expressiveness.

I don't really understand the goal of full hierarchical models.  I
would have thought that flat namespaces that include auto-open modules
would be quite sufficient.

One difference of opinion that I do have is that my intuition about
shadowing and dependencies on orders of opens is the opposite of
yours.  In particular, shadowing of names seems quite ordinary in
OCaml, it's an expected part of the language, and I would prefer for
namespaces to share that property.  Indeed, places where shadowing of
names is not possible (modules, in particular) have been a point of
frustration for us.

My inclination is that if you have two namespaces that define the same
module, and you open them in sequence, the second should simply shadow
the first.  Thus, I would expect to be able to write:

   open namespace Core#Std   (* defines a UNIX module *)

   ... some code that uses Core's UNIX ....

   open namespace Async#Std  (* defines a different UNIX module *)

   ... some code that uses Async's UNIX ....

I would be quite distressed to find that the namespace system
prevented the second open on the ground that it shadowed something in
the first.  This is precisely the behavior I would want and expect!

> If namespaces are hierarchical, the simplest model is that of a graph where
> inner nodes are namespaces, leave nodes are module objects, edges are
> names (and directed).
>
> One question raised is whether edges should also carry auto-open flags, so
> that whenever a node is opened all other nodes reachable by auto-open edges
> are also opened.  This feature is strongly desired by Yaron, but it raises
> further issues...
>
> There is no reason for having just one such edge. Indeed, the idea is to
> break big modules into smaller ones so that only actually used modules need
> to be linked. The same should hold for auto-open modules.
>
> However, since opening modules is not commutative, auto-open edges must be
> ordered.  This means that the mental model to give to the user is not just
> a directed graph with label edges whose leaves are module objects, as
> described above, but whose edges (at least those flagged auto-opened) are
> also ordered.  I think this is a bit more complicated and not so nice that
> edges need to be both ordered and named...
>
> The problem comes from module leaves.  For inner nodes one could check the
> absence of conflicting names at the end of auto-open edges when building
> namespaces so that the order of opens would not matter.  But one cannot do
> so for leaves, because modules should remain opaque to namespaces (I think),
> hence conflicts cannot be detected when building the namespace,
>
> Yet another question raised by auto-open flags, which Gabriel and I
> discussed  today is how auto-open behaves on access paths.  Assume,
> for instance, a namespace containing:
>
>         ROOT . --Core--> C --Pervasives*--> M
>                           \
>                             --Std*--> S

I apologize, but I just don't understand the example.  Is Pervasives a
module or a namespace?  I would assume it would be a module, since
I thought only modules are auto-open.  And what is the * for after Std
and Pervasives?

> and that the module M contains a submodule List.  Should the user always
> write Core#Pervasives#List or could he simply write Core#List, since given
> that Pervasives is auto-open it could have just written
>
>            open Core
>            List
>
> to designate module M.List. Indeed, one could expect that both
>
>         let open path in Name
> and
>         path#Name
>
> return the same object.  This way, the ordinary user would not need to know
> about the Pervasives module.  Only the expert user who wishes to open just
> the Pervasives module and not its parent will have to know its existence and
> write
>
>         open Core#Pervasive
>
> Unfortunately, this choice of semantics implies that paths must be
> interpreted in a much more complex way: path#name need not only look at the
> node at path for an edge labeled Name but also at all nodes (recursively)
> reachable by auto-open edges!
>
> With these (small) complications in mind, do we still wish to have
> auto-open edges? It this the good model?

As I said earlier, I'm pretty confused about the example above, so I
can't quite comment!

> ----------------
>
> In their absence, one would have to perform the auto-opens manually, and
> the order of opens would therefore be given by the user.
>
> The reason for auto-opens is to allow the user to just write
>
>         open Core
>
> instead of
>
>         open Core
>         open Pervasive
>         open Std
>
> Perhaps, an alternative to auto-open would be to allow one to write open
> directives in .ns files as well. Then, a file myenv.ns could both build a
> namespace and pre-open some of the nodes, including some of its leaves.  The
> user could then just put something as concise as
>
>         use myenv.ns
>
> at the top of his *.ml files. And if he opens myenv.ns he will have a very
> clear idea of what names should be visible.
>
> Just an idea, which I don't like so much either.  But is shows that there
> are still important details to be thought of and choices to be made.

How would the declaration "use myenv.ns" differ from "open namespace
Myenv"?  Are you proposing two different primitives, or would one only
have the "use" declaration?

I think my mental model of how namespaces should work is quite similar
to what you described above.  In particular, I would think that
opening a namespace would be the equivalent of adding a collection of
module aliases (possibly shadowing existing module names), and opening
some collection of modules (possible shadowing yet more module names
and other values).  All this should happen in whatever order is
described in the namespace definition file.

And in all of this, I view the dependency on order to be a feature,
not a bug.  OCaml is through and through full of this kind of
dependency, and I can see no justification for breaking that property
specifically with namespaces.  We like the behavior for values and
types, after all.  Why should namespaces be different?

> ----------------
>
> Notice that the problem comes from the fact that leaves of namespaces are
> opaque modules (and I think modules should remain opaque to namespaces), and
> as a result conflicting interfaces of modules cannot be detected when
> building a namespace, combined with the fact that the open construct of the
> module language is not itself strict (i.e. it may override bindings) and
> thus does not commute.