[ocaml-platform] on the need and design of OCaml namespaces

[Yaron Minsky]

On Fri, Mar 1, 2013 at 8:18 AM, Didier Remy <Didier.Remy at inria.fr> wrote:
>>> 2) However, you cannot open several nodes at the same time without
>>>     specifying an ordering between those nodes.
>>
>>
>> What does it mean to open several nodes "at the same time"?  I would
>> have thought that opening of nodes would always be ordered, since the
>> open statements would need to be written in some order.
>
>
> If Core has two auto-open subnodes (e.g Std and Pervasives), then opening
> core will open the two auto-open subnodes at the same time. This is what I
> meant.

Got it.  In my thinking, these auto-opens could only be modules, to be
clear, not namespaces.  So i'm thinking about this all as very flat,
much as Alain is.

> In this case, you write with the auto-open semantics
>
>         open Core
>
> and what happens is equivalent to (without the auto-open semantics)
>
>         open Core
>         open Std
>         open Pervasives
> or
>         open Core
>         open Std
>         open Pervasives
>
> So the edges Std and Pervasives need both leaving from Core need to be
> ordered in the namespace.

Yup.

>>> The * is the auto-open flag, as used in Gabriel's description.
>>> Sorry for leaving this implicit.
>>
>>
>> Got it.  That's quite helpful, though I'm still a little lost about
>> the problem you're describing.  It sounds to me like the issue dervies
>> from the fact that you're proposing opening up a parent namespace as
>> an operation that opens all children as well.  Is that right?
>
>
> Yes, an operation that opens all children marked auto-open.
>
>> I hadn't even begun to imagine wanting that feature, so maybe that isn't a
>> real issue at all?  Or perhaps I'm just confused....
>
>
> I thought this is what you where asked for when saying that you want
>
>         open Core
>
> to behaves as
>
>         open Core
>         open Std
>         open Pervasives
>
> because you have marked Std and Pervasives as auto-open.

Maybe the difference is, I was only imagining this one level deep:
single-level namespaces, and namespaces contain list of module
aliases, plus a list of module opens, all of which are in a single
total order.

>>> I think namespaces should be unordered to scale up.
>>>
>>> You would like to be able to represent all available libraries in
>>> the OCaml world as a namespace, with libraries coming from many
>>> different sources and there is no natural order on these, i.e. no
>>> order that could be given a meaningful semantics.  Would you put
>>> World#JaneStreet before World#Lexify or the other way round?
>>>
>>> If name spaces are unordered, it is also easy to populate them by
>>> reading and merging different sources in a strict way: name spaces
>>> are meant to avoid conflicting names to start with, so if a merge
>>> introduces conflicts, this is really the right place to fix it by
>>> renaming one side or reorganizing the hierarchy rather than do
>>> some silent overriding.
>>
>>
>> I wonder if this all comes down to different thoughts about hierarchy.
>> I guess I wouldn't think the namespace World#JaneStreet would be
>> ordered compared to World#Lexify, but the module aliases and opens
>> contained in a namespace would, I thought, have been ordered.  And
>> surely, when you open multiple namespaces in your environment, you can
>> be including the same name multiple times, at which point I believe
>> the proper semantics is shadowing.
>
>
> Here, you are reasoning in Alain's model where a namespace file is
> just a sequence of operations (alias or open).  So you don't have a
> proper notion of namespace model and instead understand a namespace
> file as its immediate action on the current scope.

Correct.

> This is find to reason in the small, to reason about the imports of a
> module source file.
>
> But I don't think this is a good model that would scale to providing a
> unified view of all available libraries where you wish to construct
> namespaces a posteriori by combining existing ones.

Like Alain, I'm not convinced that hierarchy materially helps matters,
though it doesn't seem implausible either.

As a separate note, especially for a first version, but perhaps
forever, it seems to me preferable to have /building/ of namespaces
happen at the build-system level, and /references/ to namespaces
happen at the soruce level.  In other words, being able to explicitly
do things like open, alias, and qualify expressions using namespaces
seem essential at the source level.  But constructing them at the
source seems less important.

It's possible that a language like the one you and Gabriel propose for
manipulating namespaces is important.  I just don't see the full
detail.


>>> Hope this helps,
>>
>>
>> I think what I'm mostly confused about is what is the nature of the
>> hierarchical namespace system you're thinking about. What are the
>> operations available?
>
>
> The model is what has been described by Gabriel:
>
>   http://gallium.inria.fr/~scherer/namespaces/spec.pdf

Yes!  I somehow missed this earlier, but read it over this morning.  I
now have a better sense of what you mean, but it would be even more
instructive to have some examples.  Would it be possible to put
together some source examples on github (or wherever) that we could
look over?  I find it quite helpful to see examples, rather than just
the grammar of the DSL.

> and
>
>   http://gallium.inria.fr/~scherer/namespaces/pack_et_functor_pack.html
>
>
>> Do you intend to have namespace opens also recursively open all children?
>
>
> I thought this was your wish.
> But I think it is more a drawback than a convenience.

I was thinking of something that only went one-level down: opening a
namespace can open a module, never opening a namespace causing another
namespace to be opened.

That said, in constructing namespaces, I would very much want an
include-like functionality, so that the Core namespace would include
the mappings in the Ucore namespace.  That, however, seems doable at
the build-system level

>
>> Is there an operation for merging namespaces?
>
>
> Yes.
>
>> Aliasing upper namespaces?
>
>
> Yes, you can have several paths leading to the same module object.
>
>
>> Right now, I lack both a precise mental model of what you mean by
>> hierarchical namespaces,
>
>
>   http://gallium.inria.fr/~scherer/namespaces/spec.pdf
>
>
>> and an understanding of the language features that motivate this
>> hierarchy.
>
>
> Although you do not say it clearly, I think you wish at least Alain's
> 2-level hierarchal model.
>
> This may be fine from your point of view:
>
>         level 1 is for JaneStreet different libraries
>         level 2 is for the modules of each library
>
> Most of your libraries are for internal use. So in a way you do not have to
> share your namespaces with the rest of the world. I think the problem comes
> when people want to share the namespaces.
>
> For instance, there could be two actors Foo and Bar proposing complementary
> set of libraries for programming the web.
>
> When doing web programming I may wish to use parts of both providers
> Then, I may wish to structure names as
>
>         Web
>             Foo
>                 Lib1
>                    <list of modules>
>                 Lib2
>                    <list of modules>
>             Bar
>
> In the 2-level model I would be forced to rename namespace files
> to avoid clashes and get some long list of namespace_files that would look
> like:
>
>         Web_Foo_Lib1
>         Web_Foo_Lib2
>         Web_Bar
>
> In this model, the reorganization of namespaces exported by users Foo and
> Bar has into my own view has to be done manually.  The fact that names carry
> some implicit structure is an indication that one has some hierarchy in
> mind.
>
> In the hierachical view, it would suffice to assemble them within the
> language of namespaces, using operations like merge, perhaps pruning, etc..
>
> Besides, I do not think that a 2-level model is so much simpler than a
> hierarchical one.  The apparent simplificty of Alain's proposal is more I
> think in the fact that he does not manipulate namespaces explicitly (such as
> pruning, merging, etc.)

I have no real objection at a technical level to hierarchy (though it
sounds like some difficulty has been introduced by the combination of
auto-open and hierarchy's.  I'm not sure if my proposal that auto-open
only applies to modules is helpful here.).

I'm not sure as a social matter that hierarchy helps, really.  I
roughly think that picking unique names for every module is a little
crazy, but picking unique names for larger packages seems reasonable.
After all, Debian manages to get away with doing this for every
software package they release, without hierearch.

We would surely be happy to adjust our internal package names to avoid
clashes with external projects, and certainly as we release more and
more software to the world, that becomes clearer.  I'd be happier to
do that than to have every refernce to an internal package be prefixed
by "com.janestreet".

That said, I have no strong objections to hierarchy, and I'm sure it
would be helpful in some cases.  I'm just not particularly in favor.

y