[ocaml-platform] on the need and design of OCaml namespaces

[Gabriel Scherer]

Yaron Minsky <yminsky at janestreet.com> wrote:
> One thing I'll say is that it is important to be able to add values,
> and not just modules, to the namespace.  Open Core.Std also adds
> top-level values, as does the traditional standard library (i.e.,
> Pervasives), and I don't want to lose that.

This is handled as a "bonus feature" discussed in
http://gallium.inria.fr/~scherer/namespaces/pack_et_functor_pack.html
, under the name "flat access". The idea is that adding values to
namespaces directly doesn't work very well (in OCaml, values cannot
live outside a compilation unit), but you can have some modules that
will be explicitly opened if you open the namespace (or included if
you use it as a module). However, we're not yet sure that this works
as well as the other aspects of the design, and would recommend
starting without it to get a feel of the system.

On Tue, Feb 26, 2013 at 3:38 PM, Yaron Minsky <yminsky at janestreet.com> wrote:
> Leo, do you have a summary of what you don't like about Alain's
> namespace proposal?  I'm not presently able to identify any obvious
> weaknesses in it.  The downsides I see are:
>
> - UNSANITARY: Having both "open namespace Core.Std" and Core_List as
>   names seems a little unsanity.  Indeed, to provide a decent user
>   experience, you probably want to hide the Core_List name almost
>   everywhere.  You don't want it showing up in error messages,
>   documentation, source files, etc.  When you need to do a bunch of
>   work to hide something, maybe it's better not to include it at all.

I want to point out that Alain's idea of using long, hopefully-unique
names for the source files, that you deem "unsanitary" here, is also a
workaround for the problem of internal name conflict. In Alain's
proposal, internal names conflicts are (hopfeully) avoided with no
implementation change, just by judicious name choices and letting the
implementation do its work. This imposes this "unsanitary" aspect you
mention, but it is a complexity tradeoff: the other alternative would
be to design a more clever implementation to choose internal names
(than the module's source filename), which could be hidden from the
user (no exposed Core_List) but require an implementation change. I
think that could warrant an exploration (because it could support use
cases such as linking together two versions of the same library, which
isn't possible with a purely filename-based solution), but in the
context Alain's proposal this "unsanitary" aspect has more advantages
than downsides.

> - NO HIDING: I'm not sure that the other namespace proposals do
>   support this, but I'd like to be able to hide some modules so that
>   they are not reachable outside of the namespace.  We can do this
>   with the current Core.Std, but I don't see how to do it in Alain's
>   proposal.

I'm surprised by this "hiding" idea. What does it mean, and what would
be an use case for that?

> Are there other issues I'm missing?

My gut feeling is that a hierarchical model would add little
complexity to Alain's proposal and give a saner semantics to "open
namespace". With a flat model there can be no real notion of "open
namespace", only handcrafted renamings en masse (Core_list -> List,
Core_array -> Array...). (On the other hand "open namespace", if they
are done at the source level, are a source language change, while none
are necessary so far, so they will require more work to handle in any
case.)

There is the problem of internal name conflicts, but that is one of
the more "advanced" questions that may be left out of a first
experiment.

When deciding to leave out some questions for later, it is however
important to wonder whether we're sure that we will actually be able
to extend the design to support them. Some early design choices may
make future extensions harder (without breaking compatibility). One
solution is to make a coherent design that covers advanced features,
and decide to implement only a subset. One other is to try to guess
what will be easy to add afterwards, and be lucky. Finally, one can
experiment with a first design and move to another, but without
committing to backward compatibility (meaning no language release
between the two design iterations).

My personal guess is that
- "flat access" will be easy an easy extension.
- "hierarchical rather than flat mappings" will be relatively easy in
theory but make a painful transition period for providers that have
modules organized flatly and desire to change; but it depends on a
question of "seeing non-leaf namespaces as modules" that Alain's flat
namespaces don't need to consider.
- "extending the mapping description language" must be planned for
beforehand to avoid growing pains.
- "finer implementation of internal names to avoid linking conflicts"
is yet of unknown difficulty, either today and after any first
proposal not tackling this aspect is acted upon.

>
> y
>
> On Tue, Feb 26, 2013 at 9:30 AM, Yaron Minsky <yminsky at janestreet.com> wrote:
>> On Tue, Feb 26, 2013 at 8:03 AM, Alain Frisch <alain.frisch at lexifi.com> wrote:
>>> On 02/25/2013 10:53 PM, Yaron Minsky wrote:
>>>>
>>>> I understand your point Alain, but while what you're saying is
>>>> technically reasonable, I think it doesn't hold together.  When
>>>> programming in the large, it is useful to be able to manipulate the
>>>> namespace and group parts of the world together.
>>>
>>> Can you give concrete examples of which manipulations are desired (and why)?
>>
>> I think the conversation has gotten confused.  You said "who needs
>> pack?  Just use hierarchically named modules".  I'm saying: that's
>> crazy, you need to be able to do simple manipulations of namespaces
>> (like the ones implied by "open Core.Std" or "open Core.Stable".)
>>
>> Now you're pointing out that your namespace proposal covers the
>> manipulations I'm describing.  That may well be right, and I wasn't
>> contesting that point.
>>
>> All I'm saying is that simply relying on long module names without any
>> kind of explicit namespace control does not scale.  I stand by that,
>> without necessarily objecting to your namespace proposal.
>>
>>>> Many libraries, not
>>>> just Core or Async, want to be able to remap the world by adding a
>>>> collection of related names to the namespace.  The ability to do the
>>>> moral equivalent of:
>>>>
>>>>     open Core.Std
>>>>
>>>> is powerful and important.  Your proposal of having people add
>>>> prefixes to module names does not fit the bill, and the resulting
>>>> system does not, in my opinion, scale.
>>>
>>>
>>> I believe my proposal covers this use case.  If you depend a lot on a
>>> specify library which exports many modules, you might indeed want to avoid
>>> using long names everywhere to access these modules.  That's why I propose
>>> to give simple ways to alias long modules names to short ones, in a way
>>> which can be factorized (with external mapping files).
>>>
>>> In my proposal, the equivalent of "open Core.Std" would simply be to tell
>>> the compiler (through a command-line option or with a directive in the code)
>>> to use a mapping file.  The same can be done within the library itself.
>>>
>>> Here is a minimalistic version of my proposal, restricted to specifying
>>> those mapping files on the compiler and tools command-lines.  To make it
>>> clear that this is only about mapping module references to compiled units,
>>> let's piggy-back the -I option.  If its argument is a file with a .ns suffix
>>> and not a directory, the compiler interprets it as a mapping file (a
>>> sequence of lines of the form "Module_name =
>>> relative_path_to_compiled_unit", e.g. "List = core_list"), and when this -I
>>> option is considered during the resolution of a module reference (such as
>>> "List") and the module is defined in the file, the compiler simply resolves
>>> the module to the corresponding unit.
>>>
>>> This approach could also be used to restrict which units are visible by the
>>> compiler (and avoid repeated lookup on the file system) without moving files
>>> around on the file system.
>>>
>>> ocamldep (without -modules) would apply the same logic; ocamldep -modules
>>> could either implement the same logic, or leave it to the build system
>>> interpreting its output (allowing more dynamic scenarios where the mapping
>>> file themselves are generated).  I expect ocamldoc to work mostly
>>> out-of-the-box, even though one could think about using the mapping files
>>> (in reverse direction) to provide shorter names in the generated
>>> documentation (or not).
>>>
>>> Users of a library are never forced to use the mapping files, and they can
>>> always refer to a module with its full name (provided the corresponding -I
>>> <path> is used).
>>>
>>> The OCaml stdlib would be adapted to use longer names (stdlib_list,
>>> stdlib_array, etc) and shipped with a stdlib.ns file opened by default
>>> (unless -nostdlib is used).  To be clear: this mapping file will be used by
>>> the stdlib itself, so references within itself don't need to use long names.
>>>
>>> I'm interested to see concrete examples of manipulation or scenarios not be
>>> covered by this proposal.
>>
>> I am unaware of any.  I think this does mostly support the
>> Core.Std/Core.Stable tricks we use.
>>
>> One thing I'll say is that it is important to be able to add values,
>> and not just modules, to the namespace.  Open Core.Std also adds
>> top-level values, as does the traditional standard library (i.e.,
>> Pervasives), and I don't want to lose that.
>>
>> y
> _______________________________________________
> Platform mailing list
> Platform at lists.ocaml.org
> http://lists.ocaml.org/listinfo/platform