[ocaml-platform] on the need and design of OCaml namespaces

[Gabriel Scherer]

> I disagree, precisely because the presence or absence of namespace
> declarations in the source language is in my mind an important part of
> the semantics of namespaces.

It is a very important part of the *syntax* of namespaces.

On Thu, Feb 28, 2013 at 3:23 PM, Yaron Minsky <yminsky at janestreet.com> wrote:
> On Thu, Feb 28, 2013 at 7:44 AM, Gabriel Scherer
> <gabriel.scherer at gmail.com> wrote:
>> I think the whole discussion of "compilation argument vs. information
>> in source header" is misguided. The two places are essentially
>> isomorphic, and a robust system could make them equi-expressive. In
>> particular, the idea that what putting namespace information outside
>> the source code is a regression is incorrect, as the semantics of an
>> OCaml program *already* depends on the state of the filesystem on
>> which it is compiled and the search path set at compilation time.
>> Namespaces by themselves introduce no regression or improvement on
>> this aspect.
>
> I strongly disagree with this line of argument.  Sure, they are
> technically equally powerful, but that doesn't tell you much.  Good
> programming language design is in part about thinking about people,
> not just about isomorphisms.
>
> And yes, the OCaml program's semantics depend on the state of the
> filesystem.  But right now, if you read the source files, you're
> pretty close to understanding the program.  Build systems are hard to
> read, and are idiosyncratic, differing a lot from project to project.
> The more semantics issues you bury there, the more you hide them from
> the programmer, and the worse off you are.
>
>> There are practical reasons why one would favor one or the approach in
>> a particular case, but it is not possible to satisfy all criterions
>> simultaneously.
>>
>> 1. People are very sensitive about *source language* changes, not so
>> much about changes in compilation interface (perceived as anecdotical
>> tooling); this makes it easier to suggest new flags than new language
>> features. Arguably both should treated with the same care, and in
>> particular design decisions regarding tooling should be inspected with
>> scientific rigor. This doesn't happen in practice (which of the
>> proposed tool changes during this discussion was accompanied with a
>> rigorous specification of the semantics?), so tooling changes are
>> easier and faster to get in a language release, and easier to get
>> half-specified and half-baked.
>
> I agree that it's more work to make a language change.  But
> programming in the large is important, and is worth putting real work
> into.  Up until now, the OCaml community hasn't worried about the
> issue, and as the community grows up and the use of the language
> grows, I believe we need to get this right.
>
>> 2. Such header information may need to be shared by several files of a
>> project and may become a source of painful redundancy. Compilation
>> flags are an easy way to factorize these redundancies away. In
>> principle this could also be done in the source code (by having
>> separate files with this source information, and a semantic
>> #include-like language construct), but this means yet another language
>> change that has to be carefully designed.
>
> We have all of this overhead now within Jane Street, with packed
> libraries being the rule.  My experience is it's just not that big of
> a deal.  Putting this in the build files is a clear problem in my
> mind.
>
>> 3. People naturally share source snippets without thinking of sharing
>> the surrounding contextual information (the state of the compilation
>> environment), so in practice having information in the source helps to
>> be more robust wrt. environmental change. In mails, Q&A sites, etc.,
>> sharing source text is better than a tarball with a _tags file or
>> whatnot.
>
> Agreed.  This seems like an argument in favor of opening namespaces
> explicitly in the source.
>
>> It's well and good to hope that any semantic features associated to
>> namespaces will eventually end up in the source language as well. But
>> that may be significantly harder than agreeing on those features in
>> the first place. Because picking a good syntax is really hard, and
>> because this may require more change to other tools that are not part
>> of the OCaml compiler distribution, and is therefore more costly for
>> everyone involved (this is Alain's understandable reason for shunning
>> any such change). So I would like to encourage people to discuss which
>> semantics they need and want, what a good design would be, rather than
>> whether to implement this inside or outside the source language.
>
> I disagree, precisely because the presence or absence of namespace
> declarations in the source language is in my mind an important part of
> the semantics of namespaces.
>
>> On Thu, Feb 28, 2013 at 10:01 AM, Erkki Seppala
>> <flux-ocaml-platform at inside.org> wrote:
>>> TLDR; I agree with the parts of proposal suggesting statement "namespace
>>> X" to be introduced to .ml and .mli files and then statements "open
>>> namespace X", "let namespace Y = X" and their expression forms. And that
>>> this all should be controlled from the source code, not the build
>>> system.
>>>
>>> Yaron Minsky <yminsky at janestreet.com> writes:
>>>
>>>>   My biggest objection to having opens be at the build system level is
>>>>   that it makes your code more ambiguous.  When you do namespace
>>>>   manipulations, you very much want to see what's happening by
>>>>   inspecting the source.  We have a vigorous code review system here,
>>>>   and I don't want to start adding code review of the build rules to
>>>>   it, and this change would require that.
>>>
>>> I agree that namespaces should be defined and accessed through the
>>> source code. Not the least because it depends on the project's chosen
>>> build system on how I should go on changing those definitions. While
>>> currently the situation is the same regarding findlib package
>>> management, I think it is worth considering that perhaps this kind of
>>> namespace system could automatically lead into the build system
>>> (ocamldep?)  inferring which packages you need to use and eliminate that
>>> piece of configuration from the build system. Only special cases would
>>> need more configuration.
>>>
>>> While C++ might not be the golden standard to look upon on language
>>> development, perhaps it can serve as an inspiration. Its namespace
>>> system seems to work pretty nicely without complaints from the
>>> developers. It has the following constructs:
>>>
>>> namespace X {
>>>   /* introduce values for the namespace X */
>>>   int a = 42;
>>>   namespace Z {
>>>     /* introduce namespace X::Z */
>>>     int b = a;
>>>   }
>>> }
>>>
>>> namespace X {
>>>   /* introduce more values for the namespace X */
>>>   int z = 5;
>>> }
>>>
>>> namespace Y = X; /* alias a namespace */
>>>
>>> int b = X::a; /* access a value in a namespace */
>>>
>>> using namespace X; /* bring X to current scope (works in function scope
>>>                       as well */
>>>
>>> using X::a; /* bring one value from a namespace to current scope */
>>>
>>> In implementation side it works by simply relabeling symbols in the
>>> generated object files which makes the namespaces open.
>>>
>>> C++'s compilation system with include files is of course very different
>>> from OCaml, but a similar solution to OCaml would mean having a hidden
>>> internal name (such as namespace_modulename or possibly even something
>>> that like namespace#modulename) for the module created from that
>>> compilation unit. It would need to be limited to a single top-level
>>> namespace specification per compilation unit, so it would work sort of
>>> in reverse compared to C++. For example if x.ml contained statement
>>> "namespace Foo", a module Foo#X would be generated instead of plain X.
>>>
>>> This solution would not allow the same module names from different
>>> namespaces to coexist in a single directory, but I don't see this as an
>>> essential feature. The source file names already would be the same, and
>>> OCaml compiler produces object files to the same directory as the
>>> source. If you have a build system copying files around, it could (and
>>> perhaps should!) preserve the directory hierarchy.
>>>
>>>> - NAMESPACES WITH VALUES.  I have argued for allowing the opening of a
>>>>   namespace to also implicitly open some modules, this essentially
>>>
>>> The previosuly described system would also eliminate the C++-like
>>> ability of introducing values to a namespace, unless special module
>>> names or other changes, such as an automatic opening of some modules,
>>> are implemented. It would be nice to be able to define such top-level
>>> values from multiple compilation units, but I'm uncertain how that would
>>> be implemented.
>>>
>>> This approach would need the compiler to find many similarly named .cmi
>>> files and decide the proper one to use based on their contents, as
>>> mentioned in a previous proposal.
>>>
>>> Everyone seems to agree that # is a good operator for accessing modules
>>> (or values) from namespaces. Or maybe we could go with \ ;-).
>>>
>>>>   But what Alain is proposing is to make opening a namespace silent at
>>>>   the source level.  This strikes me as a grave error.
>>>
>>> I agree. At worst this could mean that in large projects developers let
>>> the build system recompile their files so they can see what namespaces
>>> they are using. It seems much better to me to just read it from the
>>> source code, which is the same regardless of the project.
>>>
>>> --
>>>   _____________________________________________________________________
>>>      / __// /__ ____  __               http://www.modeemi.fi/~flux/\   \
>>>     / /_ / // // /\ \/ /                                            \  /
>>>    /_/  /_/ \___/ /_/\_\@modeemi.fi                                  \/
>>> _______________________________________________
>>> Platform mailing list
>>> Platform at lists.ocaml.org
>>> http://lists.ocaml.org/listinfo/platform