[ocaml-platform] on the need and design of OCaml namespaces

[Yaron Minsky]

On Thu, Feb 28, 2013 at 7:44 AM, Gabriel Scherer
<gabriel.scherer at gmail.com> wrote:
> I think the whole discussion of "compilation argument vs. information
> in source header" is misguided. The two places are essentially
> isomorphic, and a robust system could make them equi-expressive. In
> particular, the idea that what putting namespace information outside
> the source code is a regression is incorrect, as the semantics of an
> OCaml program *already* depends on the state of the filesystem on
> which it is compiled and the search path set at compilation time.
> Namespaces by themselves introduce no regression or improvement on
> this aspect.

I strongly disagree with this line of argument.  Sure, they are
technically equally powerful, but that doesn't tell you much.  Good
programming language design is in part about thinking about people,
not just about isomorphisms.

And yes, the OCaml program's semantics depend on the state of the
filesystem.  But right now, if you read the source files, you're
pretty close to understanding the program.  Build systems are hard to
read, and are idiosyncratic, differing a lot from project to project.
The more semantics issues you bury there, the more you hide them from
the programmer, and the worse off you are.

> There are practical reasons why one would favor one or the approach in
> a particular case, but it is not possible to satisfy all criterions
> simultaneously.
>
> 1. People are very sensitive about *source language* changes, not so
> much about changes in compilation interface (perceived as anecdotical
> tooling); this makes it easier to suggest new flags than new language
> features. Arguably both should treated with the same care, and in
> particular design decisions regarding tooling should be inspected with
> scientific rigor. This doesn't happen in practice (which of the
> proposed tool changes during this discussion was accompanied with a
> rigorous specification of the semantics?), so tooling changes are
> easier and faster to get in a language release, and easier to get
> half-specified and half-baked.

I agree that it's more work to make a language change.  But
programming in the large is important, and is worth putting real work
into.  Up until now, the OCaml community hasn't worried about the
issue, and as the community grows up and the use of the language
grows, I believe we need to get this right.

> 2. Such header information may need to be shared by several files of a
> project and may become a source of painful redundancy. Compilation
> flags are an easy way to factorize these redundancies away. In
> principle this could also be done in the source code (by having
> separate files with this source information, and a semantic
> #include-like language construct), but this means yet another language
> change that has to be carefully designed.

We have all of this overhead now within Jane Street, with packed
libraries being the rule.  My experience is it's just not that big of
a deal.  Putting this in the build files is a clear problem in my
mind.

> 3. People naturally share source snippets without thinking of sharing
> the surrounding contextual information (the state of the compilation
> environment), so in practice having information in the source helps to
> be more robust wrt. environmental change. In mails, Q&A sites, etc.,
> sharing source text is better than a tarball with a _tags file or
> whatnot.

Agreed.  This seems like an argument in favor of opening namespaces
explicitly in the source.

> It's well and good to hope that any semantic features associated to
> namespaces will eventually end up in the source language as well. But
> that may be significantly harder than agreeing on those features in
> the first place. Because picking a good syntax is really hard, and
> because this may require more change to other tools that are not part
> of the OCaml compiler distribution, and is therefore more costly for
> everyone involved (this is Alain's understandable reason for shunning
> any such change). So I would like to encourage people to discuss which
> semantics they need and want, what a good design would be, rather than
> whether to implement this inside or outside the source language.

I disagree, precisely because the presence or absence of namespace
declarations in the source language is in my mind an important part of
the semantics of namespaces.

> On Thu, Feb 28, 2013 at 10:01 AM, Erkki Seppala
> <flux-ocaml-platform at inside.org> wrote:
>> TLDR; I agree with the parts of proposal suggesting statement "namespace
>> X" to be introduced to .ml and .mli files and then statements "open
>> namespace X", "let namespace Y = X" and their expression forms. And that
>> this all should be controlled from the source code, not the build
>> system.
>>
>> Yaron Minsky <yminsky at janestreet.com> writes:
>>
>>>   My biggest objection to having opens be at the build system level is
>>>   that it makes your code more ambiguous.  When you do namespace
>>>   manipulations, you very much want to see what's happening by
>>>   inspecting the source.  We have a vigorous code review system here,
>>>   and I don't want to start adding code review of the build rules to
>>>   it, and this change would require that.
>>
>> I agree that namespaces should be defined and accessed through the
>> source code. Not the least because it depends on the project's chosen
>> build system on how I should go on changing those definitions. While
>> currently the situation is the same regarding findlib package
>> management, I think it is worth considering that perhaps this kind of
>> namespace system could automatically lead into the build system
>> (ocamldep?)  inferring which packages you need to use and eliminate that
>> piece of configuration from the build system. Only special cases would
>> need more configuration.
>>
>> While C++ might not be the golden standard to look upon on language
>> development, perhaps it can serve as an inspiration. Its namespace
>> system seems to work pretty nicely without complaints from the
>> developers. It has the following constructs:
>>
>> namespace X {
>>   /* introduce values for the namespace X */
>>   int a = 42;
>>   namespace Z {
>>     /* introduce namespace X::Z */
>>     int b = a;
>>   }
>> }
>>
>> namespace X {
>>   /* introduce more values for the namespace X */
>>   int z = 5;
>> }
>>
>> namespace Y = X; /* alias a namespace */
>>
>> int b = X::a; /* access a value in a namespace */
>>
>> using namespace X; /* bring X to current scope (works in function scope
>>                       as well */
>>
>> using X::a; /* bring one value from a namespace to current scope */
>>
>> In implementation side it works by simply relabeling symbols in the
>> generated object files which makes the namespaces open.
>>
>> C++'s compilation system with include files is of course very different
>> from OCaml, but a similar solution to OCaml would mean having a hidden
>> internal name (such as namespace_modulename or possibly even something
>> that like namespace#modulename) for the module created from that
>> compilation unit. It would need to be limited to a single top-level
>> namespace specification per compilation unit, so it would work sort of
>> in reverse compared to C++. For example if x.ml contained statement
>> "namespace Foo", a module Foo#X would be generated instead of plain X.
>>
>> This solution would not allow the same module names from different
>> namespaces to coexist in a single directory, but I don't see this as an
>> essential feature. The source file names already would be the same, and
>> OCaml compiler produces object files to the same directory as the
>> source. If you have a build system copying files around, it could (and
>> perhaps should!) preserve the directory hierarchy.
>>
>>> - NAMESPACES WITH VALUES.  I have argued for allowing the opening of a
>>>   namespace to also implicitly open some modules, this essentially
>>
>> The previosuly described system would also eliminate the C++-like
>> ability of introducing values to a namespace, unless special module
>> names or other changes, such as an automatic opening of some modules,
>> are implemented. It would be nice to be able to define such top-level
>> values from multiple compilation units, but I'm uncertain how that would
>> be implemented.
>>
>> This approach would need the compiler to find many similarly named .cmi
>> files and decide the proper one to use based on their contents, as
>> mentioned in a previous proposal.
>>
>> Everyone seems to agree that # is a good operator for accessing modules
>> (or values) from namespaces. Or maybe we could go with \ ;-).
>>
>>>   But what Alain is proposing is to make opening a namespace silent at
>>>   the source level.  This strikes me as a grave error.
>>
>> I agree. At worst this could mean that in large projects developers let
>> the build system recompile their files so they can see what namespaces
>> they are using. It seems much better to me to just read it from the
>> source code, which is the same regardless of the project.
>>
>> --
>>   _____________________________________________________________________
>>      / __// /__ ____  __               http://www.modeemi.fi/~flux/\   \
>>     / /_ / // // /\ \/ /                                            \  /
>>    /_/  /_/ \___/ /_/\_\@modeemi.fi                                  \/
>> _______________________________________________
>> Platform mailing list
>> Platform at lists.ocaml.org
>> http://lists.ocaml.org/listinfo/platform