[ocaml-platform] on the need and design of OCaml namespaces

[Yaron Minsky]

On Thu, Feb 28, 2013 at 9:30 AM, Gabriel Scherer
<gabriel.scherer at gmail.com> wrote:
>> I disagree, precisely because the presence or absence of namespace
>> declarations in the source language is in my mind an important part of
>> the semantics of namespaces.
>
> It is a very important part of the *syntax* of namespaces.

I'm not a PL guy by training, so maybe I have the terminology wrong.
I would have thought that things like local namespace opens, which
interact with the scoping of the language, would be part of the
semantics.

But I suppose if you include in the language the build scripts, then
where that specification goes is perhaps a syntactic question.

But it is, in either case, important, and more important than whether
you use a paren or a square bracket.

> On Thu, Feb 28, 2013 at 3:23 PM, Yaron Minsky <yminsky at janestreet.com> wrote:
>> On Thu, Feb 28, 2013 at 7:44 AM, Gabriel Scherer
>> <gabriel.scherer at gmail.com> wrote:
>>> I think the whole discussion of "compilation argument vs. information
>>> in source header" is misguided. The two places are essentially
>>> isomorphic, and a robust system could make them equi-expressive. In
>>> particular, the idea that what putting namespace information outside
>>> the source code is a regression is incorrect, as the semantics of an
>>> OCaml program *already* depends on the state of the filesystem on
>>> which it is compiled and the search path set at compilation time.
>>> Namespaces by themselves introduce no regression or improvement on
>>> this aspect.
>>
>> I strongly disagree with this line of argument.  Sure, they are
>> technically equally powerful, but that doesn't tell you much.  Good
>> programming language design is in part about thinking about people,
>> not just about isomorphisms.
>>
>> And yes, the OCaml program's semantics depend on the state of the
>> filesystem.  But right now, if you read the source files, you're
>> pretty close to understanding the program.  Build systems are hard to
>> read, and are idiosyncratic, differing a lot from project to project.
>> The more semantics issues you bury there, the more you hide them from
>> the programmer, and the worse off you are.
>>
>>> There are practical reasons why one would favor one or the approach in
>>> a particular case, but it is not possible to satisfy all criterions
>>> simultaneously.
>>>
>>> 1. People are very sensitive about *source language* changes, not so
>>> much about changes in compilation interface (perceived as anecdotical
>>> tooling); this makes it easier to suggest new flags than new language
>>> features. Arguably both should treated with the same care, and in
>>> particular design decisions regarding tooling should be inspected with
>>> scientific rigor. This doesn't happen in practice (which of the
>>> proposed tool changes during this discussion was accompanied with a
>>> rigorous specification of the semantics?), so tooling changes are
>>> easier and faster to get in a language release, and easier to get
>>> half-specified and half-baked.
>>
>> I agree that it's more work to make a language change.  But
>> programming in the large is important, and is worth putting real work
>> into.  Up until now, the OCaml community hasn't worried about the
>> issue, and as the community grows up and the use of the language
>> grows, I believe we need to get this right.
>>
>>> 2. Such header information may need to be shared by several files of a
>>> project and may become a source of painful redundancy. Compilation
>>> flags are an easy way to factorize these redundancies away. In
>>> principle this could also be done in the source code (by having
>>> separate files with this source information, and a semantic
>>> #include-like language construct), but this means yet another language
>>> change that has to be carefully designed.
>>
>> We have all of this overhead now within Jane Street, with packed
>> libraries being the rule.  My experience is it's just not that big of
>> a deal.  Putting this in the build files is a clear problem in my
>> mind.
>>
>>> 3. People naturally share source snippets without thinking of sharing
>>> the surrounding contextual information (the state of the compilation
>>> environment), so in practice having information in the source helps to
>>> be more robust wrt. environmental change. In mails, Q&A sites, etc.,
>>> sharing source text is better than a tarball with a _tags file or
>>> whatnot.
>>
>> Agreed.  This seems like an argument in favor of opening namespaces
>> explicitly in the source.
>>
>>> It's well and good to hope that any semantic features associated to
>>> namespaces will eventually end up in the source language as well. But
>>> that may be significantly harder than agreeing on those features in
>>> the first place. Because picking a good syntax is really hard, and
>>> because this may require more change to other tools that are not part
>>> of the OCaml compiler distribution, and is therefore more costly for
>>> everyone involved (this is Alain's understandable reason for shunning
>>> any such change). So I would like to encourage people to discuss which
>>> semantics they need and want, what a good design would be, rather than
>>> whether to implement this inside or outside the source language.
>>
>> I disagree, precisely because the presence or absence of namespace
>> declarations in the source language is in my mind an important part of
>> the semantics of namespaces.
>>
>>> On Thu, Feb 28, 2013 at 10:01 AM, Erkki Seppala
>>> <flux-ocaml-platform at inside.org> wrote:
>>>> TLDR; I agree with the parts of proposal suggesting statement "namespace
>>>> X" to be introduced to .ml and .mli files and then statements "open
>>>> namespace X", "let namespace Y = X" and their expression forms. And that
>>>> this all should be controlled from the source code, not the build
>>>> system.
>>>>
>>>> Yaron Minsky <yminsky at janestreet.com> writes:
>>>>
>>>>>   My biggest objection to having opens be at the build system level is
>>>>>   that it makes your code more ambiguous.  When you do namespace
>>>>>   manipulations, you very much want to see what's happening by
>>>>>   inspecting the source.  We have a vigorous code review system here,
>>>>>   and I don't want to start adding code review of the build rules to
>>>>>   it, and this change would require that.
>>>>
>>>> I agree that namespaces should be defined and accessed through the
>>>> source code. Not the least because it depends on the project's chosen
>>>> build system on how I should go on changing those definitions. While
>>>> currently the situation is the same regarding findlib package
>>>> management, I think it is worth considering that perhaps this kind of
>>>> namespace system could automatically lead into the build system
>>>> (ocamldep?)  inferring which packages you need to use and eliminate that
>>>> piece of configuration from the build system. Only special cases would
>>>> need more configuration.
>>>>
>>>> While C++ might not be the golden standard to look upon on language
>>>> development, perhaps it can serve as an inspiration. Its namespace
>>>> system seems to work pretty nicely without complaints from the
>>>> developers. It has the following constructs:
>>>>
>>>> namespace X {
>>>>   /* introduce values for the namespace X */
>>>>   int a = 42;
>>>>   namespace Z {
>>>>     /* introduce namespace X::Z */
>>>>     int b = a;
>>>>   }
>>>> }
>>>>
>>>> namespace X {
>>>>   /* introduce more values for the namespace X */
>>>>   int z = 5;
>>>> }
>>>>
>>>> namespace Y = X; /* alias a namespace */
>>>>
>>>> int b = X::a; /* access a value in a namespace */
>>>>
>>>> using namespace X; /* bring X to current scope (works in function scope
>>>>                       as well */
>>>>
>>>> using X::a; /* bring one value from a namespace to current scope */
>>>>
>>>> In implementation side it works by simply relabeling symbols in the
>>>> generated object files which makes the namespaces open.
>>>>
>>>> C++'s compilation system with include files is of course very different
>>>> from OCaml, but a similar solution to OCaml would mean having a hidden
>>>> internal name (such as namespace_modulename or possibly even something
>>>> that like namespace#modulename) for the module created from that
>>>> compilation unit. It would need to be limited to a single top-level
>>>> namespace specification per compilation unit, so it would work sort of
>>>> in reverse compared to C++. For example if x.ml contained statement
>>>> "namespace Foo", a module Foo#X would be generated instead of plain X.
>>>>
>>>> This solution would not allow the same module names from different
>>>> namespaces to coexist in a single directory, but I don't see this as an
>>>> essential feature. The source file names already would be the same, and
>>>> OCaml compiler produces object files to the same directory as the
>>>> source. If you have a build system copying files around, it could (and
>>>> perhaps should!) preserve the directory hierarchy.
>>>>
>>>>> - NAMESPACES WITH VALUES.  I have argued for allowing the opening of a
>>>>>   namespace to also implicitly open some modules, this essentially
>>>>
>>>> The previosuly described system would also eliminate the C++-like
>>>> ability of introducing values to a namespace, unless special module
>>>> names or other changes, such as an automatic opening of some modules,
>>>> are implemented. It would be nice to be able to define such top-level
>>>> values from multiple compilation units, but I'm uncertain how that would
>>>> be implemented.
>>>>
>>>> This approach would need the compiler to find many similarly named .cmi
>>>> files and decide the proper one to use based on their contents, as
>>>> mentioned in a previous proposal.
>>>>
>>>> Everyone seems to agree that # is a good operator for accessing modules
>>>> (or values) from namespaces. Or maybe we could go with \ ;-).
>>>>
>>>>>   But what Alain is proposing is to make opening a namespace silent at
>>>>>   the source level.  This strikes me as a grave error.
>>>>
>>>> I agree. At worst this could mean that in large projects developers let
>>>> the build system recompile their files so they can see what namespaces
>>>> they are using. It seems much better to me to just read it from the
>>>> source code, which is the same regardless of the project.
>>>>
>>>> --
>>>>   _____________________________________________________________________
>>>>      / __// /__ ____  __               http://www.modeemi.fi/~flux/\   \
>>>>     / /_ / // // /\ \/ /                                            \  /
>>>>    /_/  /_/ \___/ /_/\_\@modeemi.fi                                  \/
>>>> _______________________________________________
>>>> Platform mailing list
>>>> Platform at lists.ocaml.org
>>>> http://lists.ocaml.org/listinfo/platform