[ocaml-platform] An alternative proposal for namespaces

[Leo White]

>> If omake continues to use "ocamldep -modules" then it will have to
>> implement the resolution strategy. To support *search path files* this
>> will involve either parsing them itself or using the proposed "ocamlns
>> ... -resolve Foo#Bar".
>
> With my proposal, ocamldep -modules will emit two kinds of dependencies: file dependencies for namespaces modules
> (resolved by ocamldep itself, using search path files) and module dependencies (for non-namespaced modules), to be
> resolved by the build exactly as of today (only using -I 
> directories, ignoring search path files).  Basically, we keep the legacy behavior (-I directories) for non-namespaced
> modules, and we provide a simpler API between ocamldep and the build system for namespaced modules (equivalent to
> "ocamldep" without -modules, but precise even in presence of generated source files or "renamed" units).  There will be
> no new resolution strategy to be implemented in, say, omake, just a trivial adaptation of the "ocamldep -module"
> postprocessing to accept file dependency directly.

This is the strategy I have referred to as "regular ocamldep with
generated search path files", it works just as well with simple
namespaces. The only difference is that the build system generates the
search path file to give to ocamldep, rather than making the user write
it by hand.

As a side note, I think that "ocamldep -modules" should continue to be a
purely syntactic version that ignores the search path. It is regular
ocamldep that should be used for this purpose.

>> I'm not particularly worried about hypothetical build systems. If you
>> want to implement such a build system then you should really add hooks
>> into the OCaml compiler. This argument also assumes that catching
>> "Sys.file_exists" is fine but catching "Sys.readdir" is impossible.
>
> No, this argument does not assume that.  But catching Sys.readdir is useless, since you don't know which files the
> compiler is interested in. The tool would have to assume that the dependency is on the entire directory, which is of
> course way too weak.

The tool would only have to know what files it could produce, but it
should already know that to answer Sys.file_exists queries.

> I've done an experiment with a build system based on capturing open/stat calls, and it worked really well, not only for
> ocaml, but also for C code.

This assumes that a C compiler won't read a directory (say to cache its
contents) in order to check for the existence of a file. It is not
exactly the most robust basis for a build system, which is probably why
it is only a hypothetical build system.

> Sorry, I maintain that my strategy will allow to have a better ocamldep which will result in simpler support in, say,
> omake and ocamlbuild.

It doesn't allow better ocamldep results, the results are the same. You
simply pass in a search path file, generated by the build system, that
lists all the target files.

> I also maintain that people using simple namespaces could have weird behaviors of their build
> system (errors appearing or disappearing after a 'make clean') even
> though dependencies are "correct".

Depending on how it is implemented, they *may* fail to get a warning
that they should get, but only in a situation where the warning is
irrelevant.

>  And I maintain that a
> build system based on capturing file system accesses could be a viable strategy, and that this wouldn't work any more if
> the compiler starts to load whole directories.

And I maintain that if checking for a file's existence by reading the
contents of a directory breaks it then it is not a viable build system.

> Consider this piece of code:
>
>  open namespace Foo
>  module X = Bar
>
> You decide to rename the namespace Foo in your project to Baz but you forget to update this file.  You still have a
> foo-bar.cmi in your directory, even though this file is now out of scope of your build system (it considers it as
> "source").  When you compile the wrong code above, the compiler does not complain, and the build system probably
> considers that foo-bar.cmi is a dependency of this code. Now you "make clean", and boom, the code does not compile any
> more.  (Or you commit your changes and other people from your update and get the same error.) Or, even worse, you start
> changing some more code, and you get incomprehensible "Interface mismatch" because you still use the "dead" foo-bar.cmi.

This kind of behaviour already exists in OCaml. Consider this piece of code:

    type t = Bar.t (* Bar only contains type definitions *)

If you rename bar.mli to baz.mli but don't remove bar.cmi then it will
continue to compile until you run "make clean". The moral of the story
is don't leave old .cmi files lying around.

I really don't think that preventing a very unlikely scenario, which can
already happen anyway, is a good reason to make namespaces significantly
less convenient for the average user.