[opam-devel] OPAM on Windows (1.3 Roadmap)

David Allsopp david.allsopp at metastack.com
Wed Nov 5 18:06:32 GMT 2014

Louis Gesbert wrote:
> Thanks for these comments !
> * Rewrite the parallel command engine
> This is well underway, I've got a working prototype with limited backends
> at the moment.
> I'm not relying on fork anymore, but we'll still need some windows way of
> implementing `Unix.waitpid 0`; and to add OS switches to the build system.

Although Windows has them, Windows and Unix process groups aren't quite the same concept. I expect you want more the behaviour of Unix.waitpid (-1) (any child) which is sort of what I implemented in my branch. In fact, the way waiting works under Windows, the stub I implemented specifically waits for any one of a list of PIDs to terminate. 

> * Native system manipulation
> I agree on re-coding cp, mv etc.; here is also a possible idea, it may be
> worth checking if it's practical: make sure we may work if limited to
> busybox, and include some Windows version of busybox in the Windows
> distribution.

But unless you guarantee that all 700/800 packages in OPAM do the same thing, at some point you will need a "real" environment - make, patch, m4, etc (you can't even get as far as findlib without m4) and ocamlopt's need for a C compiler, or at least an assembler, is never far away - and the two MSVC ports will *always* require you to install the Windows SDK separately to get the Microsoft assembler (I don't think packaging that in other setups is permitted under the licence).

I completely agree that making OPAM more portable is a good idea (ocaml-fileutils would be a very good thing to use), but my point is that it has *nothing* to do with making Windows usage of OPAM better, at least for non-toy usage so, at least for this part of the roadmap, it feels like effort targeted at the wrong problem.

> * Test & fix on Windows
> > "Cygwin, then, maybe, native." I'm afraid implies a crucial
> > misunderstanding of how OCaml-on-Windows works.
> I know that Cygwin is hardly a step towards native windows compatibility
> and about the cross-compilation tangle. But given the difference in cost,
> I think it's worth ensuring Cygwin is well supported before attacking the
> bulk of the problem: you quickly trip over tiny issues even with Cygwin at
> the moment.

What tiny issues have you seen? I've just bootstrapped OPAM on OCaml 4.01.0 on Cygwin 1.7 with no problems, as I expected. I've been able to switch into 4.01.0, run opam install opam-lib --deps-only and re-compile OPAM 1.2 using its "own" compiler. I had trouble with 4.02.1, but that's because there's a bug in ocamlbuild, which is hardly OPAM's fault! I've put a patch at http://people.ds.cam.ac.uk/~dra27/patches/ocaml-4.02.1+cygwin.patch and will shortly see if the issue is already mentioned in Mantis and with that patch was able to switch into 4.02.1 and similarly bootstrap OPAM.

Incidentally, the problem with Cygwin and OCaml 4.02.1 is a reasonable demonstration of why stripping out external calls to cp, etc. can be a BAD idea! Cygwin neatly masks the ".exe" problem if you do things POSIX-ly. Consider:

$ test -e /usr/bin/ocaml && echo OCaml is installed

In a Cygwin bash prompt, this will echo that OCaml is indeed installed. Now, in reality /usr/bin/ocaml is C:\cygwin\bin\ocaml.exe and test -e /usr/bin/ocaml.exe will also be true. ocamlbuild fails because for reasons best understood by Nicolas Pouillard, it replaces Sys.file_exists with a manual attempt to read the directory entries itself (see ocamlbuild/my_std.ml and notes in ocamlbuild/Changelog) - and in so doing bypasses Cygwin's compatibility layers which handle the ".exe" problem for Cygwin.


More information about the opam-devel mailing list