[opam-devel] OPAM on Windows (1.3 Roadmap)

Sylvain Le Gall sylvain+ocaml at le-gall.net
Wed Nov 5 11:02:59 GMT 2014


[resending because in the former mail my comment was inlined]

Le Tue Nov 04 2014 at 8:42:09 AM, Louis Gesbert <louis.gesbert at ocamlpro.com>
a écrit :

> Thanks for these comments !
>
> * Rewrite the parallel command engine
>
> This is well underway, I've got a working prototype with limited backends
> at the moment.
> I'm not relying on fork anymore, but we'll still need some windows way of
> implementing `Unix.waitpid 0`; and to add OS switches to the build system.
>
* Native system manipulation
>
> I agree on re-coding cp, mv etc.; here is also a possible idea, it may be
> worth checking if it's practical: make sure we may work if limited to
> busybox, and include some Windows version of busybox in the Windows
> distribution.
>

I am about to release a new version of ocaml-fileutils that have
better implementation of cp, mv than the previous version. You may
consider using it. POSIX implementation of those command require some
work.


ocaml-fileutils has also a low number of dependencies that makes it
more suitable for inclusion.



>
> * Test & fix on Windows
> > "Cygwin, then, maybe, native." I'm afraid implies a crucial
> misunderstanding
> > of how OCaml-on-Windows works.
>
> I know that Cygwin is hardly a step towards native windows compatibility
> and about the cross-compilation tangle. But given the difference in cost, I
> think it's worth ensuring Cygwin is well supported before attacking the
> bulk of the problem: you quickly trip over tiny issues even with Cygwin at
> the moment.
>
> > Related to this, are there general plans at the moment for how OPAM may
> deal
> > with cross-compiling? This becomes relevant with four Windows ports
>
> There is quite some amount of thought about it, but no plan yet.
>
> @Anil, on the "os" variable, we have "cygwin" and "win32", following
> Sys.os_type().
>
>
> Cheers,
> Louis
>
>
> Le dimanche 2 novembre 2014, 20:48:37 David Allsopp a écrit :
> > For OCaml 4.02, instead of spending the time I would normally use on a
> > compiler upgrade bringing my private build scripts up-to-date, I've been
> > doing some work porting OPAM to Windows.
> >
> > I'm chipping away at this - mainly at 'weekends'! - and will publish to
> > dra27/opam on GitHub hopefully within the next fortnight (if someone
> else is
> > also actively working on a Windows port and would like to see the stuff
> > sooner, then ping me a message - there're just too many hacks on the TODO
> > list to bother with pushing commits at the moment). So far I've got opam
> > init working (including installing the base packages) though I'm still
> > polishing some of the scripts for opam config env. Next up is being able
> to
> > switch to another compiler instance and install some packages, but I'm
> > feeling that the bulk of the hard work is already done now.
> >
> > All that said, the main purpose of this message is to offer some comment
> on
> > the portability section of the 1.3 Roadmap
> > (https://github.com/ocaml/opam/wiki/1.3-Roadmap) which I hope will be
> > helpful.
> >
> > * Rewrite the parallel command engine
> > This is a 'de facto' requirement for porting to Windows, because it's
> used
> > to install the base packages and removing it was easy. My own code only
> > avoids Unix.fork (and Unix.wait) for Windows, but it would be beneficial
> for
> > both Unix and Windows to use the same code path, obviously. I simply
> > converted everything to Marshal - the child closure is sent on stdin to
> > additional opam processes (using a special --fork command line switch).
> It
> > seems to be working, though some subtleties with global variables might
> rear
> > their heads when I start installing packages and trying to push the
> --jobs
> > switch later.
> >
> > * Native system manipulation
> > While I think invocations of wget/curl could beneficially be replaced
> with
> > ocamlnet and tar with ocaml-tar, I think replacing all other calls (rm,
> cp,
> > etc.) this is a bad idea or, at least it should certainly not be done on
> the
> > false altar of making Windows-use better/easier. OPAM is a source-based
> > package manager - OCaml itself requires a bash-like environment to
> compile
> > (for *all* Windows ports, including the Microsoft-based ones) and let's
> not
> > forget that a possible bash-free alternative for building OCaml was
> > subsequently removed from the source tree. And that's before looking at
> > packages - it's naïve to imagine, given the majority of package authors
> > being Unix-based in one flavour or another, that there will ever a point
> > where anything approaching a majority of package build systems are
> totally
> > shell-agnostic. It seems to me to be much better to head towards a system
> > which can expect various standard Unix commands to be available (via
> Cygwin,
> > MSYS, GnuWin32, etc.) but aim not to require the actual shell to be Unix
> > (i.e. work within Command Prompt, PowerShell, etc.). In this model, the
> aim
> > is to make Windows sufficiently Unix-like so that supporting Windows
> becomes
> > a matter of small configure-style tweaks already required to support
> various
> > flavours of Unix anyway. In my experience, packages which end up with two
> > build systems (the main one, and something hacked for Windows) end up
> with a
> > very stale, broken Windows build system, because the maintainer quite
> > reasonably ends up releasing later versions without testing whether the
> > Windows build system still works.
> >
> > * Test & fix on Windows
> > "Cygwin, then, maybe, native." I'm afraid implies a crucial
> misunderstanding
> > of how OCaml-on-Windows works. I haven't checked, but OPAM should already
> > compile for *Cygwin* because Cygwin is simply Unix. The only thing which
> may
> > be broken is the odd configure check (i.e. correctly parsing the output
> of
> > uname). But understand that that means compiling OPAM in *Cygwin*'s OCaml
> > (which is OCaml built using ./configure in the normal way, not by using
> the
> > manual files in config/). Having OPAM working for Cygwin is a good idea,
> > indeed it'd be a good to have a Cygwin package of OPAM if someone were
> > willing to maintain it. But it's a trivial target - virtually nothing
> should
> > require altering, because that's the point of Cygwin. So it's either
> > "native" (i.e. the MinGW and MSVC ports) or not really worth talking
> about!
> >
> > The four Windows ports are cross-compiled using a Cygwin bash shell as
> the
> > *environment* (note that a lot of this can be done using an MSYS bash
> shell
> > as the environment too). But the compilers are all cross-compilers
> (although
> > because the architecture for both compilation and target is the same,
> most
> > configure scripts do not realise that they are cross compiling!) - which
> is
> > why the build system can use things like "/bin/sh" (which Cygwin C
> runtime
> > translates it) but the resulting OCaml environment cannot (because the
> > Microsoft Visual C Runtime, unsurprisingly, does not).
> >
> > Fortunately, it's easy to interact with Cygwin commands (rm, cp, etc.)
> > because they're all coded to recognise Windows paths if they're specified
> > with forward slashes (C:/Windows\system32 is a perfectly valid Windows
> path
> > and, indeed, for the Command Interpreter, even C:/"W"indows\""system32
> is a
> > valid path!).
> >
> > My experience (based on my own installation of around 45 or so OCaml and
> > OCaml-related packages) is that building for Windows usually just
> involves
> > overriding some lazy assumptions in build systems (for example, the most
> > common one being the assumption "we're building on Windows" => "object
> files
> > end in .obj") so nothing which can't be quickly committed upstream, or
> > applied in opam-repository patches. I don't expect there will be
> "generic"
> > fixes, beyond trying to cp "foo.exe" if "foo" cannot be found in a
> .install
> > file (which I'm coding up anyway in my fork). What may be useful is to
> have
> > certain non-OCaml packages packaged up in OPAM for Windows-only - for
> > example, openssl, pcre and so on to deal with libraries which aren't
> > normally packaged for Windows.
> >
> > Related to this, are there general plans at the moment for how OPAM may
> deal
> > with cross-compiling? This becomes relevant with four Windows ports -
> one of
> > the big advantages with opam (and the whole model of opam config env) is
> > that it will make it much easier to run all four Windows ports on the
> same
> > machine, but it seems very tedious (and difficult to maintain) to need to
> > specify every single compiler switch 5 times (e.g. 4.02.0, 4.02.0+msvc,
> > 4.02.0+mingw, 4.02.0+msvc64, 4.02.0+mingw64). I was thinking of adding a
> > Windows-only (for now) --target option to opam switch for an enumeration
> of
> > auto/mingw/msvc/mingw64/msvc64. So opam switch 4.02.0+foo
> --target=mingw64
> > would build the 64-bit MinGW port of OCaml in
> ~/.opam/4.02.0+foo+mingw64. It
> > would obviously also be useful for opam files to be able to reference the
> > value of --target.
> >
> >
> > David
> >
> > _______________________________________________
> > opam-devel mailing list
> > opam-devel at lists.ocaml.org
> > http://lists.ocaml.org/listinfo/opam-devel
>
> _______________________________________________
> opam-devel mailing list
> opam-devel at lists.ocaml.org
> http://lists.ocaml.org/listinfo/opam-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ocaml.org/pipermail/opam-devel/attachments/20141105/80cef550/attachment-0001.html>


More information about the opam-devel mailing list