[wg-windows] Improving the state of OCaml on Windows
Alain Frisch
alain.frisch at lexifi.com
Mon Jun 1 10:31:18 BST 2015
Hi all,
Setting up this list is a great initiative!
1. A few words about the use of OCaml under Windows at LexiFi
Windows is one of the most important platform we need to support, and we
deliver GUI applications in both 32 and 64-bit, using the two MSVC
ports. We also ensure compatibility of our code base with the two mingw
ports "just in case" (technically, these ports allow to produce
standalone applications that run even on a plain WinXP box, whereas the
MSVC ports require msvcrtXX.dll, which might not be available). We
don't have any use for the two Cygwin ports.
We are not a "Windows pure player": Linux is also an important platform
for some of our customers and projects, and most our code base is also
supported there.
All our code lives in a single tree whose build is managed mainly by
omake, assuming a Cygwin environment under Windows. Our tree also has
code in other languages (C, Fortran, C#, etc). We integrate a very
small number of third-party OCaml libraries, simply by injecting their
source code into our tree and adding tiny OMakefiles to integrate them
in our global build system. This is often easier than fighting with
upstream Makefiles, etc.
For now, we don't use (nor install) ocamlfind, camlp4 or ocamlbuild.
Our developers work typically in emacs or vim, and ocp-indent is available.
We interact with Windows and Windows libraries (in addition to what
OCaml provides in Sys and Unix) mostly through .Net. To this end, we
have developped CSML, a bridge between .Net and OCaml (
https://www.lexifi.com/csml ). We use .Net for the UI (Windows Forms),
for simple bindings e.g. with Oracle, Outlook, MS Excel, etc.
Our Windows application comes with a built-in small IDE that allows end
users to edit and compile OCaml addins (which are dynlinked in the
application upon startup). This means that our application is shipped
with a minimal toolchain required to support generation of .cmxs files
from source code: ocamlopt, flexdll and some .cmi/.cmx files. Note that
we don't need any external assembler or linker (our users don't need to
install any "native toolchain"):
- flexdll has a mode where it can build simple dll (cmxs) without
relying on an external linker. This wouldn't support linking arbitrary
.obj files (with resources, debug information, etc), but we only need to
support code produced by OCaml.
- We use a modified OCaml backend that replaces emit.mlp with a
direct x86 (32 or 64-bit) machine code emitter and a COFF object file
output (this was developped with the help of OCamlPro; the code is
owned jointly by OCamlPro and LexiFi). In OCaml's trunk, we have
unified emit.mlp/emit_mt.mlp (for both i386 and amd64) to produce an AST
of the generated assembly code, so our binary backend is now just an
extra backend on this AST (in addition to the Intel and AT&T textual
emitters), which simplifies maintenance of this part quite a bit (this
also benefits to all Windows users, since this avoids the duplication
between emit.mlp/emit_nt.mlp, which sometimes resulted in emit_nt.mlp
lagging behind).
Note that this couldn't be used directly to produce main programs
without a native toolchain (with use startup objects, c lib, etc). This
isn't a big theoretical limitation: we could always ship an ocamlrunopt
program that would simply dynlink a main.cmxs file appended to it -- in
the same spirit as ocamlrun.
We also modified OCaml a little bit to make it easier to relocate the
toolchain (which is important when you ship binary code that can be
installed anywhere). For instance, we changed the default path for
looking up the standard library to make it relative to the path of the
compiler executable (e.g. we use ../lib, starting from the directory of
Sys.executable_name).
2. Ideas about how to improve OCaml under Windows
Globally, we find that the current state of OCaml itself is not too bad
under Windows. Some tiny touches would make the life easier to people
hacking on the OCaml toolchain itself under Windows, or just building it
from sources: for instance, adapting ./configure (or providing another
script) instead of telling users to copy three files and edit one. Some
internal cleanup would also be a good idea, e.g. merging various
Makefile and Makefile.nt, or at least sharing common parts, to avoid
duplication and divergence, although the Continuous Integration system
which is now in place catches many Windows build failures early, so this
is less critical these days. There are also deeper projects to address
embarrassing parts, such as supporting Unicode when interacting with the
filesystem ( https://github.com/ocaml/ocaml/pull/153 ); and the
Unix.select emulation still has problems (see the "OCaml windows"
category on OCaml's Mantis).
The story seems to be less nice with external tools and libraries,
although LexiFi is not overly affected by that. This is mostly a
consequence of the composition of the OCaml community, which is largely
Unix oriented. Windows is considered as a second-class citizen by most
OCaml core developers and package maintainers.
Excellent support of OCaml (and its ecosystem) under Windows can only be
achieved if we manage to attract people who are primarily Windows
developers, not Unix developers who are somehow forced to support
Windows, or -- even worse -- who are mainly interested in porting
existing projects from Unix to Windows. Those people would typically be
reluctant to use Cygwin or emacs/vim/etc. It's not clear to me how to
attract such people, though, since F# is a strong competitor for OCaml
under Windows. At least, one would need to think about how to
articulate the benefits of OCaml, to remove technical obstacles and
facilitate first steps with OCaml under Windows. My intuition is that
our goal should be to attract (enlightened) hobbyists developers under
Windows for now (as opposed to, say, trying to make OCaml directly the
natural choice for pure Windows companies), in order to bootstrap the
process.
Good support for existing OCaml libraries (at least those which are
relevant under Windows) is certainly an important goal. Even if we
never reach a state where OCaml is considered as a natural choice for a
pure Windows developer or company, this will already give more
visibility to OCaml, reinforce its already excellent cross-platform
reputation, and help people who are forced to use Windows (or just port
from Unix to Windows).
An important step to improve support of existing libraries under Windows
is to have good Windows support for OPAM. This will make it easier for
people to test libraries under Windows and so give incentive to upstream
developers to fix them (and themselves set up a working OCaml Windows
dev environment more easily). Something like the OPAM Weather Service
could be a great help here. From what I understand, OPAM is not very
far from supporting Windows, and I think that it's already possible to
compile OPAM with a Cygwin port of OCaml (and have it manage native
switches?).
I believe Cygwin will remain an important part of the story for OCaml
under Windows for some time. OCaml itself and many existing packages
require a Unix-like environment in their build system, and it doesn't
seem realistic trying to fix that soon. Since I don't believe in binary
distributions of OCaml packages in general (because updating a library
forces to update all other packages that depend on it), this means that
users would need a Cygwin environment. Except perhaps for
well-controlled dev environments (e.g. for LexiFi's built-in IDE, we
don't expect users to install their own extra libraries; something
similar could make sense for Ocsigen addins, etc). I don't know if we
should try to hide it as much as possible (e.g. by creating bundled
installers) or not. If/when OPAM supports installing native switches,
even if itself built with Cygwin, then it could make sense to contribute
a Cygwin package for OPAM. At least for people using the mingw port,
this would mean that setting up their environment could be as simple as
installing Cygwin with a well-documented list of packages (the mingw
compilers, flexdll, OPAM, and required extra tools such as make, etc),
and then using OPAM to compile/install OCaml and other packages from
sources. (People wanting to use the MSVC ports would need to install
MSVC themselves.)
Making sure OPAM and existing libraries work under Windows is only the
beginning. Hopefully, this could be enough to attract "native" Windows
developers, who could then contribute to a richer Windows experience,
such as:
- Binding a larger subset of the Win32 API.
- Tools and bindings to interact with Windows applications (e.g. to
support writing easily an OCaml addin for Excel).
- Some GUI frontend for OPAM and a decent graphical toplevel for OCaml.
- Better support for .Net (CSML is enough for LexiFi needs, but would
deserve some polishing).
- Support for popular IDEs.
Alain
On 05/29/2015 05:10 PM, Thomas Braibant wrote:
> Hi,
>
> Thanks for joining this working group. Its goal is to find out how to
> improve the state of OCaml and its ecosystem on Windows.
>
> Some of the tasks that we might address (this list is by no mean meant
> to be exhaustive nor normative):
>
> - Gather information from the community about the use of the various
> versions of OCaml available on Windows, and understand what kind of
> environment people are working with. (BTW, I think a good way to get the
> discussion rolling might be to describe each other's setup to use OCaml
> on Windows.)
>
> - Finalize OPAM support for Windows (this is listed on the OPAM 1.3
> roadmap). When this support is in place, many OCaml packages should work
> out of the box (with the proviso that many packages assume a unix-ish
> environment for their build system).
>
> - Discuss how to help package developers improve Windows support. One
> way to do that could be to provide know-how about how to setup
> continuous integration on services like Appveyor to check that packages
> build on typical Windows setups.
>
> - Find a solution for users of WODI, the GODI based OCaml distribution
> for Windows, which has been shutdown. WODI was able to compile OCaml
> related software (using Cygwin as an environment), but was also able to
> install binary packages. (In some cases, some non-trivial patches were
> applied to make package usable on Windows.) Some of the sources needed
> to use WODI are still available, but the system does not work out of the
> box now that the server has gone offline. With a little work, we might
> be able to bring it back, to smooth the transition.
>
> - Discuss the state of cross compilation to Windows.
>
> - Advertise OCamlPro's OCPWin commercial effort. (I will leave Fabrice
> tell a bit more about that.)
>
> - Discuss how to pool resources (if needed) to work toward these goals.
> Many thanks to Anil and OcamlLabs for hosting this mailing list.
>
> Thomas
>
>
>
>
>
> _______________________________________________
> wg-windows mailing list
> wg-windows at lists.ocaml.org
> http://lists.ocaml.org/listinfo/wg-windows
>
More information about the wg-windows
mailing list