[ocaml-ctypes] Best practices for wrapping ocaml-ctypes?

[Jeremy Yallop]

Hi Andi,

Please excuse the slow response.

On 27 June 2015 at 14:25, Andi McClure <andi.m.mcclure at gmail.com> wrote:
> Hello, I have a project which is a programming language (
> http://emilylang.org/ ) and the current interpreter is implemented in ocaml.
> I want to add a C FFI, and I am looking at ocaml-ctypes, but I am having
> trouble figuring out how best to use the ocaml-ctypes library given my
> project's specific needs as a language.

Well, using ocaml-ctypes to build an FFI for another language is a bit
different from the usual use case of exposing a particular C library
to OCaml, so it'll be interesting to see how things turn out.

> Basically: ocaml-ctypes seems to take libFFI and recast its operations in
> terms of ocaml idioms and primitives. This looks great if I am writing an
> ocaml program. However, my goal is to present an interface to libffi (or
> something like it) in terms of *Emily* idioms and primitives (Emily being my
> language) so that I can write an Emily program. This means directly wrapping
> ocaml-ctypes probably won't work out well (the ocaml idioms ocaml-ctypes
> uses can't all be directly represented in my language, and they might come
> across as weird to the language users who are probably not ocaml users).
>
> My initial thought was that what I probably wanted to do was try to present
> an interface between the interpreter and interpreted code that closely
> tracks the programming interface of libffi. Then I could write an
> in-language library on top of this with a friendlier/more idiomatic
> interface. (The reason I believe I want the interpreter-interpreted
> interface to resemble libffi is that I might someday switch from ocaml to
> another language, and libffi will probably be available in other contexts
> but ocaml-ctypes will not). Looking over the ctypes code it looked like a
> "raw" libffi-flavored interface might be possible using
> ctypes-foreign-base/ctypes_ffi.mli and the function_of_pointer interface,
> but it looks like this mli is present in the code but not exposed in the
> opam package.

Right: that module is intended to be internal-only.  However, some of
the functionality is expressible via the public interface.  For
example, here are alternative implementations of function_of_pointer
and pointer_of_function using only functions that are currently
publicly exposed:

  let function_of_pointer ?name ~abi ~check_errno ~release_runtime_lock fn p =
    Ctypes.coerce (ptr void)
      (Foreign.funptr ?name ~abi ~check_errno
~runtime_lock:release_runtime_lock fn)
      p

  let pointer_of_function ~abi ~acquire_runtime_lock fn f =
    Ctypes.coerce (Foreign.funptr ~abi ~runtime_lock:acquire_runtime_lock fn)
      (ptr void)
      f

> What would you recommend in this case?

I can think of three approaches that are not obviously wrong, and it's
likely that there are others.

First, you might do what you've been considering already -- i.e. using
the low-level parts of ctypes as a basis for a more Emily-flavoured
libffi binding.  The two essential modules are probably
Ctypes_memory_stubs (src/ctypes/ctypes_memory_stubs.ml), which
provides an untyped API for accessing C-managed memory, and
Ctypes_ffi_stubs (src/ctypes-foreign-base/ctypes_ffi_stubs.ml), which
provides a fairly direct OCaml binding onto libffi.  Neither of these
modules is exposed directly, but it should be reasonably
straightforward to make them available by tweaking the Makefile to add
them to the appropriate '.public' list.

Second, you might find some way to massage the ctypes API into a form
that's more suitable for use in Emily.  For example, I can imagine the
type parameters causing problems if you want to use the type
representations (for int, float, etc.) in a more uniform way.  It
probably wouldn't be too much work to build an 'untyped' interface on
top of ctypes along the following lines

  module Ctypes_untyped :
  sig
    type typ
    val int : typ
    val float : typ
    val (@->) :  typ -> typ -> typ
    (* ... *)

This is just a guess, of course, and perhaps there are other OCaml
idioms that aren't so easy to factor out.

Finally, you might use ctypes to build a binding to libffi -- i.e.
just treat libffi like any other C library and describe its interface
using ctypes, ignoring the fact that ctypes uses libffi internally.
This approach isn't as absurd as it might sound at first, since recent
versions of ctypes make it possible to bind to C libraries by
generating C and OCaml code rather than by routing calls through
libffi, so you wouldn't necessarily end up using libffi twice.

Feel free to ask if you'd like more detail on any of the above!

Kind regards,

Jeremy.