[ocaml-ctypes] Best practices for wrapping ocaml-ctypes?

[Jeremy Yallop]

On 14/08/2015, Andi McClure <andi.m.mcclure at gmail.com> wrote:
> With help from people on IRC, I have a working version of this now. You can
> find it here:
> https://bitbucket.org/runhello/test-emily-game/src/e1cac5bf0546234ab7b69977b53625efaef7d8d7/src/internalPackage.ml?at=default
> And a demo of the code running:
> https://pbs.twimg.com/media/CMRG8wZUcAAd7B2.png:large

Thanks for the update!  It's great to see the foreign call working end-to-end.

> What I have is a module FfiSupport. It is based around functions
> cToValueFor and a valueToCFor, which given a type name can spit out a
> converter from any specified C type to my "value" variant, or a converter
> from my "value" variant to any specified C type. The 'ctype->value (or vice
> versa) "converter" function comes paired with a ctypes typ, so I can pass
> that directly into 'foreign'. The key part is:
>
>       let ValueToCFn (argType, argConvert) = valueToCFor argTypeName in
>       let CToValueFn (retType, retConvert) = cToValueFor retTypeName in
>       let ffi_binding = foreign name (argType @-> returning retType) in
>       fun arg -> retConvert (ffi_binding (argConvert arg))

This looks like a sensible scheme.

> So that is very satisfying, but there is an obvious problem, which is that
> this only works for unary functions. I need it to work for functions of any
> arity.

I think that should be possible.

> I have an intuition about how to do this, but a couple of days of tinkering
> have gotten me nowhere and I am very concerned about whether, if I
> completed my implementation, OCaml could type it. I'd like to describe this
> possibly-unworkable solution, and then ask a question.
>
> My thought is this: If I have for example a 2-ary function, I feel like I
> need to construct two things, an ocaml-ctypes type spec ('a typ @-> 'b typ
> @-> returning 'c typ), and a function ('a -> 'b ->
> 'c)->(value->value->value).

Yes, except that things will become easier if you give the second thing the type

   ('a -> 'b -> 'c) -> value

by applying the BuiltinFunctionValue constructor at appropritate
points, since then every bound function will have the same type,
regardless of arity.

> So maybe the way I achieve this is I construct a base function ('a ->
> 'b)->(value->value), and then I construct a "successor" function which
> transforms
> ('b typ @-> ... @-> returning 'z typ) and ('b -> ... ->
> 'z)->(value->...->value)
> into
> ('a typ @-> 'b typ @-> ... @-> returning 'z typ) and ('a->'b -> ... ->
> 'z)->(value->value->...->value).

Right.  So, more concretely, if you're building a binding to a C
function of type

   void(float, int)

you want to build a Ctypes type representation

   float @-> int @-> returning void

and a conversion function of type

   (float -> int -> unit) -> value

for turning the function returned from Foreign.foreign into an Emily
value, like this:

   let ffi_binding = foreign name (argType @-> returning retType) in
   convert ffi_binding

So the "successor" function should turn the type representation from this

   int @-> returning void

into this

   float @-> int @-> returning void

and it should turn the OCaml-to-Emily conversion function from
something of this type

  (int -> unit) -> value

into something of this type

  (float -> int -> unit) -> value

Converting the type representation is the easy part, since you can
just use '@->' to add the new argument type:

  let unary = int @-> returning void in
    float @-> unary

Converting the OCaml-to-Emily function involves writing something of type

  ((int -> unit) -> value) -> (float -> int -> unit) -> value

i.e. something that looks like this:

  fun
   (f : ((int -> unit) -> value))
   (g : (float -> int -> unit)) ->
      (... : value)

where the '...' is somehow built from 'f' and 'g' and has type
'value'.  Of course, filling in the '...' is the tricky part, but the
types help quite a bit.  First, since you're extending the conversion
function to take an extra argument you know that the result should be
constructed using BuiltinFunctionValue:

  fun
   (f : ((int -> unit) -> value))
   (g : (float -> int -> unit)) ->
      (BuiltinFunctionValue
        (fun (x : value) -> (... : value)))

Next, you know that x should hold a float, so you can unpack it:

  let float_of_value = function
    Value.FloatValue y -> y
  | _ -> failwith "type error"

  fun
   (f : ((int -> unit) -> value))
   (g : (float -> int -> unit)) ->
      (BuiltinFunctionValue
         (fun (x : value) ->
            let y = float_of_value x in
               (... : value)))

What can you do with 'y'?  There's only one function around that takes
a float, namely 'g', so you can pass 'y' to 'g'. (Thinking through
this decision a little more carefully reveals that it is actually the
right thing to do, since 'g' is the function returned from
'Foreign.foreign', and 'y' is the argument passed in to that function
from an Emily program.)

  fun
   (f : ((int -> unit) -> value))
   (g : (float -> int -> unit)) ->
      (BuiltinFunctionValue
         (fun (x : value) ->
            let y = float_of_value x in
            let h : int -> unit = g y in
               (... : value)))

Finally, we have a value, 'h', of type 'int -> unit', and there's
exactly one obvious thing to do with it (which once again also turns
out to be the correct thing): pass it to 'f' to build a 'value':

  fun
   (f : ((int -> unit) -> value))
   (g : (float -> int -> unit)) ->
      (BuiltinFunctionValue
         (fun (x : value) ->
            let y = float_of_value x in
            let h : int -> unit = g y in
              (f h : value)))

This is sufficient to support this particular example ('float -> int
-> unit'), and it's not difficult to generalise it to handle extending
functions of any arity and type by removing the type ascriptions and
turning the 'float_of_value' function into a parameter:

  let extend_fn =
     fun of_value f g ->
      (BuiltinFunctionValue
         (fun x ->
            let y = of_value x in
            let h = g y in
              f h))

which can be written more simply as

   let extend_fn of_value f g = BuiltinFunctionValue (fun x -> f (g
(of_value x)))

Now 'extend_fn' can be used to build conversions between functions of
arbitrary arity.  For example:

   # extend_fn float_of_value value_of_string;;
   - : (float -> string) -> value = <fun>
   # extend_fn string_of_value (extend_fn float_of_value value_of_string);;
   - : (string -> float -> string) -> value = <fun>
   # extend_fn string_of_value (extend_fn string_of_value (extend_fn
float_of_value value_of_string));;
   - : (string -> string -> float -> string) -> value = <fun>

Kind regards,

Jeremy.