[ocaml-ctypes] Best practices for wrapping ocaml-ctypes?

[Andi McClure]

Thanks again— I have an update and then another question or three.

With help from people on IRC, I have a working version of this now. You can
find it here:
https://bitbucket.org/runhello/test-emily-game/src/e1cac5bf0546234ab7b69977b53625efaef7d8d7/src/internalPackage.ml?at=default
And a demo of the code running:
https://pbs.twimg.com/media/CMRG8wZUcAAd7B2.png:large

What I have is a module FfiSupport. It is based around functions
cToValueFor and a valueToCFor, which given a type name can spit out a
converter from any specified C type to my "value" variant, or a converter
from my "value" variant to any specified C type. The 'ctype->value (or vice
versa) "converter" function comes paired with a ctypes typ, so I can pass
that directly into 'foreign'. The key part is:

      let ValueToCFn (argType, argConvert) = valueToCFor argTypeName in
      let CToValueFn (retType, retConvert) = cToValueFor retTypeName in
      let ffi_binding = foreign name (argType @-> returning retType) in
      fun arg -> retConvert (ffi_binding (argConvert arg))

So that is very satisfying, but there is an obvious problem, which is that
this only works for unary functions. I need it to work for functions of any
arity.

I have an intuition about how to do this, but a couple of days of tinkering
have gotten me nowhere and I am very concerned about whether, if I
completed my implementation, OCaml could type it. I'd like to describe this
possibly-unworkable solution, and then ask a question.

My thought is this: If I have for example a 2-ary function, I feel like I
need to construct two things, an ocaml-ctypes type spec ('a typ @-> 'b typ
@-> returning 'c typ), and a function ('a -> 'b ->
'c)->(value->value->value).

So maybe the way I achieve this is I construct a base function ('a ->
'b)->(value->value), and then I construct a "successor" function which
transforms
('b typ @-> ... @-> returning 'z typ) and ('b -> ... ->
'z)->(value->...->value)
into
('a typ @-> 'b typ @-> ... @-> returning 'z typ) and ('a->'b -> ... ->
'z)->(value->value->...->value).
I hope, if I could implement this "successor" function, it would all stay
within what the existential types feature can type sensibly, and this will
compile :) but, I am feeling stymied, and in my tries so far I haven't
really even succeeded in formulating the ('a->'b)->(value->value) "base
case". (This is as far as I got
https://bitbucket.org/runhello/test-emily-game/src/86e8bf4598e64ef0e1288f5cc6b86f70287d378b/src/internalPackage.ml?at=default
and
line 56 is... wrong.) The problem seems to be that the OCaml type system is
not set up to describe curried functions of indeterminate arity; I have no
idea what the type signature for my "successor" function would even look
like.

So, my question is: How would you recommend attacking the n-arity case? Do
you think this will even be possible?

OCaml-ctypes must have a solution to this kind of problem because it itself
works with functions of arbitrary arity. Could I unify my implementation
approach better with OCaml-ctypes's?

I notice that OCaml-ctypes on the inside seems to be working with a thing
like:
type _ fn =
  | Returns  : 'a typ   -> 'a fn
  | Function : 'a typ * 'b fn  -> ('a -> 'b) fn
This looks much superior to what I am doing, which is pure noodly function
manipulation. It seems like I would benefit a lot from using something like
that, with my retConvert and argConvert list embedded in the structure. But
it is unclear to me how to get from what I have to there (GADTs are very
new to me) and, I'm not sure if THAT will ultimately be permitted by the
OCaml type checker either!

Any advice welcome, in the meantime I am going to try just starting by
manually writing out 2-ary, 3-ary, 4-ary implementations to unblock this
particular project in the short term, and probably reading the Types and
Programming Languages chapter on existential types to see if it leaves me
less confused. >_>

One last thing. You mentioned in a previous email the idea of a
"Ctypes_untyped"
module. I think if I got this working, it would *probably* be something
that could be generalized by making a first-class module paramerizing the
value type and the "constructor factory" functions. It seems like a is a
useful thing. If this eventually worked, would it be eventually worth
submitting that parameterized Ctypes_variant back to ocaml-ctypes as a pull
request?

Thanks again!

- Andi

On Tue, Aug 11, 2015 at 2:41 AM, Jeremy Yallop <yallop at gmail.com> wrote:

> On 10/08/2015, Andi McClure <andi.m.mcclure at gmail.com> wrote:
> > The syntax could use some work :P But never mind that. I've so far got it
> > where when you invoke ffi.c.function, the function specification you
> > provided gets piped into OCaml and turned into this OCaml data structure:
> >
> >     type foreignSpec = { name : string; args : string list; returning:
> > string; }
> >
> > This seems like a straightforward representation of the information @->
> > encodes; it seems something parsing a header would need to represent the
> > data the same way.
> >
> > My naive impulse is to imagine that this is straightforward, and I need
> to
> > just build up a typ variable by repeatedly chaining @->, something like:
> >
> >     let typeConvert = function "void" -> void | "int" -> int | _ ->
> > failwith "??"
>
> Right: it should be possible to do things like that once you've dealt
> with the typing problem you describe.
>
> The easiest approach to dealing with the fact that 'int', 'void', etc.
> have different types is to use an existential type.  Existential types
> in OCaml are a sort of variant, but defined with a different syntax;
> instead of writing something like this:
>
>    type t = A of x
>           | B of y
>
> you give the type signature for each constructor, like this:
>
>    type t = A : x -> t
>           | B : y -> t
>
> By itself that doesn't give you any extra power, and you can actually
> write any variant type definition this way.  The distinctive feature
> of existential types is that you can mention type variables on the
> right of the '=' that don't appear in the type parameters on the left.
> Here's an example:
>
>    type ty = Ty : 'a Ctypes.typ -> ty
>
> As the type signature says, 'Ty' takes a value of type 'a Ctypes.typ'
> for any type 'a' and builds a value of type 'ty'.  You can therefore
> use 'Ty' to wrap Ctypes.typ values of different types up as 'ty'
> values.  For example, you can build a list of typs:
>
>    # [Ty void; Ty int];;
>    - : ty list = [Ty void; Ty int]
>
> and you can wrap the variously-typed results of your 'typeConvert'
> function so that they all have the same type 'ty':
>
>    let typeConvert = function
>        "void" -> Ty void
>      | "int" -> Ty int
>      | _ -> failwith "??"
>
> The same difficulty arises with 'Ctypes.fn' and your 'functionFrom'
> definition, which needs to build function type descriptions of all
> sorts of different types.  Here's a second existential type which
> wraps 'Ctypes.fn':
>
>    type fn = Fn : ('a -> 'b) Ctypes.fn -> fn
>
> This time the type signature says that 'Fn' takes a value of type '(a
> -> b) Ctypes.fn' for any types 'a' and 'b' and builds a value of type
> 'fn'.  It should be possible to build something like 'functionFrom'
> that wraps and unwraps intermediate values using 'Ty' and 'Fn' to deal
> with the immediate typing problems (leaving the interesting question
> of how to turn the function returned by 'Foreign.foreign' into
> something that can be called from your interpreter).
>
> I hope that helps a bit.
>
> Kind regards,
>
> Jeremy.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ocaml.org/pipermail/ctypes/attachments/20150814/f8e1a4de/attachment.html>