<div dir="ltr">Hello—<div><br></div><div>Thanks Jeremy for this response. After some time I've been able to come back and try these things out. I do have some new questions :) I might write this out a little more verbosely than I need to just to be certain I'm speaking precisely.</div><div><br></div><div>I have a test program with a c function</div><div><br></div><div><div> int increment3(int x) { return x + 3; }</div></div><div><br></div><div>On the OCaml side, this is accessed as</div><div><br></div><div> let increment3 = foreign "increment3" (int @-> returning int) in<br></div><div> increment3 3</div><div><br></div><div>And that works and I'm happy with it. Actually, after playing with the foreign / "@->" interface, I'm finding it seems to be about as good as I need, and I'm trying to figure out if I can use that instead of having to work with the low-level interfaces (which as discussed before are not necessarily directly exposed at this time anyway).</div><div><br></div><div>Anyway, I'm trying to set up my OCaml-implemented interpreter, so that this line of code in Emily is equivalent to the "let increment3" above:</div><div><br></div><div> (* Just pretend .int is shorthand for "int" *)</div><div> increment3 = package.emily.ffi.c.function [ name = .increment3; args = [ .int, ]; return = .int ]<br></div><div><br></div><div>The syntax could use some work :P But never mind that. I've so far got it where when you invoke ffi.c.function, the function specification you provided gets piped into OCaml and turned into this OCaml data structure:</div><div><br></div><div><div> type foreignSpec = { name : string; args : string list; returning: string; }</div></div><div><br></div><div>This seems like a straightforward representation of the information @-> encodes; it seems something parsing a header would need to represent the data the same way.</div><div><br></div><div>My naive impulse is to imagine that this is straightforward, and I need to just build up a typ variable by repeatedly chaining @->, something like:</div><div><br></div><div> let typeConvert = function "void" -> void | "int" -> int | _ -> failwith "??"<br></div><div> let functionFrom spec =</div><div> foreign <a href="http://spec.name">spec.name</a> ( List.fold_right</div><div> ( @-> )</div><div> ( List.map typeConvert wrap.args )</div><div> ( returning ( typeConvert wrap.returning ) ) )</div><div><br></div><div>If I start trying to implement this, OCaml immediately objects that even just typeConvert is impossible to compile:</div><div><br></div><div><div> Error: This expression has type int Ctypes.typ = int Ctypes_static.typ</div><div> but an expression was expected of type</div><div> unit Ctypes.typ = unit Ctypes_static.typ</div><div> Type int is not compatible with type unit </div></div><div><br></div><div>This seems to make immediate sense: I don't fully understand what the type of the thing @-> returns is (Merlin says it's '_a -> '_b -> '_c ...) but it seems like whatever @-> may be doing, in all cases @-> or foreign actually gets used, the type of that @->'s return must be something known at the call site at compile time-- because when OCaml code invokes "foreign" OCaml will need to statically compile a statically typed result. So I can't possibly store one of the typs in a variable which could contain sometimes "int" and sometimes "void". I'm assuming this works something like C++ template arguments, which are something I have a little more familiarity with than OCaml type variables.</div><div><br></div><div>It seems like, if I really understood what @-> is doing, I might be able to maybe make this work anyway. In Emily I currently have only one "type", a big variant which is modeled in OCaml like:</div><div><br></div><div><div>type value = Null | True | FloatValue of float | StringValue of string | AtomValue of string</div><div> | BuiltinFunctionValue of (value -> value)</div></div><div><br></div><div>I eventually intend for the ffi.c.function implementation to return a BuiltinFunctionValue that takes an argument; either unpacks it from a "value" into the correct tagged type or throws an error; then packs the return back into an appropriately tagged "value". The "value" pack and unpack functions will know their types statically, so maybe in principle they can work in participation with whatever type-chaining trick @-> does and still produce well-typed code. But it's very unclear to me how to get started with that, or whether it will ultimately work.</div><div><br></div><div>Do you have any thoughts on this? Is this approach (engaging foreign / @-> directly) one which can eventually work, or should I back off and attempt one of the lower-level approaches mentioned in your previous email? Your previous email proposed building out a <span style="font-family:arial,sans-serif;font-size:12.8000001907349px">Ctypes_untyped with a slightly different interface, but it was not clear to me what implementation of that would look like.</span></div><div><span style="font-family:arial,sans-serif;font-size:12.8000001907349px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:12.8000001907349px">Thanks!</span></div><div><span style="font-family:arial,sans-serif;font-size:12.8000001907349px"><br></span></div><div><span style="font-family:arial,sans-serif;font-size:12.8000001907349px"> - Andi</span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jun 29, 2015 at 4:16 PM, Jeremy Yallop <span dir="ltr"><<a href="mailto:yallop@gmail.com" target="_blank">yallop@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Andi,<br>
<br>
Please excuse the slow response.<br>
<span class=""><br>
On 27 June 2015 at 14:25, Andi McClure <<a href="mailto:andi.m.mcclure@gmail.com">andi.m.mcclure@gmail.com</a>> wrote:<br>
> Hello, I have a project which is a programming language (<br>
> <a href="http://emilylang.org/" rel="noreferrer" target="_blank">http://emilylang.org/</a> ) and the current interpreter is implemented in ocaml.<br>
> I want to add a C FFI, and I am looking at ocaml-ctypes, but I am having<br>
> trouble figuring out how best to use the ocaml-ctypes library given my<br>
> project's specific needs as a language.<br>
<br>
</span>Well, using ocaml-ctypes to build an FFI for another language is a bit<br>
different from the usual use case of exposing a particular C library<br>
to OCaml, so it'll be interesting to see how things turn out.<br>
<span class=""><br>
> Basically: ocaml-ctypes seems to take libFFI and recast its operations in<br>
> terms of ocaml idioms and primitives. This looks great if I am writing an<br>
> ocaml program. However, my goal is to present an interface to libffi (or<br>
> something like it) in terms of *Emily* idioms and primitives (Emily being my<br>
> language) so that I can write an Emily program. This means directly wrapping<br>
> ocaml-ctypes probably won't work out well (the ocaml idioms ocaml-ctypes<br>
> uses can't all be directly represented in my language, and they might come<br>
> across as weird to the language users who are probably not ocaml users).<br>
><br>
> My initial thought was that what I probably wanted to do was try to present<br>
> an interface between the interpreter and interpreted code that closely<br>
> tracks the programming interface of libffi. Then I could write an<br>
> in-language library on top of this with a friendlier/more idiomatic<br>
> interface. (The reason I believe I want the interpreter-interpreted<br>
> interface to resemble libffi is that I might someday switch from ocaml to<br>
> another language, and libffi will probably be available in other contexts<br>
> but ocaml-ctypes will not). Looking over the ctypes code it looked like a<br>
> "raw" libffi-flavored interface might be possible using<br>
> ctypes-foreign-base/ctypes_ffi.mli and the function_of_pointer interface,<br>
> but it looks like this mli is present in the code but not exposed in the<br>
> opam package.<br>
<br>
</span>Right: that module is intended to be internal-only. However, some of<br>
the functionality is expressible via the public interface. For<br>
example, here are alternative implementations of function_of_pointer<br>
and pointer_of_function using only functions that are currently<br>
publicly exposed:<br>
<br>
let function_of_pointer ?name ~abi ~check_errno ~release_runtime_lock fn p =<br>
Ctypes.coerce (ptr void)<br>
(Foreign.funptr ?name ~abi ~check_errno<br>
~runtime_lock:release_runtime_lock fn)<br>
p<br>
<br>
let pointer_of_function ~abi ~acquire_runtime_lock fn f =<br>
Ctypes.coerce (Foreign.funptr ~abi ~runtime_lock:acquire_runtime_lock fn)<br>
(ptr void)<br>
f<br>
<span class=""><br>
> What would you recommend in this case?<br>
<br>
</span>I can think of three approaches that are not obviously wrong, and it's<br>
likely that there are others.<br>
<br>
First, you might do what you've been considering already -- i.e. using<br>
the low-level parts of ctypes as a basis for a more Emily-flavoured<br>
libffi binding. The two essential modules are probably<br>
Ctypes_memory_stubs (src/ctypes/<a href="http://ctypes_memory_stubs.ml" rel="noreferrer" target="_blank">ctypes_memory_stubs.ml</a>), which<br>
provides an untyped API for accessing C-managed memory, and<br>
Ctypes_ffi_stubs (src/ctypes-foreign-base/<a href="http://ctypes_ffi_stubs.ml" rel="noreferrer" target="_blank">ctypes_ffi_stubs.ml</a>), which<br>
provides a fairly direct OCaml binding onto libffi. Neither of these<br>
modules is exposed directly, but it should be reasonably<br>
straightforward to make them available by tweaking the Makefile to add<br>
them to the appropriate '.public' list.<br>
<br>
Second, you might find some way to massage the ctypes API into a form<br>
that's more suitable for use in Emily. For example, I can imagine the<br>
type parameters causing problems if you want to use the type<br>
representations (for int, float, etc.) in a more uniform way. It<br>
probably wouldn't be too much work to build an 'untyped' interface on<br>
top of ctypes along the following lines<br>
<br>
module Ctypes_untyped :<br>
sig<br>
type typ<br>
val int : typ<br>
val float : typ<br>
val (@->) : typ -> typ -> typ<br>
(* ... *)<br>
<br>
This is just a guess, of course, and perhaps there are other OCaml<br>
idioms that aren't so easy to factor out.<br>
<br>
Finally, you might use ctypes to build a binding to libffi -- i.e.<br>
just treat libffi like any other C library and describe its interface<br>
using ctypes, ignoring the fact that ctypes uses libffi internally.<br>
This approach isn't as absurd as it might sound at first, since recent<br>
versions of ctypes make it possible to bind to C libraries by<br>
generating C and OCaml code rather than by routing calls through<br>
libffi, so you wouldn't necessarily end up using libffi twice.<br>
<br>
Feel free to ask if you'd like more detail on any of the above!<br>
<br>
Kind regards,<br>
<br>
Jeremy.<br>
</blockquote></div><br></div>