[ocaml-ctypes] Help with strings

[Jeremy Yallop]

Dear John,

On 25/12/2017, john skaller <skaller at internode.on.net> wrote:
> So roughly .. when is it safe to use “string”?
> At the moment the answer for me is “never” because I don’t understand the
> memory management protocol.

It works like this:

  * passing a string using 'string' makes a copy in either direction.
Furthermore,

    - the copy created when passing a string from OCaml to C lives for
the lifetime of the C call.
      (It's possible this will be strengthened in the future:
https://github.com/ocamllabs/ocaml-ctypes/issues/556)

    - the copy created when passing a string from C to OCaml is a
regular OCaml string, subject to usual GC behaviour.  Ctypes makes no
attempt to deallocate the memory used by the original C string.

So, for your three examples

>         char const *get_version();
>         char *get_buffer();
>         char *strdup(char *);
>
>
> The first function returns a pointer to an immutable global object.

It's fine, and probably the best choice, to use 'string' for
'get_version'.  Each call to the function will create a fresh OCaml
string.

> The second to a mutable part of a buffer.

Mutability means that it's probably not useful to use `string` here,
since 'string' will create a copy that won't track changes.  It would
be better to write

   let get_buffer = foreign "get_buffer"
      (void @-> returning (ptr char))

> The third to a freshly malloc()ed copy of a NTBS.

Here 'string' is not the right thing for the *return type*, since (as
you say) it prevents access to the pointer needed to free the returned
memory.  Again, 'ptr char' is probably the best starting point.  On
the other hand, 'string' is a reasonable choice for the *argument*,
since Ctypes will automatically deallocate the copy of the string
after the call.  So the following binding is reasonable:

    let strdup = foreign "strdup"
      (string @-> returning (ptr char))

In this case it's possible to do a little better.  The 'ocaml_string'
type description is an alternative to 'string' that avoids the copy:

    let strdup = foreign "strdup"
      (ocaml_string @-> returning (ptr char))

In the general case, where the C function can call back into OCaml,
'ocaml_string' is not safe.  But it's safe in the common (first-order)
case, which includes 'strdup'.

> So roughly, if you wanted to do it right, Ctypes “type” system is inadequate.
> The “types” must contain mutablility and ownership information.
> For example
>
>         immutable_lend_noincrementable_nonnull_pointer_char

You're quite right, both that Ctypes doesn't capture these properties
in the type system, and that it's quite challenging to do so, since C
is so flexible.  (For example, the 'realpath' function can return
either a malloc-allocated buffer or a caller-supplied buffer,
depending on the value of its second argument!)

Regarding mutability, the 4.06.0 release of OCaml distinguishes
mutable from immutable strings.  When Ctypes drops support for older
OCaml versions it's likely that string mutability will be more clearly
marked in Ctypes bindings, too.

> Of course I can do that by passing a char ptr manually created
> (C types can do that I think),

Indeed (see above).

> copying Ocaml string contents manually (how?),

A quick way is to use the 'coerce' function, which can convert between
'ptr char' and 'string' because 'string' is a view for 'ptr char'.
Here's an example using the 'strdup' binding above.

    # let p = strdup "hello, world";;
    val p : char Ctypes_static.ptr = (char*) 0x5601c2a90130
    # coerce (ptr char) string p;;
    - : string = "hello, world"

More generally the 'ocaml-memcpy' package provides functions that can
copy between several different representations
(https://github.com/yallop/ocaml-memcpy).

> and then manually free-ing it (C types can do that?)

A simple way is to bind 'free'!  Here's an example, freeing the memory
returned from the call to 'strdup' above.

    # let free = foreign "free" (ptr char @-> returning void);;
    val free : char Ctypes_static.ptr -> unit = <fun>
    # free p;;
    - : unit = ()

Kind regards,

Jeremy