[ocaml-ctypes] Help with probable GC problem

[Jeremy Yallop]

Dear Andre,

On 19 December 2017 at 23:00, Andre Nathan <andre at digirati.com.br> wrote:
> It seems like a memory value is being overwritten, or maybe claimed by the
> OCaml GC, although in the library I explicitly copy the username string, and
> libmariadb calls strdup() on it so I'm not sure that's the problem.
>
> Since the library is very large, I created a branch with the minimal
> bindings to open and close a connection, so that inspection of the code
> becomes feasible:
>
>   https://github.com/andrenth/ocaml-mariadb/tree/minimal

Thanks for the clear explanation and reproduction case.  I think that
your diagnosis is correct

It appears that the problem arises from the nonblocking nature of the
code.  When a program creates a block of memory using Ctypes.allocate,
like this

    let user = allocate ...

ctypes ensures the memory is not freed so long as 'user' is reachable.
Additionally, if 'user' is passed to a C function then ctypes ensures
that the memory is not freed for the duration of the function call.

However, in the MariaDB bindings 'user' appears to be accessed by
MariaDB after the C function it's passed to returns.  More concretely,
the pointer is passed into C like this:

    let user = allocate ... in
    ...
    B.mysql_real_connect_start ret mysql host user pass db port socket flags

and then mysql_real_connect_start keeps a reference to 'user' and
returns immediately.  Since OCaml/ctypes can no longer see any uses of
'user', the memory is assumed to be no longer needed and so it's
reclaimed by the GC.  However, MariaDB still has a reference (that it
later accesses asynchronously) which is unsafe to use after the GC
reclaims the memory.

If the above is correct then there are at least two solutions.  You
could ensure somehow that 'user' and other such pointers are kept
around on the OCaml side until the associated memory is no longer
needed by MariaDB.  Or you could switch from 'allocate' to a
malloc/free style arrangement to avoid the automatic collection
altogether.

There's a bit of guesswork involved in the above, so please follow up
if it doesn't fix the problem for you!

Kind regards,

Jeremy

PS You may find ocaml-memcpy (https://github.com/yallop/ocaml-memcpy)
a useful alternative to char_ptr_buffer_of_string for copying strings
between OCaml and C memory.