[ocaml-ctypes] Help with probable GC problem
yallop at gmail.com
Tue Dec 19 23:55:15 GMT 2017
On 19 December 2017 at 23:00, Andre Nathan <andre at digirati.com.br> wrote:
> It seems like a memory value is being overwritten, or maybe claimed by the
> OCaml GC, although in the library I explicitly copy the username string, and
> libmariadb calls strdup() on it so I'm not sure that's the problem.
> Since the library is very large, I created a branch with the minimal
> bindings to open and close a connection, so that inspection of the code
> becomes feasible:
Thanks for the clear explanation and reproduction case. I think that
your diagnosis is correct
It appears that the problem arises from the nonblocking nature of the
code. When a program creates a block of memory using Ctypes.allocate,
let user = allocate ...
ctypes ensures the memory is not freed so long as 'user' is reachable.
Additionally, if 'user' is passed to a C function then ctypes ensures
that the memory is not freed for the duration of the function call.
However, in the MariaDB bindings 'user' appears to be accessed by
MariaDB after the C function it's passed to returns. More concretely,
the pointer is passed into C like this:
let user = allocate ... in
B.mysql_real_connect_start ret mysql host user pass db port socket flags
and then mysql_real_connect_start keeps a reference to 'user' and
returns immediately. Since OCaml/ctypes can no longer see any uses of
'user', the memory is assumed to be no longer needed and so it's
reclaimed by the GC. However, MariaDB still has a reference (that it
later accesses asynchronously) which is unsafe to use after the GC
reclaims the memory.
If the above is correct then there are at least two solutions. You
could ensure somehow that 'user' and other such pointers are kept
around on the OCaml side until the associated memory is no longer
needed by MariaDB. Or you could switch from 'allocate' to a
malloc/free style arrangement to avoid the automatic collection
There's a bit of guesswork involved in the above, so please follow up
if it doesn't fix the problem for you!
PS You may find ocaml-memcpy (https://github.com/yallop/ocaml-memcpy)
a useful alternative to char_ptr_buffer_of_string for copying strings
between OCaml and C memory.
More information about the Ctypes