[ocaml-ctypes] Help with probable GC problem

[Andre Nathan]

Jeremy, it appears you are right, as usual.

I've replace my type definition:

   type 'm connection = B.mysql

with

   type 'm connection =
     { handle : B.mysql
     ; host   : char Ctypes.ptr option
     ; port   : int
     ; user   : char Ctypes.ptr option
     ; pass   : char Ctypes.ptr option
     ; db     : char Ctypes.ptr option
     ; socket : char Ctypes.ptr option
     }

and that made the error disappear (initially the record only had a field 
for the user, which caused and error on the password, then on the db 
name, etc. which I guess confirms your diagnosis). The port is there 
only for completeness.

Tomorrow I'll make the fix in the real library code.

Regarding ocaml-memcpy, would it have the same effect in this case if I 
replaced the "ptr_opt char" parameters in the binding specifications 
with just "string", thus letting ctypes make the copy? In that case, 
would I still need to keep a reference to the strings on the OCaml side?

Cheers, and thanks a lot,
Andre


On 12/19/2017 09:55 PM, Jeremy Yallop wrote:
> Dear Andre,
> 
> On 19 December 2017 at 23:00, Andre Nathan <andre at digirati.com.br> wrote:
>> It seems like a memory value is being overwritten, or maybe claimed by the
>> OCaml GC, although in the library I explicitly copy the username string, and
>> libmariadb calls strdup() on it so I'm not sure that's the problem.
>>
>> Since the library is very large, I created a branch with the minimal
>> bindings to open and close a connection, so that inspection of the code
>> becomes feasible:
>>
>>    https://github.com/andrenth/ocaml-mariadb/tree/minimal
> 
> Thanks for the clear explanation and reproduction case.  I think that
> your diagnosis is correct
> 
> It appears that the problem arises from the nonblocking nature of the
> code.  When a program creates a block of memory using Ctypes.allocate,
> like this
> 
>      let user = allocate ...
> 
> ctypes ensures the memory is not freed so long as 'user' is reachable.
> Additionally, if 'user' is passed to a C function then ctypes ensures
> that the memory is not freed for the duration of the function call.
> 
> However, in the MariaDB bindings 'user' appears to be accessed by
> MariaDB after the C function it's passed to returns.  More concretely,
> the pointer is passed into C like this:
> 
>      let user = allocate ... in
>      ...
>      B.mysql_real_connect_start ret mysql host user pass db port socket flags
> 
> and then mysql_real_connect_start keeps a reference to 'user' and
> returns immediately.  Since OCaml/ctypes can no longer see any uses of
> 'user', the memory is assumed to be no longer needed and so it's
> reclaimed by the GC.  However, MariaDB still has a reference (that it
> later accesses asynchronously) which is unsafe to use after the GC
> reclaims the memory.
> 
> If the above is correct then there are at least two solutions.  You
> could ensure somehow that 'user' and other such pointers are kept
> around on the OCaml side until the associated memory is no longer
> needed by MariaDB.  Or you could switch from 'allocate' to a
> malloc/free style arrangement to avoid the automatic collection
> altogether.
> 
> There's a bit of guesswork involved in the above, so please follow up
> if it doesn't fix the problem for you!
> 
> Kind regards,
> 
> Jeremy
> 
> PS You may find ocaml-memcpy (https://github.com/yallop/ocaml-memcpy)
> a useful alternative to char_ptr_buffer_of_string for copying strings
> between OCaml and C memory.
>