"Out of memory" when using data constructors in FFI

[lzheng5]

Hello,

I am trying to experiment with FFI for my project after reading through the documentation. But I am still not quite sure what's missing in my trivial case here.

In test.c

#include "scheme.h"
#include <stdlib.h>

Compile test.so

gcc -dynamiclib -o test.so test.c kernel.o -liconv -lncurses

In test.ss

(load-shared-object "./test.so")
(define mycons
  (foreign-procedure "Scons"
    (ptr ptr) ptr))

(define myalloc
  (foreign-procedure "malloc"
    (unsigned-int) void*))

(myalloc 10) ;; => 140216889262768
(mycons 1 2) ;; => out of memory invalid memory reference.  Some debugging context lost
(myalloc 10) ;; => 140216889262784 ;; 16 bytes allocated though

I am running with Chez Scheme 9.6.4 on MacOs 10.15.4.

FYI, Chez Scheme 9.5.8a gives 'Segmentation Fault' instead of the 'out of memory' message.

Is there anything missing in my setup? Any help would be greatly appreciated.

Thanks,
Ling

[mflatt]

It looks like you're linking a second copy of Chez Scheme as kernel.o in test.so, and then you're calling Scons on that copy without it being initialized.

Although you should be able to initialize the second copy...

(define myinit
  (foreign-procedure "Sscheme_init"
    (uptr) void))
(define mybuild
  (foreign-procedure "Sbuild_heap"
                     (string uptr) void))

(myinit 0)
(mybuild "scheme" 0)

... I think you probably wanted to access the Scons of the copy that's already running. For that, you can use (load-shared-object #f):

(load-shared-object #f)
(define mycons
  (foreign-procedure "Scons"
    (ptr ptr) ptr))
(mycons 1 2)

[lzheng5]

"It looks like you're linking a second copy of Chez Scheme as kernel.o in test.so, and then you're calling Scons on that copy without it being initialized."

Oh! Two copies of the system ... Thanks for pointing that out.

What I thought I could do here is to call into a foreign C function just like calling a regular Scheme function, which in turn calls Scons to construct a return value for the Scheme side. So I definitely needed the original Scons.

ptr myfunc(ptr inputs)
{
    // do some processing for inputs, which is a list
    // construct and return a new list
}

But then I have to link kernel.o when compiling test.o, resulting a separate copy in test.o and it will give me the previous unintended behavior.

So kernel.o should only be linked when we want to use Chez Scheme as a subsystem?

Possible workarounds.

1. For my case, since the input and output are pretty simple data types, can I pass in a pair, where the car of the pair is the input and the cdr is the output, which will be set via Sset_cdr? Note allocation on the Scheme side can be avoided but I will still link a separate copy of the system in test.o.

2. Or I can define a ftype for the output list C type, like in the example.

(define-ftype
  [OutputList (struct
                [head int]
                [tail (* OutputList)])])

Now, I can avoid linking kernel.o but I will have to free up the memory allocated once I am done with the list.

Which one will be more appropriate I wonder? Or is there an even better workaround?

Thanks,
Ling

[mflatt]

What I thought I could do here is to call into a foreign C function just like calling a regular Scheme function, which in turn calls Scons to construct a return value for the Scheme side. So I definitely needed the original Scons.

Yes, that can work.

But then I have to link kernel.o when compiling test.o

No, don't do that. On macOS, link with -dynamiclib -undefined dynamic_lookup.

[lzheng5]

Thanks! That's exactly what I wanted.