[ffi/isolate] Support passing/receiving Pointers to/from Isolates.

Question

[ffi/isolate] Support passing/receiving Pointers to/from Isolates.

modulovalue opened this issue 2 years ago · 18 comments

Consider the following example:

import 'dart:async';
import 'dart:ffi';
import 'dart:isolate';
import 'package:ffi/ffi.dart';

void main() async {
  final pointer = malloc.allocate<Uint8>(1);
  print(
    await asyncCompute<Pointer, int>(
      fn: fn,
      input: pointer,
    ),
  );
}

int fn(
  final Pointer p,
) {
  return p.address;
}

Future<O> asyncCompute<I, O>({
  required final FutureOr<O> Function(I) fn,
  required final I input,
}) async {
  final receivePort = ReceivePort();
  final inbox = StreamIterator<dynamic>(
    receivePort,
  );
  await _IsolateMessage(
    sendPort: receivePort.sendPort,
    fn: fn,
    input: input,
  ).send();
  final movedNext = await inbox.moveNext();
  assert(
    movedNext,
    "Call to moveNext is expected to return true.",
  );
  final typedResult = inbox.current as O;
  receivePort.close();
  await inbox.cancel();
  return typedResult;
}

class _IsolateMessage<I, O> {
  final SendPort sendPort;
  final FutureOr<O> Function(I) fn;
  final I input;

  const _IsolateMessage({
    required final this.sendPort,
    required final this.fn,
    required final this.input,
  });

  Future<Isolate> send() {
    return Isolate.spawn<_IsolateMessage<I, O>>(
      (final message) {
        Isolate.exit(
          message.sendPort,
          message.fn(
            message.input,
          ),
        );
      },
      this,
    );
  }
}

Running it throws the following error:

mvs-iMac:state mv$ ~/Downloads/dart-sdk/bin/dart --version
Dart SDK version: 2.19.0-255.2.beta (beta) (Tue Oct 4 13:45:53 2022 +0200) on "macos_x64"
mvs-iMac:state mv$ ~/Downloads/dart-sdk/bin/dart pointer.dart 
Unhandled exception:
Invalid argument(s): Illegal argument in isolate message: (object is a Pointer)
#0      Isolate._spawnFunction (dart:isolate-patch/isolate_patch.dart:399:25)
#1      Isolate.spawn (dart:isolate-patch/isolate_patch.dart:379:7)
#2      _IsolateMessage.send (.../pointer.dart:58:20)
#3      asyncCompute (.../pointer.dart:34:5)
#4      main (.../pointer.dart:9:11)
#5      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#6      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

It looks like moving Pointers through isolates isn't supported.

One workaround is to read and pass the integer address itself, and to create a pointer manually from that address in the target isolate. However, this is highly inconvenient, as one has to architect his models around integer addresses and not Pointers if he wants to be able to send the models itself across isolates.

I intend to share native memory across isolates and it would be great for these use cases if this restriction could be lifted. cc: @mraleph

Answer 1 · 2022-11-14T07:56:23.000Z

I think the main reason we decided to disable this is to catch situations when users accidentally send a pointer to another isolate without realising that they need to do something special around lifetime. (e.g. if you send a pointer to some reference counted resource to another isolate you might need to increment reference count and install a finaliser on the receiving end, etc.).

/cc @dcharkes

Answer 2 · 2022-11-14T08:34:48.000Z

I think the main reason we decided to disable this is to catch situations when users accidentally send a pointer to another isolate without realising that they need to do something special around lifetime. (e.g. if you send a pointer to some reference counted resource to another isolate you might need to increment reference count and install a finaliser on the receiving end, etc.).

/cc @dcharkes

Correct, this is why we decided that.

I did actually make a prototype for supporting sending Finalizers in send-and-exit: https://dart-review.git.corp.google.com/c/sdk/+/235141

We could consider adding support for sending NativeFinalizers in send-and-exit and allowing sending Finalizables and Pointers over.

@modulovalue what is your native-memory/native-resource management strategy?
(I'd imagine that if your model includes native finalizers to clean up your model implements Finalizable. However, since you didn't mention it, you might not be using finalizers at all.)

Answer 3 · 2022-11-14T16:02:02.000Z

what is your native-memory/native-resource management strategy?

I plan to manage reference counters on my own.
Edit: my description below is confusing. Here is a diagram #50452 (comment).

i.e. An Isolate exiting would remove all its references at once, or an isolate could report atomic changes through for example the native finalizer provided by Dart_NewExternalUTF16String. (_Passing around Pointers to native memory intended to be used by external strings was my primary motivation for this issue. The second use case was to share thin wrappers over pointers to Wasmer objects between isolates. Some context around that can be found here dart-archive/wasm#97. _)

Answer 4 · 2022-12-14T13:15:39.000Z

Our Pointer objects are no longer reified. We won't allow attaching finalizers to them. We may just represent them as unboxed integers in Dart.

Given that, it does seem reasonable we allow sending Pointer objects across isolates (since they are immutable, they can also be shared). If the message that is being sent includes a Finalizable we disallow sending the message anyway.

Currently people can still achieve it by taking pointer.address before sending and then re-construct the pointer on the other side.

(There's one caveat: If a message got sent but receiver dies or doesn't handle them, any messages are dropped. This dropping will not cause freeing of any memory (we have support on C side to do that, but not in SendPort api). But this problem seems orthogonal.)

/cc @mraleph @dcharkes

@mkustermann in #50715 (comment)

edit: sgtm

Answer 5 · 2023-04-21T11:54:42.000Z

Is this issue considered wontfix? I'd like to pass pointers to an isolate without the ceremony of covertly serializing pointer addresses.

Answer 6 · 2023-04-21T12:03:27.000Z

Is this issue considered wontfix? I'd like to pass pointers to an isolate without the ceremony of covertly serializing pointer addresses.

Pointers are already no longer reified (they don't remember their type argument at runtime). We'd like to eventually represent them as simple integers when passing them around.

It's assumed that if the pointer hangs on to native memory which needs freeing / releasing / ... that an enclosing class refers to it and has a finalizer attached. Sending such objects with finalizers is disallowed. But we'll allow sending Pointer objects.

Someone just has to make a CL for it. @dcharkes ?

Answer 7 · 2023-04-21T12:08:32.000Z

It's assumed that if the pointer hangs on to native memory which needs freeing / releasing / ... that an enclosing class refers to it and has a finalizer attached. Sending such objects with finalizers is disallowed.

This is actually my explicit usecase. My classes implement NativeFinalizers, and I'm keeping references to them from the main isolate. Is this disallowed because of the memory cloning aspect of isolates? I.e., when the isolate is done the copied objects get GC'd and then the finalizer is triggered, rendering the object held outside the isolate invalid?

Answer 8 · 2023-04-21T12:11:45.000Z

Is this disallowed because of the memory cloning aspect of isolates? I.e., when the isolate is done the copied objects get GC'd and then the finalizer is triggered, rendering the object held outside the isolate invalid?

Yes, allowing a clone of a finalizable object would make things problematic: Should the finalizer run when all clones are dead, should it run multiple times / for every clone, does cloning a finalizable need to perform some action (e.g. increasing a refcount), ...

Answer 9 · 2023-04-21T12:15:26.000Z

Is there some generally accepted solution for making asynchronous FFI functions? I have potentially long running methods in my underlying libraries and don't want to block the UI on them. I figured isolates was an easy way to do that, but I guess not, due to the finalizer restrictions.

Answer 10 · 2023-04-21T12:18:20.000Z

Is there some generally accepted solution for making asynchronous FFI functions? I have potentially long running methods in my underlying libraries and don't want to block the UI on them. I figured isolates was an easy way to do that, but I guess not, due to the finalizer restrictions.

Yes, the most simple way is to spawn an isolate and run a (possibly blocking) ffi call there. Is it possible for you to pass finalizable state via C memory (not dart objects) to the other isolate or make the isolate create this state on it's own?

Answer 11 · 2023-04-21T12:19:45.000Z

If you do more than 16 isolates, make sure to use what @mkustermann added recently to address:

#51254

Answer 12 · 2023-04-21T12:23:08.000Z

Yes, the most simple way is to spawn an isolate and run a (possibly blocking) ffi call there. Is it possible for you to pass finalizable state via C memory (not dart objects) to the other isolate or make the isolate create this state on it's own?

I figured it would come to this. It's not ideal, but it's probably the only real solution. Thanks for the help.

Answer 13 · 2023-04-22T07:18:35.000Z

We'd like to eventually represent them as simple integers when passing them around.

What are the implications of this? Will the type signatures of Pointer<NativeType> and associates go away at some point in the future, and are there any issues in particular that discuss this?

Answer 14 · 2023-04-24T07:39:22.000Z

We'd like to eventually represent them as simple integers when passing them around.

What are the implications of this? Will the type signatures of Pointer<NativeType> and associates go away at some point in the future

No, the static types will all stay.

and are there any issues in particular that discuss this?

#50777

Answer 15 · 2023-09-18T21:27:19.000Z

If Pointer is passable between isolates then I think that ffigened Objective-C classes will also be passable. But the reference counts would be wrong and users can expect crashes.

For cases like that, the generated classes should probably have @pragma('vm:isolate-unsendable') applied or we need some approach of copying finalizer-like things (as @mkustermann mentioned).

@liamappelbe

Answer 16 · 2023-10-08T14:38:40.000Z

I also encountered this problem. I find a simple solution, but I can't say whether it may have other problems.

void blockingFunc() async {
  final argPtr = malloc<Char>();
  final argPtrInt = argPtr.address;
  await Isolate.run(() => _bindings.blocking_func(Pointer.fromAddress(argPtrInt));
  malloc.free(argPtr);
}

Can't pass Pointer, then I directly pass the address, which is int.

Answer 17 · 2023-11-24T20:32:38.000Z

I made an example of SharedPointer that can be shared between isolates and it will call the native callback when it is no longer needed. Perhaps this will be useful to someone

https://github.com/nikitadol/ffi_isolate_test

Answer 18 · 2024-02-07T13:17:03.000Z

As mentioned above in #50457 (comment) we consider Pointers to be simple integers and may actually represent them as such in the future.
=> With eba0e68 landed, we do allow sending Pointers across SendPorts now.

Users should be mindful that using FFI is stepping outside the safe sandbox. As a result one should be careful with how pointers are handled. If they represent a resource that needs freeing, a class implementing Finalizable should encapsulate it. Such finalizables cannot be sent across ports.