Design and implement Dart VM FFI
mraleph opened this issue · 129 comments
The present feature tracks the implementation of Dart FFI support enabling interop with C & C++ code from Dart.
Status
The feature will be in stable as of Dart 2.12. More details here:
https://dart.dev/guides/libraries/c-interop
For any discussion of this feature, or feedback or questions regarding the feature, kindly join and post the dart-ffi group: https://groups.google.com/forum/#!forum/dart-ffi
We will still actively adding new features to dart:ffi. Please refer to
library-ffi
for open issues and feature requests.
Background
Some inspirational references:
A general FFI could also be useful for interaction with JavaScript, so we should consider whether it's possible to make something general wihout compromising on the usability on each platform.
The current JS-interop functionality is not a clean design, it should be possible to improve it.
The current JS-interop functionality is not a clean design, it should be possible to improve it.
I would not assume that without talking to @vsmenon. The syntax might leave a little to be desired, but I have no desire to make a breaking change to the syntax unless it vastly improves end users, not just the fact it isn't a "clean design".
Providing a new syntax / library doesn't necessarily imply removing the existing support. With that in mind, I think @lrhn 's suggestion is good.
Is this referring to Dart calling C/C++ interfaces or vice versa?
@krisgiesing Initially it will be Dart calling C, later we would like to extend this to C calling Dart.
Though it is very hard to draw a line because we plan to support callbacks as part of the initial offering - so C would be able to call back into Dart.
Here is the markdown version of the vision doc (implementation is currently being prototyped).
Let me know if you have any comments on that one.
This potentially requires using conditional imports.
Conditional compilation is a pain for IDEs. It's worth looking into how Kotlin handles a similar problem with expect/actual declarations on a language level. In a nutshell, expect/actual means that you write a single header/interface file, and a corresponding implementation for each platform. Because the interface is shared between all platforms, IDE does not need to know the "current" platform to do IDE stuff.
@matklad the expect/actual stuff is pretty close to interface libraries that were discussed in context of the conditional imports long time ago - exactly to solve IDE problem. However I don't think this went anywhere - and I think conditional imports are pretty much dead.
I think in the context of FFI we will be implementing what Structure Layouts and Portability describes.
I will amend the section that still talks about conditional imports.
Any updates for dart lang FFi ? very happy if there have usable FFI .
We can import c/cpp language lib with it.
@netroby I'm working on it. We plan to add an initial version during Q1. So far the prototype closely follows the vision doc. Let me know whether that covers your use case.
Seems great, calling C is a must have. Is there a way to get access to the proto? Thanks.
@fimbault we plan to release prototype for public consumption later this quarter in few stages: first we will land support for FFI in JIT mode on X64 only, then it will be expanded to cover X64, ARM, ARM64 in JIT and AOT.
@matklad the expect/actual stuff is pretty close to interface libraries that were discussed in context of the conditional imports long time ago - exactly to solve IDE problem. However I don't think this went anywhere - and I think conditional imports are pretty much dead.
I think in the context of FFI we will be implementing what Structure Layouts and Portability describes.
I will amend the section that still talks about conditional imports.
are you talking about this: #24581 ?
I use it and it is great.
Are there plans to support calling into shared libraries similar to python's ctypes?
@robertmuth yes, please see the doc referenced from #34452 (comment) for more details.
It will be great to have FFI in Dart.
From my perspective, FFI includes the following major features:
- Load dynamic library by name
- Lookup function in library by name
- Define native function signature
- Map dart types to native types and vice versa (including structs and unions)
There is proposal for all of them here.
Some thoughts:
For 1.
It's proposed to load library by name manually:
final lib = DynamicLibrary.open('libfoo.so');It is good approach. However there are some alternative ways:
a. Load library manually but return class were all native (or abstract) functions resolved to dynamic library functions:
class Foo extends DynamicLibrary {
int add(int a, int b) native;
}
final Foo lib = DynamicLibrary.open<Foo>(Platform.isLinux ? 'libfoo.so' : 'foo.dll');
lib.add(1,2); //3Dart VM lookup all native function in class automatically.
JNA
b. The same as a. but use class name as dynamic library name:
//mapped to libfoo.so on Linux, to libfoo.dylib on macOS and foo.dll on Windows
class Foo extends DynamicLibrary {
int add(int a, int b) native;
}
final Foo lib = new Foo();This automatic mapping is implemented in native extensions: Dart's library name mapped to dynamic library name. Nothing new.
c. The same as a. but use annotation on class:
@ffi.Library('foo')
class MyFoo {
int add(int a, int b) native;
}
final Foo lib = new Foo();If library has different names (for instance OpenGL) than annotation can accept list of names.
@ffi.Library('Opengl32.dll', 'libGL.so.1')
class OpenGL {
}d. Don't use class and define library name for each function:
@ffi.Library('foo')
int add(int a, int b) native;C# DllImport.
Dart automatically load dynamic library by name and lookup function (symbol).
Disadvantages: developer doesn't control when dynamic library is loaded/unloaded. At least special API is needed.
e. Use Dart library name as in native extensions
Disadvantages: developer doesn't control when dynamic library is loaded/unloaded, there is no way to define different names.
For 2.
It is OK to be able to lookup function manually.
However, imagine a library with hundreds functions.
Have hundreds lines of code like:
final add = lib.lookupFunction<ffi.Int32 Function(ffi.Int32, ffi.Int32), int Function(int, int)>('add');isn't great.
There are some other options to lookup function automatically:
a. Lookup by function name:
int add(int a, int b) native; //lookup 'add' function in dynamic libraryb. Define name after native keyword as in native extensions:
int myadd(int a, int b) native 'add'; //lookup 'add' function in dynamic libraryc. Lookup by name in annotation:
@ffi.Function('add')
int myadd(int a, int b) native; // lookup 'add' function in dynamic libraryIt works in for cases 1.a-1.c and for 1.d.
Anyway, I believe, it should be a way to load functions automatically based on function signature, which is already defined in my Dart code.
For 3. and 4.
It would be great to have something like this:
int add(@ffi.Type('Int') int a, @ffi.Type('Int') int b)where annotation @ffi.Type define C type of parameter.
Top C data types should be supported: integer types (char, short, int, long, long long including unsigned), float types, pointers etc.
Dart VM should resolve their sizes in bits automatically based on run-time platform.
It isn't enough to support platform-independent types like int32_t or unit64_t.
It would be inconvenient to define sizes for each platform manually.
Some real world examples:
void srand(@ffi.Type('Uint') int seed); //libc on Linux
@ffi.Type('Ulong') int GetLastError(); //kernel32 on Windows
int SDL_Init(@ffi.Type('Int32') int flags); //SDL cross-platform
void SDL_ShowWindow(@ffi.Type('IntPtr') int window); //SDL cross-platformFor instance, SDL uses both platform-independent (int32_t) and platform-dependent (int, int *) types in API.
For structs it would be great to have automatic struct packing based on field order, its C type and run-time platform.
Instead of:
@ffi.struct({
'x64 && linux': { // Layout on 64-bit Linux
'x': ffi.Field(ffi.Double, 0),
'y': ffi.Field(ffi.Double, 8),
'next': ffi.Field(ffi.Double, 16)
},
'arm && ios': { // Layout on 32-bit iOS
'x': ffi.Field(ffi.Float, 4),
'y': ffi.Field(ffi.Float, 8),
'next': ffi.Field(ffi.Pointer, 0)
},
})
class Point extends ffi.Pointer<Point> {
double x;
double y;
Point next;
}it would be great to write something like:
class Point {
@ffi.Type('Double')
double x;
@ffi.Type('Double')
double y;
@ffi.Type('Pointer')
Point next;
}It is enough information for Dart VM to pack structure on each supported platform.
However, ability to define field offset and struct size manually is welcome:
@ffi.StructSize(24)
class Point {
@ffi.Offset(0)
@ffi.Type('Double')
double x;
@ffi.Offset(8)
@ffi.Type('Double')
double y;
@ffi.Offset(16)
@ffi.Type('Pointer')
Point next;
}The dart:ffi prototype has progressed to the point where we are deciding on API decisions. (Some design decisions are discussed here.)
To make informed design decisions, we would like more examples on what C APIs you would like to bind. The Flutter C++ interop issue mentions SQLite, Realm, OpenCV, Superpowered, and ffmpeg. Any other APIs we should consider?
@dcharkes two APIS i've been thinking about are libvlc and telegrams tdlib https://github.com/tdlib/td and possibly SDL2
I believe databases support is often arises as a problem. ODBC, Oracle OCI and so on would be helpful
I personally made some half baked wrapper for http://www.libxl.com/ . It would be much prettier with the new FFI, I think
Having a convenient way to interact with Kotlin/Native would be extremly usefull (even if it would of course "somehow" work with a simple C FFI).
I'm looking forward to the possibility of accelerated cryptographic ops using libsodium or openssl
It isn't popular case (server-side) but still: SDL2, OpenGL, Vulkan, OpenAL. I wish I can remove all native extensions code and achieve better performance with FFI.
Also kernel32.dll and user32.dll on Windows and libc.so on Linux.
I am interested in using GTK+ with Dart.
Is there a plan for distributing packages that use the new FFI, i.e. on Pub?
@thosakwe I've discussed this @jonasfj, but at this point we don't know how yet we want to approach this. Should we distribute source or binaries? (Or provide the option to do either?) It probably depends on the use case. And if the package is used within Flutter, can we reuse something that Flutter does? We're aware of these questions, but do not have an answer yet.
@thosakwe i believe providing both options would be a huge plus.
I'm interested in embedding my own business-logic as a library in Flutter.
crypto lib write by C or Rust
It would be nice to have a rust-bindgen alternative for the Dart
Using Dart / Flutter with Qt might be interesting for desktop apps.
Using Dart / Flutter with Qt might be interesting for desktop apps.
Why?
Skia works in all the major platforms. Sublime Text uses Skia and is pretty fast and lightweight as well :)
Using Dart / Flutter with Qt might be interesting for desktop apps.
Why?
Skia works in all the major platforms. Sublime Text uses Skia and is pretty fast and lightweight as well :)
Skia is brilliant and it can be used in Qt apps as well (either by painting an SkSurface to a QWidget with QPainter, or by painting directly to a QWindow's context) with Qt providing things like cross-platform windowing and docking, c++ plugin support, installers/bundle builders, networking APIs, parsers and more.
@vejmartin This would increase the size of the executable by a big margin in case you use native features, I think
Another great thing would be using the FFI to access serial ports.
- for serial support but I'd rather see it implemented as part of the standard libraries.
c.f.: #25360
We are now at a point where we would like to offer an early preview to get some feedback. We’re looking for feedback on how ergonomic the feature is to use, what the main gaps are, and lastly on general stability and performance.
We are looking for testers who are willing to:
- Test the feature by interfacing with a concrete C based library such as SQLite or libphonenumber
- Tolerate a feature that is still incomplete and unstable
- Offer us feedback on their experience
There is a small SQLite sample demonstrating the FFI feature in the SDK: Readme, FFI wrapper, sample program.
If you are interested, please consider one or more potential libraries you are interested in, and send an email with a description of those to dart-ffi-team@googlegroups.com (we’ll try to make sure we don’t several wrapping the same lib).
Also, you should be aware of the following current main limitations:
- C++ not yet supported
- Callbacks from C back into Dart not supported
- Finalizers not supported
- Field support is limited to 64-bit desktop
- No iOS support
On behalf of the Dart FFI team, Michael
I'm so sorry, but we had an issue with the google group mentioned above, so if you already sent an email to dart-ffi-team@googlegroups.com, can you please send it again?
Thanks much!
@mit-mit tried loading a basic shared library built from go on Linux but i am unable to get it to work
shared library build command
go build -o dart/bin/test.so -buildmode=c-shared lib.go
and tried opening using
ffi.DynamicLibrary l = dlopenPlatformSpecific("test", path:"/home/kingwill101/go/src/gitlab.com/kingwill101/test/dart/bin/");
dlopenPlatformSpecific comes from the examples provided
ArgumentError (Invalid argument(s): Failed to load dynamic library(lib/home/kingwill101/go/src/gitlab.com/kingwill101/test/dart/bin/libtest.so: cannot open shared object file: No such file or directory))
loading a system library works but trying to load using a relative path to the library fails.
@glenfordwilliams I suggest extending dlopenPlatformSpecific to create a full path based on where your .dart file is. You can construct the full path in Dart.
Explanation: the paths are directly interpreted by dlopen. This means the relative paths are relative to the dart binary. (This test uses a relative path, and the .so file lives right next to the dart binary.) In general, one would want a path to the .so file relative to the .dart file using that .so file, which would be constructed in Dart.
@dcharkes i tried doing that earlier but for some reason dart adds lib in front of the created path
Exception has occurred.
ArgumentError (Invalid argument(s): Failed to load dynamic library(lib/home/kingwill101/go/src/gitlab.com/kingwill101/test/dart/bin/test.so: cannot open shared object file: No such file or directory))
@glenfordwilliams Yes, you're gonna have to modify or make your own version of dlopenPlatformSpecific. The example implementation simply prepends "lib" on Linux on line 10.
It's Linux convention to prepend lib to the names of shared libraries. However, go build won't do that automatically. If you're not planning to run your code on other platforms, you can just use DynamicLibrary.open directly.
sorry about that, vscode seems to be caching old code
import 'dart:ffi' as ffi;
import 'dart:io';
import "package:path/path.dart" show dirname;
import 'dart:io' show Platform;
typedef GoFunction = ffi.Void Function();
typedef GoFunctionOp = void Function();
ffi.DynamicLibrary dlopenPlatformSpecific(String name, {String path}) {
String fullPath = dirname(Platform.script.path) + "/" + name;
return ffi.DynamicLibrary.open(fullPath);
}
main(List<String> arguments) {
Directory current = Directory.current;
print(current.path);
ffi.DynamicLibrary l;
try {
l = dlopenPlatformSpecific("libtest.so");
} catch (e) {
print(e.toString());
}
var go = l.lookupFunction<GoFunction, GoFunctionOp>("AGoFunction");
go();
}
strange thing is i get everything to work when i run from the command line
kingwill101@kingtech:~/go/src/gitlab.com/kingwill101/test/dart$ ~/flutter/bin/cache/dart-sdk/bin/dart bin/main.dart
/home/kingwill101/go/src/gitlab.com/kingwill101/test/dart
AGOFUNCTION()
kingwill101@kingtech:~/go/src/gitlab.com/kingwill101/test/dart$
but when i run from vscode i get
bin/main.dart:25:14: Error: Expected type 'NativeFunction<Void Function()>' to be a valid and instantiated subtype of 'NativeType'.
- 'NativeFunction' is from 'dart:ffi'.
- 'Void' is from 'dart:ffi'.
var go = l.lookupFunction<GoFunction, GoFunctionOp>("AGoFunction");
^
What version of the Dart SDK is your VSCode install using, and what plugins are you using?
totally forgot another version installed system wide, so vscode may be using that one
kingwill101@kingtech:~/go/src/gitlab.com/kingwill101/test/dart$ dart --version
Dart VM version: 2.2.0 (Unknown timestamp) on "linux_x64"
The FFI Preview is not available on that version of Dart; you'll need use top-of-tree or wait for Dart 2.3.
Hi,
First of all thanks to bring such functionality in Dart it’s really awesome and a long awaited feature.
We come after some testing of the ffi on the libsodium and here are what we gather.
First of all starting is not really hard with the Sqlite example and we can quickly wrap some key function of libsodium for benchmark purpose.
Here is our repos https://github.com/santetis/libsodium.dart.
We have wrap the initialization of libsodium, sha256 and sha512 to benchmark them against the crypto package (only support sha256) and pointycastle package (which support both). And here are the result :
Really quick to have something working with ffi (something like ~2/3h to have the 3 functions)
ffi is really slow (up to 10 time slower) (in fact it’s not really ffi but the conversion between type)
Here are the result of the benchmark we run (you can see the code in the benchmark folder).
For the crypto package :
master $ dart crypto/main.dart
CryptoSHA256: FooBar
CryptoSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
CryptoSHA256(RunTime): 16.493551047336304 us.
For the pointycastle package :
master $ dart pointycastle/main.dart
PointyCastleSHA256: FooBar
PointyCastleSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
PointyCastleSHA256(RunTime): 15.91678736848807 us.
PointyCastleSHA512: FooBar
PointyCastleSHA512: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
PointyCastleSHA512(RunTime): 70.59812919166961 us.
For the libsodium package :
master $ dart libsodium/main.dart
LibSodiumSHA256: FooBar
LibSodiumSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
LibSodiumSHA256(RunTime): 172.85213032581453 us.
LibSodiumSHA512: FooBar
LibSodiumSHA512: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
LibSodiumSHA512(RunTime): 285.42451484018267 us.
LibSodiumSHA512FFIOnly: FooBar
LibSodiumSHA512FFIOnly: Pointer<Uint8>: address=0x55b2ebc8fe80
LibSodiumSHA512FFIOnly(RunTime): 7.8313598346020115 us.
How to read this result :
LibSodiumSHA512 is the benchmark for using a function that can be used in every dart program no ffi type is in the function signature
LibSodiumSHA512FFIOnly is the benchmark with only the ffi part. Here you will need to pass a Pointer<T> to the function and the result you get is also a Pointer<T>
What can we extract from this result ?
ffi seems super quick but we need to transform the data to send it and receive it since the Pointer<T> object is not really useful in dart.
Here we are converting String to Pointer<Uint8> that it seems it’s the part that is taking all the time.
So some question now :
- Is there a way to improve that part already ?
- If not is there some plan to automatically marshall built-in Dart type in a way that is faster than what we can do now.
@Kleak Thanks for trying FFI out, Kevin!
This was not explicitly highlighted in @mit-mit's invitation - but we are well aware that performance of FFI is lacking - especially when it comes to workloads which are reading and writing a lot of data through Pointer<T> on the Dart side. This is however will be fixes in the MVP and Release versions of FFI.
What we are looking for is feedback on API design - rather than performance.
To put this into perspective a line of code like this:
ptr.elementAt(i).store(codeUnits[i]);currently translates into two calls into runtime system and an allocation - which means CPU has to chew through several hundred CPU instructions (if not more). Once all planned optimizations are implemented this like of code should translate into just a few CPU instructions.
If not is there some plan to automatically marshall built-in Dart type in a way that is faster than what we can do now.
We do plan to provide some builtin ways to marshal common data types like strings and arrays.
I think one of the obvious conclusions we can make here is that CString-like class should be provided by FFI code (either dart:ffi or maybe package:ffi), rather than implemented by all FFI clients again and again.
Thanks for all the explanation @mraleph .
For the API design in found it pretty intuitive. And except that yes provide basic type in FFI code i found it's fine. I haven't test all the capabilities so this is based on what i have seen.
Happy to help on the CString-like class i think it's better to have a package for this kind of thing, like that it can evolve more quickly on a solid base provided by the dart sdk.
I will continue to explore more in the coming week.
@mraleph one more question
How to distribute the .so/.dylib/.dll ?
Should we let the user deliver this part or should we distribute it ?
I think this part should be covered in the example because it's really not obvious.
@mraleph haven't had a chance to experiment with FFI yet, but are there plans to have an easy way to bridge external allocations to typed data? Like maybe a way to create External TypedData from a pointer+length pair?
I have a Rust library that takes large images and streams in fixed-sized chunks of raw image data. I need to turn these into typed data (hopefully without unnecessary copies) so I can pass it into dart:ui's decodeImageFromPixels function, which takes a Uint8List.
It would also be helpful to have zero-copy String and non-external TypedData handling, dunno if that's possible since Dart moves heap objects around during GC.
@Kleak We are aware of the distribution question, but we do not have an answer yet.
Is there a plan for distributing packages that use the new FFI, i.e. on Pub?
@thosakwe I've discussed this @jonasfj, but at this point we don't know how yet we want to approach this. Should we distribute source or binaries? (Or provide the option to do either?) It probably depends on the use case. And if the package is used within Flutter, can we reuse something that Flutter does? We're aware of these questions, but do not have an answer yet.
@ds84182 yes, supporting marshalling of arrays and strings as external types is one the list of items we would totally want to support in FFI. I am not sure if we have a bug filed for this (/cc @dcharkes @sjindel-google) but it was definitely part of the original vision.
It would also be helpful to have zero-copy String and non-external TypedData handling, dunno if that's possible since Dart moves heap objects around during GC.
This is possible if you can guarantee that GC can't occur, e.g. if native method is declared as a leaf (@sjindel-google just filed a bug to support those #36707).
The support for strings and arrays can be built as a library on top of the API available now, but these should probably come included with dart:ffi. I've filed https://github.com/dart-lang/sdk/issues/36711
#36247 is tracking FFI performance. Other benchmarks are welcome.
Is there an example to create a flutter application with ffi ?
Especially the part where to put the .so file.
@Kleak Please check out samples/ffi/sqlite/doc/android.md in the repo.
@mraleph @sjindel-google
One little problem i'm facing right now is that in libsodium some part of the lib are available only on some platform. So i ended up writing some generic code to add the capabilities to check is the function is available before running it.
Here is the generic code :
abstract class Bindings<DartSignature extends Function, NativeSignature> {
final String functionName;
DartSignature f;
Bindings(DynamicLibrary sodium, this.functionName) {
try {
f = sodium
.lookup<NativeFunction<NativeSodiumInitSignature>>(functionName)
.asFunction<DartSignature>();
} catch (_) {}
}
bool get isAvailable => f != null;
}
and here an example of how to use it :
class SodiumInitBindings
extends Bindings<SodiumInitSignature, NativeSodiumInitSignature> {
SodiumInitBindings(DynamicLibrary sodium) : super(sodium, 'sodium_init');
int call() {
if (!isAvailable) {
throw BindingIsNotAvailableException(
'$functionName bindings is not available on ${Platform.operatingSystem}');
}
return f();
}
}
where SodiumInitSignature correspond to typedef int SodiumInitSignature(); and NativeSodiumInitSignature correspond to typedef Int32 NativeSodiumInitSignature();
the problem is when i run this code i got
lib/src/bindings/bindings.dart:12:12: Error: Expected type 'DartSignature' to be 'int Function()', which is the Dart type corresponding to 'NativeFunction<Int32 Function()>'.
- 'NativeFunction' is from 'dart:ffi'.
- 'Int32' is from 'dart:ffi'.
.asFunction<DartSignature>();
^
So my question is : Is that normal behavior ? because I found it really annoying :/
Could you please add ability to create Pointer from TypedData or ByteBuffer or List?
It should neither allocate memory nor copy data operation.
It is "must have" feature to use Dart FFI with audio, video and grapchic libraries.
@rootext Note that in general creating and using interior pointers into Dart objects is not safe, because GC is managing these objects (deleting them and moving them around) - so if you need a pointer that is valid for a longer period of time then you can't just pass around pointer into a typed data object. If GC happens the pointer might become invalid.
@mraleph, I am going to use Dart FFI for OpenGL API.
There are few functions calls where I need to send pointer to raw byte buffer:
glShaderSource, glBufferData, glTexImage2D.
What is the recomended way for this?
@rootext you will have to allocate raw native memory, then fill it in on the Dart side then send it. There will be a way to make typed data object from raw native memory to make actual filling more ergonomic.
@rootext future. FFI is not part of 2.3 anyway - it's in (super early stage) preview.
@Kleak sorry for the unhelpful error message - but the code you are writing is not expected to work. We currently require that type arguments arguments to lookup and asFunction are compile time constants. The reason for that is AOT: in AOT we need to know what kind of trampolines to generate for going between Dart and native code. These trampolines would be the most efficient when their Dart and native signatures is statically known.
But this definition is known at compile time no ?
class SodiumInitBindings
extends Bindings<SodiumInitSignature, NativeSodiumInitSignature> {
SodiumInitBindings(DynamicLibrary sodium) : super(sodium, 'sodium_init');
int call() {
if (!isAvailable) {
throw BindingIsNotAvailableException(
'$functionName bindings is not available on ${Platform.operatingSystem}');
}
return f();
}
}
all the generic are known at compile time and are constant in the subclass
EDIT: is there a way to mimic that or do i have to write that every time ? is this a limitation of the preview or this will stay like this (because it's something valid in dart) ?
@Kleak it would be know in compile time if Dart generics worked like C++ templates, which are instantiated by expansion (essentially cloning them with type parameters replaced with actual types), but they don't. When you write
abstract class Bindings<DartSignature extends Function, NativeSignature> {
final String functionName;
Bindings(DynamicLibrary sodium, this.functionName) {
try {
f = sodium
.lookup<NativeFunction<NativeSodiumInitSignature>>(functionName)
.asFunction<DartSignature>();
} catch (_) {}
}
}
that more or less means (this is not a valid syntax - but that's how it works)
abstract class Bindings {
final List<Type> :typeArguments; // secret field that contains type arguments
Bindings(DynamicLibrary sodium, this.functionName) {
try {
f = sodium
.lookup<NativeFunction<this.$typeArguments[0]>>(functionName)
.asFunction<this.$typeArguments[1]>();
} catch (_) {}
}
}
is there a way to mimic that or do i have to write that every time? is this a limitation of the preview or this will stay like this (because it's something valid in dart)?
I don't think there is a way to mimic what you want - your best bet is just to generate boilerplate code using some helper script.
This limitation is somewhat intrinsic to how things are designed and how Dart generics work. So I am not too optimistic that it would change - however if we discover that this comes up often, then we need to go back to the drawing board.
[As I mentioned above - this limitation originates from the desire to make things efficient in AOT. Maybe we can make a special annotation to force some generics expand at compile time in AOT.]
@mraleph is there an example that shows how to map something like const char * SomeFunction() to a dart function ?
@glenfordwilliams, you can use this class from sdk tests https://github.com/dart-lang/sdk/blob/master/tests/ffi/cstring.dart
typedef SomeFunction = CString Function();
@rootext tried that earlier but had issues with it. I'm trying to integrate a lib built using cgo which generates the following for strings
#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef struct { const char *p; ptrdiff_t n; } _GoString_;
#endif
tried using
@ffi.Struct()
class GoString {
CString p;
}
but that doesn't work
@glenfordwilliams do I understand correctly that you have a function that returns _GoString_ rather than const char*, e.g. _GoString_ SomeFunction().
Currently binding such functions is not supported. (See #36730)
However if you have a function that returns a pointer to _GoString_ like _GoString_* SomeFunction() - then you should be able to bind it.
@mraleph yes that's pretty much it
#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef struct { const char *p; ptrdiff_t n; } _GoString_;
#endif
#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef _GoString_ GoString;
#endif
extern GString GetString();
Hi, thanks for shipping this preview, I wrote a simple prototype using Realm https://github.com/nhachicha/realm-dart-ffi
some feedback:
-
This has been mentioned a couple of time but there's a lack of wrappers around basics types, I think
CStringis a good example (maybe extend to others likeenumwhich can map to integral types in C/C++). -
Proxy objects will be created from Dart to abstract the low-level implementation in C++(Row/Table/Queries etc), since these allocations happened in the heap from the C++ side there's a need to use a finalizer object to detect instances going out of scope so we can free the native resources.
-
I won't mention the lack of callbacks from C/C++ since this is already in your roadmap.
-
Sometimes the C/C++ needs to return an array of primitives (as opposed to a pointer), currently accessing each element via an FFI call is suboptimal for large arrays... is there any similar
approach to JNI where we can access a Dart array backed by the same memory or allow some kind of move semantics internally which transfer the array element to Dart without the need for copying/marshalling. -
Once we have callbacks from C/C++, it is unclear how the
pthreadswill interact with Dart?, will the callback invoke in an Isolates or in the Event Loop? for Realm there are still some internal resources which are thread confined, using some resources allocated in a callback in one thread then accessing them from another could be problematic.
Thanks for the feedback. The first three of suggestions we are very aware of and are on our roadmap. I'm curious about the last two though:
Sometimes the C/C++ needs to return an array of primitives (as opposed to a pointer), currently accessing each element via an FFI call is suboptimal for large arrays... is there any similar
approach to JNI where we can access a Dart array backed by the same memory or allow some kind of move semantics internally which transfer the array element to Dart without the need for copying/marshalling.
I'm not sure what you mean by "return an array of primitives (as opposed to a pointer)". What's problematic about returning a pointer to a heap-allocated array and access it via Pointer.load and Pointer.store?
Once we have callbacks from C/C++, it is unclear how the pthreads will interact with Dart?, will the callback invoke in an Isolates or in the Event Loop? for Realm there are still some internal resources which are thread confined, using some resources allocated in a callback in one thread then accessing them from another could be problematic.
In the Dart VM, each thread can be inside of at most one isolate at a time (newly created threads are not inside any isolate by default). Within each isolate, only one thread in that isolate can execute Dart code (called the "mutator" thread).
We have two plans for callbacks: synchronous callbacks and asynchronous callbacks.
A synchronous callback must be invoked on the mutator thread of the isolate it was taken from (for example, it can be invoked by C code which in turn was called by Dart via an FFI call). The VM will abort if it's called on the wrong thread.
An asynchronous callback will add a task to the microtask queue which will eventually invoke the callback on the mutator thread. An thread inside the appropriate isolate can invoke an asynchronous callback, but will block until the microtask is processed by the mutator thread.
How does this work for Realm?
@sjindel-google
Asynchronous callbacks are definitely a must-have feature.
Could you please show how it will work on an example?
Suppose we have a C++ function
using Input = std::string;
using Result = std::string;
void doHeavyTextProcessing(Input input, std::function<void(Result)> onReady);
which makes computations on an internal thread and invokes the callback on it.
What code should be written to make it available from Dart as
Future<String> doHeavyTextProcessing(String input);
?
Here's some thoughts on the current FFI api:
I somewhat dislike the current way of specifying native functions... It's great for unstructured testing, but I feel that it somewhat falls apart when you're trying to specify a large mass of functions.
Something like this would be nicer:
abstract class Kernel32 extends Library {
bool QueryPerformanceCounter(Pointer<Uint64> out);
bool QueryPerformanceFrequency(Pointer<Uint64> out);
}
abstract class FooBar extends Library {
@Int32
int Truncate64(@Int64 int x);
}
// Part of dart:ffi
class DynamicLibrary {
// ...
T asLibrary<T extends Library>();
}It would make specifying and organization a bit easier.
Another thing is that bools seem to be unsupported?
Also analyzer lints would be awesome! A lot of issues I discover with my FFI code is something that could've been caught by the analyzer instead of the frontend.
@ds84182 These are some great suggestions!
We have to specify both the Dart signature and C signature, or the FFI does not know how to marshal between the Dart and C representation of values passed to functions. This encodes both those signatures:
abstract class FooBar extends Library {
@Int32
int Truncate64(@Int64 int x);
}This we could maybe get to work, tracking in #36854.
Another thing is that bools seem to be unsupported?
Yes, we use Uint8 for those. Though I see that that requires one to convert between a bool and number on the Dart side. We could add a Bool C type that is treated as uint8_t and corresponds to the bool Dart type. Discussing in #36855.
Also analyzer lints would be awesome! A lot of issues I discover with my FFI code is something that could've been caught by the analyzer instead of the frontend.
Already on our todo list: #35777.
I have considered syntax like this
@Int32
int Truncate64(@Int64 int x);But unfortunately it does not work with typedefs:
typedef FunctionPointer = @Int32 int TruncationFunction(@Int64 int x);
// ^^^^^^ can't put an annotation hereSo you end up with two ways of specifying how native types map to Dart types - that is why I opted for the dual-signature approach which works in all cases where you need it. I do agree that the current approach is very verbose and needs to be either improved, shortened or autogenerated.
I'm agree that the current approach is too verbose and can become really messy / hard to read
I really like the annotation approach :) maybe allowing annotation on typedef might be something to consider ? and until it's done just not allowing typedef definition...
The problem with not using typedefs is that if you have a function referring to another function you have to repeat the signature of the function that is referred to.
I'm not sure what you mean by "return an array of primitives (as opposed to a pointer)". What's problematic about returning a pointer to a heap-allocated array and access it via Pointer.load and Pointer.store?
This works, however, it will be inefficient compared to a Dart array/list backed directly by the elements of this primitive array. Otherwise, you'll have to do N FFI call (ptr.load()) to access each element (that's my assumption).
An asynchronous callback will add a task to the microtask queue which will eventually invoke the callback on the mutator thread. An thread inside the appropriate isolate can invoke an asynchronous callback, but will block until the microtask is processed by the mutator thread.
How does this work for Realm?
Internally Realm has the possibility to hand over confined objects between threads, so as long as we resolve the result of the callback in the mutator thread this should work both for synchronous & asynchronous callbacks.
Here's the flow I imagine:
Dart (mutator thread) C/C++ (mutator thread*) C/C++ (worker_thread) C/C++ (mutator thread*) Dart (mutator thread)
----------------------------- --------------------------------- ----------------------------- ---------------------------------------------- --------------------------------------------------------
findAllAsync(query, callback) ------> FFI call to native method -----> spawn a worker pthread ----> hand over the result to the mutator thread ---> trigger callback in Dart with a pointer to results
* this assumes that the FFI invocation in C/C++ runs within the mutator thread?
I'm not sure what you mean by "return an array of primitives (as opposed to a pointer)". What's problematic about returning a pointer to a heap-allocated array and access it via Pointer.load and Pointer.store?
This works, however, it will be inefficient compared to a Dart array/list backed directly by the elements of this primitive array. Otherwise, you'll have to do N FFI call (
ptr.load()) to access each element (that's my assumption).
Currently you're correct that ptr.load() is slower than access through e.g. TypedData. However, we are planning to optimize it in the future and it should eventually have similar performance, and I expect we'll create wrapper classes to implement List<int> and Uint8List, etc. out of Pointers.
We are also planning to expose Dart-allocated memory to native code as well. This will always be limited to "leaf" functions: short-running native functions which don't call back into Dart. The problem with this is that GC can be triggered while Dart code is running, and GC can move the Dart-allocated memory. Exposing Dart memory to native code only works then when we pause GC across the native call.
An asynchronous callback will add a task to the microtask queue which will eventually invoke the callback on the mutator thread. An thread inside the appropriate isolate can invoke an asynchronous callback, but will block until the microtask is processed by the mutator thread.
How does this work for Realm?
Internally Realm has the possibility to hand over confined objects between threads, so as long as we resolve the result of the callback in the mutator thread this should work both for synchronous & asynchronous callbacks.
Here's the flow I imagine:Dart (mutator thread) C/C++ (mutator thread*) C/C++ (worker_thread) C/C++ (mutator thread*) Dart (mutator thread) ----------------------------- --------------------------------- ----------------------------- ---------------------------------------------- -------------------------------------------------------- findAllAsync(query, callback) ------> FFI call to native method -----> spawn a worker pthread ----> hand over the result to the mutator thread ---> trigger callback in Dart with a pointer to results
- this assumes that the FFI invocation in C/C++ runs within the mutator thread?
Your assumption is correct -- this will work as you intend. In you diagram, only synchronous calls and callbacks are being used.
Wouldn't it be less messy to just have some code generation around this rather than trying to annoate typedefs? I haven't thought all the way through it so I may be missing something, but it seems like it should be possible.
@dnfield It would be great to have an FFI implementation that is concise and doesn't require code generation (which is a heavy-weight hammer) to interface with C/C++ code. For example, when I was tracking down flutter/flutter#32121 I used FFI outside of a package. It was much quicker than other languages (C++? gotta figure out a build system (CMake? GN? Visual Studio?). Rust? needs to create a project, download the winapi crate, build it...), and I'd like it to stay that way...
To be honest I tend to stay away from code generation in my own projects because of how ergonomics suffer. The analyzer still has issues on Windows when external code updates Dart files, for example, and the edit-refresh cycle for creating your own build steps is sub-par.
Annotations on typedefs? It's more likely than you think!
@eernst Erik, where are we with filling in the annotation holes in the grammar?
For the latest update on previewing this feature, please see the updated top-most comment: #34452 (comment)
For feedback, questions, and issues regarding the feature, kindly do not post them here in the issue tracker. Please post on the dart-ffi group: https://groups.google.com/forum/#!forum/dart-ffi
Wanna follow up whether we can expect iOS support by Dec 2019?
Wanna follow up whether we can expect iOS support by Dec 2019?
We hope to have that in the preview fairly soon. Please join the email group mentioned in #34452 (comment) to get updates when that happens.
@Kleak
We've added a new feature to the Pointer api, Pointer.getExternalTypedData. This creates an external TypedData array backed by the C memory from which reads/writes are very efficient.
I modified your benchmark to use this feature, and got the following results:
Before any modifications
The results are similar to what you saw:
LibSodiumSHA256: FooBar
LibSodiumSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
LibSodiumSHA256(RunTime): 225.35954929577466 us.
LibSodiumSHA512: FooBar
LibSodiumSHA512: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
LibSodiumSHA512(RunTime): 420.7172907025663 us.
LibSodiumSHA512FFIOnly: FooBar
LibSodiumSHA512FFIOnly: Pointer: address=0x55c12c900c80
LibSodiumSHA512FFIOnly(RunTime): 9.616287064683792 us.
After using getExternalTypedData in CArray.from and CArray.bytes
LibSodiumSHA256: FooBar
LibSodiumSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
LibSodiumSHA256(RunTime): 20.373880711047725 us.
LibSodiumSHA512: FooBar
LibSodiumSHA512: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
LibSodiumSHA512(RunTime): 22.924955869879188 us.
LibSodiumSHA512FFIOnly: FooBar
LibSodiumSHA512FFIOnly: Pointer: address=0x55d2a2cd7200
LibSodiumSHA512FFIOnly(RunTime): 9.691031461839255 us
After using getExternalTypedData and caching the allocated arrays
This optimization should not be needed after we optimize calls to Pointer.allocate and Pointer.getExternalTypedData itself.
LibSodiumSHA256: FooBar
LibSodiumSHA256: 0d749abe1377573493e0df74df8d1282e46967754a1ebc7cc6323923a788ad5c
LibSodiumSHA256(RunTime): 7.746091132671043 us.
LibSodiumSHA512: FooBar
LibSodiumSHA512: 466f6f426172
LibSodiumSHA512(RunTime): 7.551476501704745 us.
LibSodiumSHA512FFIOnly: FooBar
LibSodiumSHA512FFIOnly: 0d6fc00052fee1d3ae1880d5dc59b74dd50449ffdea399449223d2c5b792395ce64153b150fc0fc01bfeed30c7347411cfb8a3b17b51fd8aa6c03acfbcd09e7b
LibSodiumSHA512FFIOnly(RunTime): 6.1046642593988745 us.
I also switched CArray.bytes to use List.from instead of List.unmodifiable because List.unmodifiable triggers a runtime call.
Result
The final performance (which can be achieved with the FFI today) is more than 2x faster than both PointerCastle and Crypto.
hey @sjindel-google ! Pretty nice optimization !! Do you mind make a PR of your modifications on the repo ? We would like to integrate those optimizations and as you already have done the migration that would be awesome ^^
What's the plan in releasing ffi on flutter? Should I stick to platform channel or should I investigate in exporting native extension myself or just wait for ffi. DESPERATE for calling cpp from dart, these is too much existing code in cpp to rewrite.