Representation of function pointer types in the reflection stack
Closed this issue · 25 comments
dotnet/csharplang#1951 discusses adding function pointer types to the C# language. The CLR has supported these for a while, but there are areas (such as the reflection stack), where function pointer types are weird/unhandled.
While this will just be a corner area of the C# language, we should probably at least track this:
- Can we fix
ldtoken method explicit instance int32 *(int32)not to return the token ofSystem.IntPtr? - Can we use
System.Reflection.Pointerto reflection invoke methods that take/return function pointers? - Do we need new APIs to inspect/reflection emit function pointer types?
@MichalStrehovsky marked future.
Do you know off the top of your head how complicated it is to do the first one?
Do you know off the top of your head how complicated it is to do the first one?
No, I didn't look into it beyond trying that single line and being happy that I didn't have to add any extra work to .NET Native/CoreRT backlog at that time.
Sample syntax below - here's a method that takes a function pointer of type int -> void as an argument.
public unsafe static void M(delegate*<int, void> ptr) {
}We should do this for 5.0 since function pointers support is being added imminently.
I hope the "we should do this for 5.0" still applies and we're tracking this as part of the overall function pointers work.
At the moment it's not currently funded for 5.0 - it's more of a stretch goal. If it's blocking you let us know and we can mark it as such. Then we can pull resources from other things to address it.
Yeah this is pretty impactful since I'm wanting to freeze these objects and it is not possible to find which are fnptrs arrays. I'm ok if I can work around it somehow but I don't think I'm able to.
@GrabYourPitchforks I see you self-assigned this, does this mean it'll happen in 5.0?!
Still not funded, but I'm growing increasingly worried that if we don't address it in 5.0 it'll be too difficult to address in 6.0 due to compat. So going to see if we can slam this in at the last minute. It'll definitely be a roller coaster. :)
Now that 5.0 has released, and I'm switching my functions that used to take IntPtr to function pointers, I'm running into this issue.
I have the following interface (simplified of course)
public interface IThing {
void Function(delegate* unmanaged[Cdecl]<int, void> cb); // This used to be an IntPtr cb
}
I'm generating at runtime using Reflection.Emit an implementation of this interface. However, with function pointers, this does not seem possible. When I iterate the parameters of that function, the parameter just shows up as an IntPtr, with now way to detect its a function pointer. If I generate the function using IntPtr, the signature doesn't match, and the importer fails with a TypeLoadException basically that the interface is not implemented.
@MichalStrehovsky @davidwrighton I've been looking into the roslyn side of this, namely how to encode typeof(delegate*<void>) in an attribute argument (dotnet/roslyn#48765). I'm concerned that we have an ambiguity problem with the encoding of types in such attributes that may take a runtime rev to address. 334 II.23.3 defines types as being serialized as follows:
If the parameter kind is System.Type, (also, the middle line in above diagram) its value is stored as a SerString (as defined in the previous paragraph), representing its canonical name. The canonical name is its full type name, followed optionally by the assembly where it is defined, its version, culture and public -key-token. If the assembly name is omitted, the CLI looks first in the current assembly, and then in the system library (mscorlib); in these two special cases, it is permitted to omit the assembly-name, version, culture and public-key-token.
The issue is that, in this format, there are no keywords, and there is no escaping of type names that are keywords, whereas the metadata format for function pointers uses a number of keywords. For example, take this program (assume that Attr is an attribute that has one Type argument):
class method {}
[Attr(typeof(method))]
class C {}Here, the attribute argument is serialized as just method, with no escapes or other content. If we need to serialize function pointers in their metadata format, this suddenly becomes ambiguous: is it a reference to the type method, or is it a malformed function pointer definition? This becomes even more insidious if you consider the following program:
class cdecl {}
[Attr(typeof(delegate* unmanaged<cdecl>))]
class C {}Without escapes, that would get serialized as method unmanaged cdecl*(), which is completely incomprehensible.
It feels to me like we might need to define a new serialization format for attribute typeofs in order to deal with these issues, potentially communicated on both the part of an individual assembly and on the part of the runtime. IE, the runtime adds a new flag for "supports escapes in typeof", and we add an attribute to a module when it's compiled in the new format, so that reflection and the compiler can both correctly understand the format of attribute type names.
#4416 is the current documentation for the SerString format used in custom attributes. It allows some escaping, but things will definitely get very interesting with modifiers. The issue mentions function pointers and modifiers ("For example, function pointers or modifiers? What does C++/CLI even persist when I pass long::typeid or (const int*)::typeid as the value of a fixed or named argument? Should we disallow them?") so it would be worth checking whether C++/CLI already solved the problem for us.
If we were to rev the format, I always wondered why we had SerString in the first place. Whether we could get away with putting a token (TypeDef/Ref/Spec) here instead. It would also be much smaller.
The custom attribute blobs version separately from the whole file format (they're all version 1 right now) so we could potentially bump the version there whenever we absolutely need the higher version. Tools might be able to ignore attribute blobs with format version they don't recognize, but we'll have to try that.
so it would be worth checking whether C++/CLI already solved the problem for us.
It didn't solve. The code like this:
using namespace System;
typedef int(*PFUNC)(int);
public ref class MyAttrAttribute : public Attribute {
public:
MyAttrAttribute(Type^ t) {
}
};
[MyAttr(PFUNC::typeid)]
public ref class TestClass {
};
is compiled into the following IL:
.custom instance void MyAttrAttribute::.ctor(class [mscorlib]System.Type) = ( 01 00 5B 53 79 73 74 65 6D 2E 49 6E 74 50 74 72 // ..[System.IntPtr
2A 2C 20 6D 73 63 6F 72 6C 69 62 2C 20 56 65 72 // *, mscorlib, Ver
73 69 6F 6E 3D 34 2E 30 2E 30 2E 30 2C 20 43 75 // sion=4.0.0.0, Cu
6C 74 75 72 65 3D 6E 65 75 74 72 61 6C 2C 20 50 // lture=neutral, P
75 62 6C 69 63 4B 65 79 54 6F 6B 65 6E 3D 62 37 // ublicKeyToken=b7
The C++/CLI compiler just substitutes function pointers with IntPtr types for attribute arguments
How the reflection API to represent function pointers information should look like? I think there are two options:
- Create new
Signaturetype, that would contain information about calling convention, return type and parameter types - Use the existing
MethodInfotype, and return the special dummy instances that would support CallingConvention, GetParameters and ReturnType, but have any other members no-op or throwing NotSupportedException.
The first looks more pleasant, but with second more existing code operating on MethodInfo could be reused.
One immediate problem with calling conventions is that we decided to encode them as modifiers going forward, but modifiers tend to disappear as reflection stack handles them.
In particular:
runtime/src/tests/reflection/ldtoken/modifiers.il
Lines 12 to 19 in ead035b
They disappear when we do LDTOKEN.
We'll need some special rules if we want to preserve that in typeof for function pointers. We are going to break compat in that direction anyway (since they're all loaded as IntPtr right now), but there might be other places that rely on modopts disappearing.
CoreCLR and Mono reflection stacks represent function pointers differently today. For examle, Console.WriteLine(typeof(delegate*<int>).FullName) prints System.IntPtr on CoreCLR vs. System.MonoFNPtrFakeClass on Mono. That's also something that has to be unified to fix this issue.
Moving to 8.0; we need to update the API and design based on feedback from #71516.
FWIW the jit currently mis-optimizes some cases involving function type comparison.
The fix is in #72136 where the jit will defer to the runtime to decide what sort of optimizations can apply. If/when the runtime type for function pointers changes those new bits of jit interface can potentially be revised to enable a little more optimization.
System.InvalidCastException: Unable to cast object of type 'ILCompiler.DependencyAnalysis.ConstructedEETypeNode' to type 'Internal.TypeSystem.TypeDesc'.
This error means that the result of embed{Generic}Handle is being used CORINFO_CLASS_HANDLE. You can get away with this with JIT since the two are identical pointers, but you cannot get away with this in AOT where the two are different pointers: One is a type handle and the other is a symbol in the AOT image.
[Above is for the failure in #72136] -- looks like the jit IR holds onto the "compile time handle" but VN doesn't know how to get to this from the embedded handle. Think I can just build a map.
What's the status of this? I hope Type.GetFunctionPointerReturnType, Type.GetFunctionPointerParameterTypes and Type.MakeFunctionPointerType will finally be possible, so that the full CLI type system can finally be expressible using reflection and emit (well, except for modopts and modreqs I guess). I hope Pointer.Box will work too!
What's the status of this?
Currently a design is being created. The reflection features for 8.0 are tracked here.
I hope Pointer.Box will work too!
This will be covered in the design. Here's the current section:
System.Pointer
The System.Reflection.Pointer class is useful to pass a pointer address and type together and was introduced to pass a pointer using reflection APIs which are based on passing arrays of System.Object. Essentially it is a Tuple<Type, IntPtr> although currently Pointer does not expose a property to obtain the Type.
There is no proposal here to either extend Pointer or add a new FunctionPointer. If code wants to pass a function pointer with reflection, it will have to pass both the Type and IntPtr separately. If the need to do so becomes more obvious, a new FunctionPointer type should be a value type, not a reference type like Pointer, since newer reflection APIs will not force boxing to occur.
The design is ready for review at dotnet/designs#282
Closing; implemented in #81006