Tracking issue for Fn traits (`unboxed_closures` & `fn_traits` feature)
aturon opened this issue · 66 comments
Inability to delegate calls to other FnOnce implementors like this:
struct A<T>(T);
impl<T, Args> FnOnce<Args> for A<T>
where T: FnOnce<Args> {
type Output = <T as FnOnce<Args>>::Output;
fn call_once(self, args: Args) -> Self::Output { FnOnce::call_once(self.0, args) }
}is the reason I have a safety issue in libloading.
FnOnce::Output is stabilized in #34365
Could someone summarise what is blocking stabilisation here please?
@nrc uncertainty about Args being the right thing given the possibility of variadic generics coming along around 2030 is the most major reason this is unstable.
There should be a way of casting fn(A) -> B to Fn(A) -> B
@brunoczim That coercion already happens implicitly, but Fn() as a type is a trait object so it needs to be behind some kind of pointer:
use std::rc::Rc;
fn main() {
let _: &Fn() = &main;
let _: Box<Fn()> = Box::new(main);
let _: Rc<Fn()> = Rc::new(main);
}One issue we've been discussing on TheDan64/inkwell#5 is the ability to mark something as unsafe to use. What about having an UnsafeFn marker trait which could be used to tell the compiler a callable is unsafe to call?
For context, inkwell is a wrapper around LLVM and we're trying to figure out how to return a function pointer to a JIT compiled function, when calling the function is fundamentally unsafe for the same reasons FFI code is unsafe to call.
There is another reason to add UnsafeFn: currently unsafe fns don't implement Fn. There is no way to pass unsafe fn as a function argument, they are second class citizen: example.
UnsafeFn could be implemented for things implementing Fn, so Fns can be used wherever UnsafeFn is required. There probably also should be UnsafeFnMut and UnsafeFnOnce.
@Michael-F-Bryan @CodeSandwich This sounds like something for which an RFC would really be appreciated. It probably wouldn't be an overly long or intricate one to write, even. I would support it, for sure, and judging by an issue I saw about this not long ago (a long-standing one), others would too.
@alexreg Ok, I'll prepare it in spare time. Unfortunately I lack knowledge and experience to actually implement it or even fully understand the idea.
@CodeSandwich Don't worry, so do I! Maybe get yourself on Rust's Discord (#design channel) and we can discuss it with some people who really know the nitty gritty? You can ping me there, same username.
With every unsafe function comes a manually written "contract" saying when that function may or may not be called.
With UnsafeFn, who is setting that contract?
With
UnsafeFn, who is setting that contract?
I'd say this is done on a case-by-case basis. It's really hard to specify the various invariants and assumptions you make in unsafe code in something as rigid as a type system, so anything that accepts an UnsafeFn would probably also need to document its assumptions.
Before writing up a RFC I thought I'd make a post on the internal forum. It'd be nice to hear what opinions other people have on this topic and the different solutions they come up with.
I don't know how one would realistically implement this, but probably the ideal semantics is to have any function that takes an unsafe function as an argument to also be unsafe.
A different interface/API likely requires a separate set of traits. Though multiplying the number of traits doesn’t sound great, especially if we later want to also support const fn closures (and unsafe const fn?).
Is there a way to implement these as wrapper traits? Unsafe<T> where T : FnOnce, Const<T> where T : FnOnce, Async<T> where T : FnOnce and so on?
UnsafeFn sounds like an unsound abstraction. When using an unsafe function, it's your responsibility to read documentation and understand all assumptions and invariants that need to be satisfied. This cannot be abstracted away because those invariants and assumptions are individual to the function in question.
Personally, I think unsafe functions should just not implement any of the Fn traits. It's up to the caller to wrap the unsafe function in another function in order to pass it around. Unsafe code doesn't need to meet the same standards of ergonomics as safe code, provided that it's still possible to get optimal performance.
For example, CodeSandwich's example can be "fixed" using a closure: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2015&gist=308c3ee61419427c617879fc4cb82738
I agree. UnsafeFn is meaningless and breaks the mechanics of unsafe proof obligations.
I understand the argument against Unsafe Fn. But wouldn't this apply to unsafe fn pointers too?
I understand the argument against
Unsafe Fn. But wouldn't this apply tounsafe fnpointers too?
Yes, I would say so.
what does the difference between Fn and fn?
@jinfagang cc https://doc.rust-lang.org/std/ops/trait.Fn.html
What's the status of this? There are a number of F-unboxed_closures issues but this tracking issue doesn't appear to be... well, tracking the issue. I'm currently working with a project that requires erased closure types and supports #![no_std] and SmallBox-style tricks don't really help when I'm writing library code that lacks any information about the maximum size of the closure state.
Is it intentional that the code below compiles? It looks wrong to me, as it appears to move out of a mutable reference.
#![feature(fn_traits)]
#![feature(unboxed_closures)]
pub struct A {}
impl std::ops::FnMut<()> for A {
extern "rust-call" fn call_mut(&mut self, args: ()) { self.call_once(args) }
}
impl std::ops::FnOnce<()> for A {
type Output = ();
extern "rust-call" fn call_once(self, _args: ()) { }
}
For comparison, this doesn't compile:
struct A { }
impl A {
pub fn my_mut(&mut self) { self.my_once() }
pub fn my_once(self) { }
}
@rcls I think that's not calling your FnOnce, but rather the blanket impl<F: FnMut> FnOnce for &mut F that uses call_mut, so you'll have infinite recursion.
We definitely want to stabilize the Fn family of traits at some point, allowing people to impl them.
Marking this as "design-concerns" because we need to determine if we should wait for variadic generics or stabilize the tuple-based rust-call ABI.
Should Args be turned into an associated type to prevent people from using it to implement operator overloading based on argument type?
no
@bjorn3 wouldn't that break higher-order signatures (which are implemented as overloads over the input lifetimes)? FWIW, this question is related to that of the Resume parameter for Generators.
a silly thing 🙈
trait FnOnce<'lifetimes..> { // variadic lifetime-generics?
type Args;
type Output;
extern "rust-call"
fn call_once (self, _: Self::Args)
-> Self::Output
where
Self : Sized,
;
}wouldn't that break higher-order signatures (which are implemented as overloads over the input lifetimes)?
Right, didn't think about that.
Silly question, but is there any particular reason why the Fn* traits actually need to stabilise the rust-call ABI in order to be implementable?
I mean, we already have the custom Fn(A, B, ...) -> C syntax as sugar for Fn<(A, B,), Output = C>, so, I don't think it'd be unreasonable to adopt a special syntax just for implementing them too.
Maybe something like:
struct MyFn(u32);
impl MyFn {
fn(self, x: u32, y: u32) -> u32 {
x + y + self.0
}
}Could get desugared to:
struct MyFn(u32);
impl FnOnce<(u32, u32)> for MyFn {
type Output = u32;
extern "rust-call" fn call_once(self, (x, y): (u32, u32)) -> u32 {
/* body */
}
}@clarfonthey I think there's some needs to implement Fn* traits for arbitrary types.
- Giving multiple function signatures to an object.
- In following URL, there's some difficulty defining trait
Handler, so we want to use Fn* trait directly.
https://users.rust-lang.org/t/type-inference-in-closures/78399
Giving multiple function signatures to an object.
This is something I think we shouldn't support in the first place, even with Fn*.
In following URL, there's some difficulty defining trait Handler, so we want to use Fn* trait directly. https://users.rust-lang.org/t/type-inference-in-closures/78399
If I understand it correctly, preventing multiple impls of Fn* would fix this issue.
@bjorn3
Thanks for replying.
we shouldn't support in the first place
preventing multiple impls of Fn*
Do you have any standings for preventing higher-order signaitures as mentioned below?
#29625 (comment)
I don't know how to support higher-order signatures while at the same time preventing multiple call signatures for a type other than keeping Fn* perma-unstable.
@peterjoel but that wouldn't scale to function-like objects with generic arguments, for which an associated type wouldn't work. What we really want to prevent here are functions with multiple simultaneous argument counts (because varying argument types is already possible, although a bit complicated, using sealed traits), which could be solved with 2 additional traits (one for tuples, one for functions, which binds tuples to the count of directly contained objects, and functions to their argument counts, either via type-level integers, or using const generics + associated constants)
but that wouldn't scale to function-like objects with generic arguments, for which an associated type wouldn't work.
Closures can't be generic either.
@zseri
What we really want to prevent here are functions with multiple simultaneous argument counts
Is this really harmful? it looks fine in current unstabilized fn-traits:
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=157729b98808b439ecc992e4ba59273e
It may be fine from a technical perspective, but from a language perspective we have chosen to not allow function overloading. See for example https://internals.rust-lang.org/t/justification-for-rust-not-supporting-function-overloading-directly/7012, https://users.rust-lang.org/t/is-it-possible-to-specialize-hashmap-index-for-copy-types/7750/5, https://users.rust-lang.org/t/what-is-the-reason-for-not-having-overloaded-versions-of-fn/55208 and https://users.rust-lang.org/t/is-there-a-simple-way-to-overload-functions/30937.
I recently did some experiments with function traits that requires the use of trait specialization to get it working.
https://github.com/nyxtom/composition/blob/main/src/lib.rs
pub trait Func<Args, T> {
type Output;
fn call(&self, args: Args) -> Self::Output;
}
// Default implementation of a func for T as output
impl<A, B, Args, T> Func<Args, ()> for (A, B)
where
A: Fn<Args, Output = T>,
B: Fn<T>,
{
type Output = B::Output;
#[inline]
fn call(&self, args: Args) -> Self::Output {
let args = self.0.call(args);
self.1.call(args)
}
}
// Subset of (A, B) T is (T,)
impl<A, B, Args, T> Func<Args, (T,)> for (A, B)
where
A: Fn<Args, Output = T>,
B: Fn<(T,)>,
{
type Output = B::Output;
#[inline]
fn call(&self, args: Args) -> Self::Output {
let args = self.0.call(args);
self.1.call((args,))
}
}Specifically it allowed me to compose between two functions that is already a tuple being returned and apply them as arguments, or in the natural case where the return value is a single type.
fn foo() {}
fn test() -> i32 {
3
}
fn plus(a: i32) -> i32 {
a + 1
}
fn multiply(a: i32, b: i32) -> i32 {
a * b
}
fn output() -> (i32, i32) {
(4, 2)
}
fn assert_func<Args, T>(_: impl Func<Args, T>) {}
#[test]
fn test_assert_funcs() {
assert_func((foo, foo));
assert_func((foo, test));
assert_func((test, plus));
assert_func((plus, plus));
assert_func((multiply, plus));
assert_func((output, multiply));
}Does this constitute function overloading or just a use of trait specialization? I did later expand on this to support more composition (than 1 argument) by using the recursive tuple structure like so:
// Subset of (A, B) where A is already a tuple that implements Func
impl<A, B, Args, T, F> Func<Args, ((), (), F)> for (A, B)
where
A: Fn<Args, Output = T>,
B: Func<T, F>,
{
type Output = B::Output;
#[inline]
fn call(&self, args: Args) -> Self::Output {
let args = self.0.call(args);
self.1.call(args)
}
}
// Subset of (A, B) where is A is Func and B takes (T,)
impl<A, B, Args, T, F> Func<Args, ((), (T,), F)> for (A, B)
where
A: Fn<Args, Output = T>,
B: Func<(T,), F>,
{
type Output = B::Output;
#[inline]
fn call(&self, args: Args) -> Self::Output {
let args = self.0.call(args);
self.1.call((args,))
}
}This allowed me to perform expressions like so:
#[test]
fn test_assert_nested_func() {
assert_func((multiply, (plus, plus)));
assert_func((plus, (plus, plus)));
assert_func((output, (multiply, plus)));
}This does require the use of the Fn<T> rather than the parenthetical notation but it's quite useful here as it allows some nice composition to happen. As well, you can make use of variadics with just a macro that turns it into a recursive structure.
pub struct Function<F, T>(F, PhantomData<T>);
impl<F, Args, T> Fn<Args> for Function<F, T>
where
F: Func<Args, T>,
{
extern "rust-call" fn call(&self, args: Args) -> Self::Output {
self.0.call(args)
}
}
impl<F, Args, T> FnMut<Args> for Function<F, T>
where
F: Func<Args, T>,
{
extern "rust-call" fn call_mut(&mut self, args: Args) -> Self::Output {
self.0.call(args)
}
}
impl<F, Args, T> FnOnce<Args> for Function<F, T>
where
F: Func<Args, T>,
{
type Output = F::Output;
extern "rust-call" fn call_once(self, args: Args) -> Self::Output {
self.0.call(args)
}
}
macro_rules! compose {
( $last:expr ) => { $last };
( $head:expr, $($tail:expr), +) => {
($head, compose!($($tail),+))
};
}
macro_rules! func {
( $head:expr, $($tail:expr), +) => {
Function(($head, compose!($($tail),+)), PhantomData::default())
};
}
fn assert_fn<Args>(_: impl Fn<Args>) {}
#[test]
fn test_assert_fn() {
assert_fn(func!(output, multiply, plus));
}With function composition I can guarantee that the composition of func!(A, B, C) is the type safe equivalent of the input to A and output of C. The main thing here that makes it easier is having Fn<T> rather than Fn(A). As without the generics argument, I end up having to create an entirely different macro that implements these cases for Fn(A), Fn(A, B), Fn(A, B, C), ...etc.
I should note that the above example doesn't necessarily require the use of impl<Args, T> Fn<Args> for Function<Args, T> it just makes it easier to do:
compose!(plus, plus, plus).call((4,));vs
func!(plus, plus, plus)(4)@bjorn3 Thanks very much, but I cannot understand the difference of higher-order function arguments and arbitrary function signatures(function overloading) clearly.
Thinking of this example, what is the main factor to distinguish the two concepts?
Is this the correct way forcing not to use arbitrary function signatures by language design?
Higher-order function arguments is when the function arguments only differ by lifetimes.
@bjorn3 If so, the following is different signatures?
fn f<T: AsRef<[u8]>>(t: T) { unimplemented!() }
f("hello");
f("hello".to_owned());
f(vec![1,2,3]);"hello".to_owned() and vec![1,2,3] have the same lifetime (they are stack-allocated). &'static T is special because it is valid for every lifetime.
if by stack you mean heap, yes
They both happen to point to the heap, but the structs themselves are stack-allocated when used in the argument position like that. Or when assigned to a variable with a let binding, for instance.
Anyway, the fact that people can already make up their own trait based operator overloading, and even paper over the arity issue with a do_call! macro or whatever, means that there's not much reason to make the Fn traits themselves specifically and magically reject the possibility of overloading. We're just giving people a hard time.
I found tour first reference link as to why not function overloading most interesting because the second reply is from an actual T-lang member that said:
the desire to not have monomorphization-time errors in generics.
and if overloading is happening strictly through the trait system it should end up preventing the post-monomorph errors.
Anyway, the fact that people can already make up their own trait based operator overloading, and even paper over the arity issue with a do_call! macro or whatever, means that there's not much reason to make the Fn traits themselves specifically and magically reject the possibility of overloading. We're just giving people a hard time.
I found tour first reference link as to why not function overloading most interesting because the second reply is from an actual T-lang member that said:
the desire to not have monomorphization-time errors in generics.
and if overloading is happening strictly through the trait system it should end up preventing the post-monomorph errors.
This has been my experience (as I've seen implemented in other places). One macro to implement arity to fix the Fn(A), Fn(A, B)..etc and another to be able to call with do_call!(A, B, C). Having the generics on the actual trait here Fn<Args> makes this less difficult to work with and avoids the extra macro expansion. That being said, it's not strictly required to have an impl Fn<Args> for F since you can do this with the trait based approach but the ability to use the extern "rust-call" hack to turn a struct into a function call is a bit different.
I agree with the fact you shouldn't be able to have multiple Fn* impl on an item.
This is why I wonder why Args are generic and not a associated type
Would it be helpful if I made a PR to move from generic to associated, just to see how it would work?
Would it be helpful if I made a PR to move from generic to associated, just to see how it would work?
I think it would be helpful, but there would need to be some work inside rustc (don't quote me on that) since Fn* traits are really bypassing the whole "how do we represent arguments" by using a custom syntax.
Tho it would allow you to get the argument out of an Fn* trait type (i don't think it is an issue, and it could be gated behind a perma-unstable flag if we really don't want that to happen).
There is more work to be done, but I believe that we should make this change. Currently it is an non-issue since you can't implement these trait, but with unboxed closure it would allow you to have some kind of function overload because you can implement a generic trait multiple times just with different generics.
I'm pretty sure that the fact it is generic can already be easily be relied upon, and is relied upon in practice. I can try to cook up am example soon.
edit: No capacity for this, sorry.
Would it be helpful if I made a PR to move from generic to associated, just to see how it would work?
I don't think this can work; if Args were an associated type, function HRTBs wouldn't be possible (e.g. for<'a> Fn(&'a [i32]) -> &'a i32).
I just come here for I want some code like this:
let x = CustomizeStruct;
let y = x(); // direct call on instanceAnd seems that it has to do with code like this:
#![feature(unboxed_closures)]
#![feature(fn_traits)]
struct CustomizeStruct;
impl Fn<()> for CustomizeStruct {
extern "rust-call" fn call(&self, _args: ()) {
println!("call CustomizeStruct");
}
}
impl FnMut<()> for CustomizeStruct {
extern "rust-call" fn call_mut(&mut self, _args: ()) {
println!("call CustomizeStruct");
}
}
impl FnOnce<()> for CustomizeStruct {
type Output = ();
extern "rust-call" fn call_once(self, _args: ()) {
println!("call CustomizeStruct");
}
}But due to the instability of the 2 features, which led to me here, then I have to worke around by using Deref:
use std::ops::Deref;
struct Tensor {
value: i32,
name: String,
}
impl Tensor {
fn new(value: i32, name: &str) -> Self {
Tensor {
value,
name: name.to_string(),
}
}
}
struct CustomizeStruct {
closure: Box<dyn Fn(&Tensor) -> i32>,
}
impl CustomizeStruct {
fn new() -> Self {
CustomizeStruct {
closure: Box::new(|tensor: &Tensor| {
println!("call CustomizeStruct");
println!("Tensor name: {}", tensor.name);
tensor.value * 2
}),
}
}
}
impl Deref for CustomizeStruct {
type Target = dyn Fn(&Tensor) -> i32;
fn deref(&self) -> &Self::Target {
&*self.closure
}
}
fn main() {
let x = CustomizeStruct::new();
let tensor = Tensor::new(21, "example tensor");
let y: i32 = x(&tensor);
println!("y = {}", y);
}The code above is compilable on v1.72.0.
I have reread this issue and the following issues seem to have been raised as in some sense blocking:
- Stabilising
extern "rust-call". - Open questions about the API of
Args, generic variadic despatch, etc. - Should there be
UnsafeFn*? Answer: no. (not a blocker, therefore) impl FnOnce for &Tcall syntax #42736 apparently due to lack of autoref?- Open questions about the relationship of the blanket impls for these Fn traits (currently there aren't any, but they existed at some point AFAICT).
1, 2 and 5 could be dealt with by desugaring along these lines:
/// plan to stabilise this:
impl FnMut(i32) for F {
fn call_mut(&mut self, x: i32) { dbg!(x); }
}
// desugars to:
impl FnMut<(i32,)> for F {
extern "rust-call" fn call_mut(&mut self, (x,): (i32,)) -> Self::Output {
dbg!(x);
}
}
impl FnOnce<(i32,)> for F {
type Output = ();
extern "rust-call" fn call_once(mut self, args: (i32,)) -> Self::Output {
FnMut::call_mut(&mut self, args)
}
}
// Correspondingly:
// impl Fn(..) => impl Fn<>, impl FnMut<>, impl FnOnce<>
// impl FnOnce(..) => impl FnOnce<> (only)
This has the following properties:
- Preserves opacity, and expansion/change possibilities, for the
Fntraits - Does not expose
"rust-call"or the type ofArgs - Makes
impl Fn*magic - which is OK because these are already magic - Prevents a manual implementor providing both (say)
FnMutandFn, but I think preserving ability to make this possible via specialisation later - Unblocks most of the obvious use cases (including wrappers for functions)
Realistically,it seems to me that there is little else that we could want that impl FnMut to mean in the future.
This disposes of all the blockers except 4, #42736, which is a despatch anti-affordance when you impl FnOnce for &T (or similar). I hope #42736 isn't actually a blocker ?
What am I missing?
Open questions about the relationship of the blanket impls for these Fn traits (currently there aren't any, but they existed at some point AFAICT).
Isn't that referring to the impls on references and Box? (both #[fundamental])
Isn't that referring to the impls on references and
Box? (both#[fundamental])
I was referring to comments like #29625 (comment) (references #19032). That's about the relationship between Fn, FnMut and FnOnce. AFAICT before #23282 there were some blanket impls.
I don't think there is any issue with references or Box, that applies to the Fn* traits, but only when the trait(s) are manually implemented?
I had a go at implementing this:
/// plan to stabilise this: impl FnMut(i32) for F { fn call_mut(&mut self, x: i32) { dbg!(x); } } // desugars to: impl FnMut<(i32,)> for F { extern "rust-call" fn call_mut(&mut self, (x,): (i32,)) -> Self::Output { dbg!(x); } } impl FnOnce<(i32,)> for F { type Output = (); extern "rust-call" fn call_once(mut self, args: (i32,)) -> Self::Output { FnMut::call_mut(&mut self, args) } }
But I encountered a difficulty. (I'm not very familiar with the compiler innards, so possibly I'm just going about it entirely the wrong way.)
I was proposing this as a desugaring, and so I think probably this wants to be done during AST lowering. I think the right place to do this would be in compiler/rustc_ast_lowering/src/item.rs, lower_item_kind. That's the last place where the information needed to do this transformation all exists together.
But lower_item_kind only gets to produce one output hir::ItemKind and my proposal calls for multiple impls. I wasn't sure how (or whether) to try to give lower_item_kind the ability to produce multiple output items.
Turning one item into many raises a question about what ought to be done about attributes applied to the user-supplied impl block. I think they may need to be copied, since pieces of the input end up in more than one of the outputs, and we might want the user's attributes to affect type Output= as well as fn call_*. Maybe this is a reason this shouldn't be done?
I had a go at inventing a helper trait instead: ie, the lowering would implement not the normal FnMut (say) but helper::FnMut and a blanket impl would provide ops::FnMut. But my blanket impls conflicted with other blanket impls for &impl Fn for example.
Maybe someone else can get this to work or give me some pointers.
I skimmed this thread so I'm sorry if I missed something, but why is there a difference between the expression of Output and Args? I get that Args has to be a parameter to encode generics, but functions can have generic parameters that only appear in their return type. In fact, std::sync::mpsc::channel does exactly that:
fn channel<T>() -> (Sender<T>, Receiver<T>);This type of generic is fairly common; a trick I use to describe infallible results in a way that is compatible with whatever the user wants to do is to template on the unconstrained error:
fn this_never_fails<E>() -> Result<(), E>Why isn't Output also a generic parameter?
Edit: it somehow elided me that Output was already standardized. I guess these traits are just not equivalent to function definitions then.
@lbfalvy You can think of fn items as syntactic sugar for of a struct and some impls. For example fn example(arg1: u32, arg2: bool) -> String {…} becomes:
struct example;
impl FnOnce<(u32, bool)> for example {
type Output = String;
extern "rust-call" fn call_once(self, args: (u32, bool)) -> String {…}
}Now if the function itself is generic like fn channel<T>() -> (Sender<T>, Receiver<T>) {…}, the equivalent expansion would have that T be a parameter of the struct. Then the impl can reference it without any problem when defining the associated type:
struct channel<T>(PhantomData<T>);
impl<T> FnOnce<()> for channel<T> {
type Output = (Sender<T>, Receiver<T>);
extern "rust-call" fn call_once(self, args: ()) -> (Sender<T>, Receiver<T>) {…}
}@SimonSapin I see, but why can't the same technique be used to make Args an associated type too?
@SimonSapin I see, but why can't the same technique be used to make Args an associated type too?
I suppose because there's no varadic associated type support?
You would have to use tuple and it'd make things complicated.
I believe it is required to handle higher-ranked types. Don't recall exactly why though.
I believe this was also waiting on variadic generics, which are waiting on the type system overhaul.