Feature: Statically checked declaration-site variance

Question

Feature: Statically checked declaration-site variance

eernstg opened this issue 5 years ago · 93 comments

This is the tracking issue for introducing a statically checked mechanism for declaration-site variance in Dart.

Background: The original request motivating this kind of feature is dart-lang/sdk#213; the initial proposal for declaration-site invariance is dart-lang/sdk#213. The initial proposal for the related feature known as use-site invariance is dart-lang/sdk#229, and the corresponding tracking issue is dart-lang/sdk#753.

Note that this issue does come up in practice. Here are a few examples gathered since April 2023:

The text below describes properties of this feature which are good candidates for being adopted. Many things can still change, and a full feature specification will be written and used to manage the discussions about the final design.

Variance in Dart Today

As of Dart 2.4 or earlier, every type variable declared for a generic class is considered covariant. The core meaning of this is that a parameterized type C<T2> is a subtype of C<T1> whenever T2 is a subtype of T1. Other subtype rules can then be used to show subtype relationships like List<int> <: Iterable<dynamic> and Map<String, String> <: Map<Object, Object> <: dynamic.

This type rule is not sound; that is, in order to maintain heap soundness it is necessary to check certain types dynamically. This means that a program with no compile-time errors can fail with a type error at run time.

For instance, with the declaration List<num> xs and some expression e with static type num, it is necessary to check during evaluation of xs.add(e) that the value of e actually has the type which is required by xs: It is possible that it is a List<int> or even a List<Never>, and it would then be a dynamic type error if the value of e is a double, even though the expression had no type errors at compile-time.

Dynamically checked covariance enables many software designs that would be rejected by a traditional statically checked approach to variance (e.g., as in Java or C#). This allows developers to make a trade-off between more flexible types (e.g., a variable of type List<num> is allowed to refer to a List<int>) in return for accepting the potential dynamic type errors (a List<int> will work safely under the type List<num> in a lot of ways, just not all).

We want to enable a statically checked typing discipline for variance as well (rejecting more programs, but providing a compile-time guarantee against the run-time type errors described above). This feature is concerned with the provision of support for that.

Declaration-site Variance

Declaration-site variance can be used to declare a strict and statically checked treatment of variance for each type variable of a generic class.

Syntactically, declaration-site variance consists in allowing each type parameter declaration of a generic class declaration to include one of the following modifiers: out, in, or inout. We say that such a type parameter has explicit variance.

The use of type parameters with explicit variance in the body of the enclosing class is restricted. It is a compile-time error for a type variable marked out to occur in a non-covariant position in the signature of a member declaration; and for a type variable marked in to occur in a non-contravariant position. For example:

abstract class Good<out X, in Y, inout Z> {
  X get m1;
  void set m2(Y value);
  Z m3<U extends Z>(List<Z> zs);
}

class Bad<out X, in Y> {
  Y get m1; // Error.
  void set m2(X value); // Error.
  Y m3<U1 extends X, U2 extends Y>(List<X> xs); // Error.
}

Here are some core properties of declaration-site variance:

We obtain the following subtype relationships: Let C be a generic class with one type parameter X. Assume that S is a subtype of T. If X is marked out then C<S> <: C<T>; if X is marked in then C<T> <: C<S>. Note that there is no subtype relationship between C<S> and C<T> if X is marked inout, unless S == T.

A type parameter with explicit variance can be used in the specification of a superinterface. For example:

class A<out X, in Y, Z> {
  X get m;
}

class B<out X, inout Y, in Z> implements A<X, Y, Z> {}

Soundness is ensured via a number of rules like the following: It is a compile-time error if a type parameter X marked out occurs in a non-covariant position in an actual type argument for a superinterface D when the corresponding type parameter of D is marked out; and if X occurs in a non-contravariant position in an actual type argument for D when the corresponding type parameter is marked in; and if X occurs at all in an actual type argument for D when the corresponding type parameter is marked inout.

The interaction with dynamically checked covariant type parameters is similarly guarded: It is a compile-time error if a type parameter X marked out occurs in a non-covariant position in an actual type argument for a superinterface D when the corresponding type parameter of D has no explicit variance.

In return for all these restrictions, we get static safety: For a class where all type parameters have explicit variance, every (non-dynamic) member access which is statically type correct is also dynamically safe (no type checks on parameter types etc. are needed at run time).

In general, declaration-site variance can be used for classes which are intended to be strictly type checked with respect to variance everywhere.

Answer 1 · 2019-09-02T17:04:04.000Z

First draft of feature specification at PR dart-lang/sdk#557.

Answer 2 · 2019-09-03T07:33:16.000Z

The "exact" type already exists in Dart in certain places - the type of a literal or object creation expression may have an exact type, which is why List<int> l = <num>[1]; is a compile-time error instead of a run-time downcast failure.
I assume those cases can be subsumed by exact types and still behave the same.

For use-site invariance, we will have both List<Foo> and List<exact Foo> as instantiations, with the latter being a subtype of the former. We don't actually introduce a nww type called exact Foo, just new syntactic forms for type arguments, and a suitably expanded type relation between instantiated generic types. We can say that we introduce exact Foo, but it's not a type, just a type pattern.

As stated, the type arguments can only be used in type pattern expressions (type annotations), not class expressions or literals.
That is, you can write List<exact Foo> x; as a type annotation, but not class C extends Iterable<exact int> or new List<exact int>(). Those make no sense, they are always "exact" in that they are run-time invocations with a single value.

The following should be allowed:

List<exact String> foo(List<exact int> list) => (list..add(42)).map((x) => "$x").toList();

It is meaningful and useful.

What about:

class C<T> {
  List<exact T> mkList() => <T>[];
}

Is this meaningful? Maybe. If you have a C<num> c, then you only know that the result of c.mkList is List<num>. If you have C<exact num> d; then the static type of d.mkList is (or should be) List<exact num>. If the code had not written exact T in the return type, then it would not have been true. The method:

  List<T> mkList() => <T>[];

will never have a static return type of List<exact anything>, not even for D<exact num>.mkList().

So, we need some substitution rules for type arguments to make this work.
exact T[exact S/T] -> exact S
exact T[S/T] -> S
T[exact S/T] -> S (?)

It also suggests that a lot of platform libraries should be changed to have exact in their return type, because they are really intended to be exact. Take: Set<T>'s Set<T> union(Set<T> other). Here we should make it Set<exact T> union(Set<T> other), so that if I have a Set<exact num> then the union returns another Set<exact num>.
(I just realized that I have written exact instead of exactly everywhere. I'm not particularly fond of all these keywords, but shorter seems better :)

That suggests a migration issue. If we change some platform libraries to return, say, List<exact String>, then any existing implementation of the same interface will no longer be valid when it returns merely List<String>, a super-type of the required return type. We may want to have an NNBD-like migration process where legacy libraries are accepted for a while, until everybody has migrated.

If we need migration anyway, then we don't need to make the syntax backwards compatible, and we could do something else, like:

class C<+T , -S, =U> {  // out T, in S, inout U
   List<+T> mkList() => <T>[]; // not exact T
   List<U> mkList() => <U>[]; // exact U
}

We do get the inherent complexity of a two-tier type system where it matters whether you write exact int or int on every type argument in your API. You have to get it right, otherwise you make lock yourself out of possible later changes.
If you say foo(List<exact num> arg) then you can't add elements to arg later, and you prevent anyone with merely a List<num> from calling you. They'll have to do list.cast<exactly num>() to convert their list.

I expect exact types in arguments to be rare (you really do need to do modification), and for them to be common in return types.

For declarations site variance, we already have something similar for function types. For those, the variance is automatically computable from the co-/contra-variance of the occurrences of the type arguments.
We could do the same for class parameters, but that would break all our existing unsafely covariant classes. We could do it anyway, and require migration which would effectively mean that all existing type arguments become covariant, and we write an explicit covariant on all existing contravariant occurrences (well, the ones in parameters, we can't help the ones in function return types).

Answer 3 · 2019-09-03T16:07:04.000Z

The "exact" type already exists

Right. The notion of an exact type is not specified, but it is used in some cases by our tools (even to raise an error for certain "impossible" casts).

The notion of a type argument which is known exactly is a different concept (an object can have dynamic type List<exactly T> even though it is an instance of some proper subtype of List). I just checked the text above to make sure that it doesn't ignore that distinction, and adjusted a couple of sentences.

For use-site invariance, we will have both List<Foo> and List<exact Foo> as instantiations

Both of those are instantiations of the generic type List (where instantiation of a generic type means providing actual type arguments), but we will not have an object whose dynamic type is List<Foo>. If the dynamic type T of a given object o is such that List<Foo> is one of the superinterfaces of T then o has type List<exactly Foo>. In general, every type argument in the dynamic type of an object is exactly something, and the superinterfaces will carry this property with them.

class A<X, Y> {}
class B<X> implements A<X, int> {}

With that, B<String>() has dynamic type B<exactly String>, and it is also of type A<exactly String, exactly int>.

We may or may not want to let Type.toString() reveal this bit, but it must be present in the dynamic representation of types in order to maintain soundness.

we introduce exact Foo, but it's not a type, just a type pattern

I'd prefer to say that a type accepts type arguments, each of which may have the modifier exactly, which also implies that exactly Foo is not a type.

the type arguments can only be used in type pattern expressions (type annotations),
not class expressions or literals.

exactly can be used on type arguments, e.g., <List<exactly num>>[], but not on types, and we need to make the distinction that "type arguments" given prescriptively are types. So List<exactly num> is OK as a type annotation, but List<exactly num>() as an instance creation is not, and <exactly num>[] is not; but when exactly is nested one level deeper then it is again a type argument, which is the reason why <List<exactly num>>[] is OK.

class C<T> {
  List<exactly T> mkList() => <T>[];
}

That can be allowed, and would be safe, but the computation of the member access type must take into account that the statically known value of T is just an upper bound.

main() {
  C<num> c = C<int>();
  List<num> xs = c.mkList(); // Static type of `c.mkList()` is `List<num>`.
  List<exactly num> ys = c.mkList(); // Error (downcast).
}

Of course, with class C<inout T> we would have an uninterrupted chain of exactness, and c.mkList() would have static type List<exactly num>.

we need some substitution rules

I believe the most straightforward way to get this right is to consider the types of member accesses, with a case for each variance of the type parameters of the class. Type parameters with no variance modifier are the most complex ones, of course, because they are allowed to occur in any position in a member signature.

We may want to have an NNBD-like migration process where legacy libraries
are accepted for a while

Indeed.

we don't need to make the syntax backwards compatible

Right, but C<=X, -Y> x; may not be optimally readable (and in, out, inout isn't all that verbose). In any case, that's probably not more breaking than the keywords.

do get the inherent complexity of a two-tier type system where it matters
whether you write exact int or int on every type argument in your API

I think that wouldn't ideally be such a common situation: The declaration site variance modifiers are required to match the usage (so if you have a type parameter out E then it just can't be the type annotation of a method parameter), and a subtype receiver will have a subtype member (including: less specific parameter types and more specific return types).

The use of exactly in a member signature would be specifically concerned enhancing the type safety in the management of legacy types (with type parameters that have no variance modifiers).

I would expect the reasonable trade-off to be somewhere between just using the legacy types as they are (with the current level of type safety, which has been used in practice for years without complete panic) and teasing out the very last drop of static type safety, by adding exactly as many places as possible. In any case, there are rules for eliminating exactly from interface type members such that the resulting typing is sound.

Answer 4 · 2019-12-20T17:50:28.000Z

I more like the +T and -T syntax in Scala

Answer 5 · 2019-12-23T01:59:33.000Z

The current in out looks like a primary student and verbose.
Kotlin and C# is using in and out too，but reads badly.

Answer 6 · 2019-12-23T17:18:51.000Z

Is it the AUTHOR of the Bad class, or the USER of said class? Sure, user will benefit, too, if the author makes fewer mistakes, but other than that?

The primary beneficiary is the USER. The List class is one of the primary examples of something that should be invariant, but is instead unsoundly covariant. We have numerous users who are frustrated that this results in unexpected runtime errors that could have been caught at compile time. Enclosed below is an innocuous looking test program that fails at runtime instead of at compile time. Sound variance allows users who strongly prefer to rule these errors out at compile time to do so.

void addToList<T>(List<T> target, List<T> src) {
  for(var x in src) {
    target.add(x);
  }
}

void test(List<num> nums) {
  addToList(nums, [3, 4, 5]);
}
void main() {
  List<double> doubles = [3.5];
  test(doubles);
}

Answer 7 · 2019-12-23T17:21:27.000Z

Question 2: you seem to be leaning towards the use of "inout" as a synonym of "exact", but I have difficulty understanding the reasoning leading from "inout" to "exact".

Reading between the lines of this question, I think you are missing the primary effect of this feature, which is that it changes the subtyping relation between types. Describing inout as exact is sensible because given class Invariant<inout X> ... we know that any variable of type Invariant<num> contains only objects that were allocated as Invariant<num> or a direct subclass thereof. Specifically, it will never contain an Invariant<int>, nor an Invariant<Object>. In this sense then, it contains objects whose generic parameters are exactly as described by the type. Hence exact. Does that help?

Answer 8 · 2019-12-23T19:03:27.000Z

Describing inout as exact is sensible

I think tatumizer is suggesting exact will make a better (easier to understand) syntax for invariance. (Compare: "let xy mean neither x, nor y.")

Answer 9 · 2019-12-23T22:09:01.000Z

I find the word inout confusing. It's difficult to see how this word may carry the meaning of "exact".

Ah, sorry, I misunderstood your question. Here's the intution:

a type variable labelled in may only be used to pass things in to methods
- void add (T x)
a type variable labelled out may only be used to pass things out of methods
- T get first
a type variable labelled inout may be used to pass things in to and out of methods
- T swap(T x)

Does that help?

(By the way, what combination of "in" and "out" characterizes the current default behavior in dart?)

Neither. Classes are allowed to use type variables everywhere in a method, which corresponds most closely to inout, but subtyping is covariant, which corresponds to out.

Let's consider your example with addToList. What will our new program look like to prevent this bad outcome? Where "in", "out" and "inout" should be added? And what line in the code will be flagged statically after we do these changes?

For completeness, here is the example rewritten to use an example of an invariant implementation of List. The call to test on the second line of main is statically rejected.

import 'dart:collection';

class Array<inout T> extends ListBase {
  List<T> _store;
  int get length => _store.length;
  void set length(int i) => _store.length = i;
  T operator[](int index) => _store[index];
  void operator[]=(int index, T value) => _store[index] = value;
  Array.fromList(List<T> this._store);
}

void addToList<T>(Array<T> target, Array<T> src) {
  for(var x in src) {
    target.add(x);
  }
}

void test(Array<num> nums) {
  Array<num> ints = Array.fromList([3, 4, 5]);
  addToList(nums, ints);
}
void main() {
  Array<double> doubles = Array.fromList([3.5]);
  test(doubles);
}

Answer 10 · 2019-12-24T01:12:57.000Z

@tatumizer I think this is getting a bit far afield - variance is definitely a confusing subject, but I don't think we're going to fix that here. Is it fair to summarize your general take here as being that you find + - = more intuitive than out in inout? Or are you expressing a preference for something different than that?

Answer 11 · 2019-12-24T04:59:12.000Z

repeat the in 、out and inout here and there make me looks like a fool; why not make use of inputParameter T outputParameterR?

Answer 12 · 2019-12-24T06:17:40.000Z

@leafpetersen I think the - and + style are more concise and the in and out style are more easy to understand.

Answer 13 · 2020-07-29T04:02:56.000Z

Having experienced this feature in Kotlin I strongly recommend we never go there. It is completely impenetrable to most developers. If you need to know type algebra to be able to use a feature then that feature doesn't, IMHO, belong in Dart.

Answer 14 · 2020-07-29T05:37:36.000Z

@Hixie Just to be sure we're on the same page: Kotlin has two different kinds of variance - this kind (declaration-site) which is what C# uses, and which I don't usually think of as requiring a lot of type algebra and use-site variance which is what Java uses, and which does require introduce some serious type algebra. And the combining the two, yeah, that gets pretty wild pretty quick. Are you specifically commenting on declaration site variance (marking type parameters classes as usable as in, out, or inout), or on the Kotlin style combination?

Answer 15 · 2020-08-04T07:04:08.000Z

The declaration-site version.

Answer 16 · 2021-01-29T03:51:23.000Z

It looks like this has been implemented by @kallentu (behind an experimental flag):

https://medium.com/dartlang/dart-declaration-site-variance-5c0e9c5f18a5

Answer 17 · 2021-01-29T08:58:51.000Z

That is true, but parts of the implementation need to be updated (many other things happened in Dart since then), so there's some work to do before it is ready to be released.

Answer 18 · 2021-06-25T18:04:33.000Z

Hi, @eernstg + @leafpetersen, any updates on this? I see a lot of issues about variance-related problems a lot, and was wondering what the timeline could look like.

Answer 19 · 2021-06-27T19:59:48.000Z

No ongoing work on this right now, but the feature is certainly on the radar, and I'm not the only one who keeps wanting it. ;-)

Answer 20 · 2021-08-07T11:29:38.000Z

If this feature is implemented, List<T> will become List<inout T> in declare by default?
it can use like following code: ?

List<out num> list = <int>[1,3,5];
// ok
var ele = list[0];
// compiler error
list.add(9);

Answer 21 · 2021-08-08T09:39:43.000Z

With declaration site variance, you can't write List<out num> list = .... That's use-site variance.

If List is declared as List<inout num> then it's invariant, so List<num> list = <int>[1, 3, 5]; would be a compile-time error.

It's undecided whether List can be migrated to List<inout T> or it has to stay the original unsafe List<T>.
That depends on how migration will work, and how iteraction between old code and new variance declarations work, and whether we actually want to make List be invariant in practice.

We have a large number of old classes which were not designed with potential invariance in mind.
(My personal worry is that we can't convert those to using variance without breaking a lot of code, not unless we introduce use-site variance as well).

Answer 22 · 2021-08-08T15:38:48.000Z

My two cents: Favor large breaking changes that make the APIs cleaner in the long run over smaller changes that ease migration. I specifically have Iterable.firstWhere and DropdownMenuItem.value in mind from null safety, which are harder to use now than they should be because they were made backwards-compatible in a time where devs were anyway going over their code to make sweeping changes. That and all the issues on here about bugs when running in mixed null safety mode, it sounds like for features as big as variance, they may just be worth a breaking version change (or if they're opt-in, no mixed mode).

List is obviously widely used, and it seems that modifying it in any way will cause breakage, but I'd rather lists be easier and safer to use in the future than worry about how to update existing apps to a new version of Dart (which is always going to take time and effort). Especially if having stricter variance can point out some potential problems during migration.

Answer 23 · 2021-08-08T23:44:05.000Z

I have another question, if List is List<inout T>, List can use covariant keyword in parameter?
such as:(Suppose List is declared as List<inout T>)

// in class Fruit
void test(List<num> fruits) {
}

// in class Apple (extends Fruit)
@override
void test(covariant List<int> apples) {
}

Answer 24 · 2021-08-09T00:57:23.000Z

@Levi-Lesches I can't speak for the Dart team, but I don't think that's practical with the sheer amount of code that's already written. At the scale we're working at – millions of lines of code – every change needs to be incremental and automated. Clean breaks simply aren't feasible.

Answer 25 · 2021-08-09T02:54:15.000Z

My two cents: Favor large breaking changes that make the APIs cleaner in the long run over smaller changes that ease migration. I specifically have Iterable.firstWhere and DropdownMenuItem.value in mind from null safety, which are harder to use now than they should be because they were made backwards-compatible in a time where devs were anyway going over their code to make sweeping changes. That and all the issues on here about bugs when running in mixed null safety mode, it sounds like for features as big as variance, they may just be worth a breaking version change (or if they're opt-in, no mixed mode).

List is obviously widely used, and it seems that modifying it in any way will cause breakage, but I'd rather lists be easier and safer to use in the future than worry about how to update existing apps to a new version of Dart (which is always going to take time and effort). Especially if having stricter variance can point out some potential problems during migration.

Agree, I sincerely hope that dart can have a clean, clear and consistent syntax

Answer 26 · 2021-08-09T07:50:28.000Z

@Silentdoer I'd say that the covariant example would not be allowed.
The covariant keyword allows a parameter to (unsoundly) be a subtype of the superclass method's parameter. It's unsound relative to the covariant generics of the class. (Which is why it has to insert a run-time if (arg is! paremeterType) throw TypeError(...); check.)

If List (or any other generic class) is declared as List<inout T>, then it's invariant. That means that List<int> is not a subtype of List<num>.

On thing I don't know whether has been define yet is whether you can use covariant inside the class itself.
Can List be declared as

abstract class List<out T> {
 T operator[](int index);
 void add(covariant T value);
 ...
}

The T in add occurs contravariantly, which is should be disallowed for a covariant ("out") type parameter.
Does the covariant allow overriding that? And if so, would "class is covariant, every covariant occurrence in a parameter is covariant" be a compatible description of the current behavior? (It's not, we can also have contravariant type variable occurrences in return types).

Answer 27 · 2021-08-09T09:09:54.000Z

if List is declared as List<inout T>, List is implements Iterable;and if Iterable is declared Iterable<out T>, then List<inout T> implements Iterable<T> is allowed, right?
Based on the above assumption, the following code is ok?

// in class Fruit, When calling it, the argument is List<Fruit> object
void test(Iterable<Fruit> fruits) {
}

// in class Apple (extends Fruit), When calling it, the argument is List<Apple> object
@override
void test(covariant Iterable<Apple> apples) {
}

Answer 28 · 2021-08-09T10:31:02.000Z

@Silentdoer It's not yet specified what happens to existing classes, or whether variance is applied implicitly. It may be possible to keep classes "unsafely covariant" (like now). If not, the List interface does have "out" uses of the type parameter, so if it must be variance annotated it'll probably have to be inout (unless we do something for the "out" uses like marking them covariant, and allowing that to keep breaking safety).

(There is no full spec for this feature yet, so some things are still undecided).

Answer 29 · 2021-08-09T11:12:48.000Z

@Silentdoer It's not yet specified what happens to existing classes, or whether variance is applied implicitly. It may be possible to keep classes "unsafely covariant" (like now). If not, the List interface does have "out" uses of the type parameter, so if it must be variance annotated it'll probably have to be inout (unless we do something for the "out" uses like marking them covariant, and allowing that to keep breaking safety).

(There is no full spec for this feature yet, so some things are still undecided).

thanks for your reply

Answer 30 · 2021-08-10T11:48:56.000Z

The feature spec proposal at #1230 does consider the use of the modifier covariant on an instance method parameter (see line 312 for an example), and I don't think we need to introduce any special rules about this modifier.

As @lrhn mentions, and assuming that the rules will be similar to that feature spec, we can't do covariant List<int> here, because List<int> is not a subtype and not a supertype of List<num>.

But this is fine, assuming class Iterable<out E> ...:

class Fruit {
  void test(Iterable<Fruit> fruits) {}
}

class Apple extends Fruit {
  void test(covariant Iterable<Apple> apples) {}
}

The semantics is unchanged: Apple.test must perform a dynamic type check on the actual argument, such that the type error of (Apple() as Fruit).test([Fruit()]) is detected.

However, I wouldn't actually expect class List<inout E> to happen, because that's a breaking change if there ever was one...

Answer 31 · 2021-08-10T12:01:56.000Z

However, I wouldn't actually expect class List<inout E> to happen, because that's a breaking change if there ever was one...

Well, only if we don't also introduce use-site variance as well, so you can write List<out num> numList = intList;. 👿

Answer 32 · 2022-04-14T21:06:41.000Z

FWIW, I continue to think that the benefits here (more expressive language) don't outweigh the costs (more complicated language). The main thing driving me to this conclusion is that I've never seen a description of this feature (for any language that has it) that is clear and complete but doesn't use terms that require a background in type theory to understand (like "soundness" or "variance"). In contrast, for example, I can explain subtyping and even generics to someone who understands only imperative programming. Pattern matching makes sense to people once they can get past the size of the feature. Function calls are easy to explain. But covariance and contravariance and inout type parameters are things that for some reason I've never seen explained in such a way (and I've certainly never managed to explain it myself in such a way).

Answer 33 · 2022-04-14T21:51:28.000Z

The problem we have is that we don't have soundness. The reasoning behind that can be hard to explain, but it does bit people occasionally, and they do understand "unsound".

I'm not sold on declaration-site variance. (I honestly can't remember which is which of "in" and "out", I find Java's ? extends an ? super more readable, which is really scary!)
It does provide a way to get soundness that we don't currently have, without the restrictions of simply being invariant.

But let's try an explanation, just for the challenge:

You understand subtyping, right? An int is a num. The reasoning behind subtyping is that a value of the subtype can be used everywhere a value of the supertype is expected. Anything you can do with a num, you can do with an int. Anything that needs a num can be given an int.
This doesn't necessarily scale to larger types containing num or int, though.
Take a List<num> vs. a List<int>. You can't safely use a List<int> everywhere you can expect a List<num>.

If you expect a List<num> and someone gives you a List<int>, and all you do is read from the list,
then it's safe, because the list gives you an int out where you expect a num, and an int is safe where you expect a num.
All is well.

However, if you expect to be given a List<num>, then you might try to add a num, even a double, to the list.
When given a List<int>, that must fail, you cannot add a double to a List<int>, because then it's no longer a list of integers.
A List<int> limits what you can put into the data structure in a way that a List<num> doesn't.
It's the List<int> which expects an int, and you who must satisfy its expectation, not the other way around.
That's why most programming languages do not allow a (mutable) List<int> to be used where a List<num> is expected.
Dart does, and Dart class generics is unsound for precisely this reason.

The purpose of this feature is to mark generic type parameters, like the E in class List<E> ... ,
with either out, int or inout,
saying whether it's safe and sound to take values of the type out, put values of the type in, or even to do both.
Then we change the subtype rules, so you can only use a MyClass<int> where a MyClass<num> is expected,
or vice-versa, if that's actually sound.

For example, we can declare class Future<out T> with an out parameter.
That would make Future<int> a valid subtype of Future<num>, meaning that a Future<int> can be used
where a Future<num> is expected.
All a future can do is to eventually provide you with a value of that type. The value comes out, and it being an
int is fine when a num is expected.

And we can declare class Sink<in T> with an in parameter, because all you can do with the type is to call
add to put values into the sink. If you expect a Sink<int>, someone giving you a Sink<num> is safe and sound,
because you can still put int values into it. Then Sink<num> is a subtype of Sink<int>.

And class List<inout E> would ... just not be safe in either direction, so it would disallow you from using
it incorrectly. If you expect a List<int>, the only type satisfying that would be an actual List<int>.
Then List<num> and List<int> would just not be related to each other, and you can't assign one
to the other.
That would be a very big breaking change, and that's a reason we will probably keep the current
"out + unsafe in" behavior as well, and let List keep being unsound.
New classes can choose to use inout, but migrating old classes is going to be hard.

Hope it makes sense. No use of "covariance" or "contravariance" :)

Answer 34 · 2022-04-15T22:39:36.000Z

In addition to "covariant/contravariant" and "in/out" language, which may be new concepts to many, it can also be explained in terms of and "supertype/subtype". I'll adapt @lrhn's explanation but replacing in with super, out with sub, and inout with exact (I know this conversation has been had before so I'm not necessarily advocating for using these keywords, just the ideas for use in the documentation):

Subtyping is relatively straightforward: you declare class A extends B to make A a subtype of B. Subtypes are just another way of saying "A is a type of B". An int is a type of num. Now you can use the subtype anywhere the supertype is expected: anything you can do with a num you can do with an int, and anything that needs a num can be given an int.

This doesn't necessarily scale to larger types with generics, though. Compare a List<num> to a List<int>. You can't safely use a List<int> everywhere you can expect a List<num>. If you expect a List<num> and someone gives you a List<int>, then you can safely read values from the list, since an int can always be used in place of a num. However, if you try to add a num (like a double), to a List<int>, that will fail since it's no longer a list of integers.

The purpose of this feature is to mark generic type parameters as either sub, super, or exact, which denotes which types are safe to use. For example, class Future means a Future<int> can be used wherever a Future<num> is expected. This is safe since all a future does is provide a value and we know an int is always safe to use when a num is expected (because it's a subtype).

Conversely, class Sink<super T> means you can use a Sink<num> when a Sink<int> is expected (remember, num is a supertype of int). That's safe since all anyone can do is add ints to it, and we know an int is always safe to use when a num is expected. However, you cannot use a Sink<num> in place of a Sink<int>, since that would allow you to add a double to a collection of ints.

But there are some cases where you can't accept a subtype or a supertype. Think back to List. If we used class List, then someone could use a List<int> instead of a List<num>, but that would mean you can no longer add other types of nums to the list, like a double. If we used class List<super E>, then someone could use List<num> instead of a List<int>, which means they could give a double when an int is expected. This is where exact comes in: class List<exact E> would only allow you to use the exact type, so List<int> means List<int> and List<num> means List<num>. This allows users to safely put values in the list, while also allowing them to safely read values from this list.

Hopefully, this doesn't use any language or concepts that are too complex. IMO, this level of detail is something I could reasonably expect to see in a Medium article or in the dart.dev documentation. Granted, in/out/inout does give a level of intuition that makes it easy to reason about the flow of types in your programs, but the supertype and subtype logic can be explained without introducing any new ideas.

Answer 35 · 2022-04-16T09:47:53.000Z

I think these explanations are great and the average dart user is able to understand the use of in, out and inout (or whatever they will be named) after reading them.
However I believe that one does not even need to understand them to use and benefit from this feature. For example if someone tries to return a generic type that is declared in, the compiler will complain. And as long as the message is something helpful along the lines of

The generic type T is declared in and can therefore only by used as a method argument to be passed into the class. Mark it as inout to allow it as a return type.

, the user could just blindly follow the instruction and usually get what they want and also what is the correct behavior.

Users may not always know ahead of time if the type should be in, out or inout, but they know what types they want to return or use as parameters.

Likewise if a generic type is declared for example inout but the class only uses that type as return type, the analyzer could show a warning:

The generic type inout T is only used as a return type and should therefore be declared as out T.

And again, just blindly following that instruction usually yields the correct result.

Answer 36 · 2022-04-19T08:55:57.000Z

Just to nitpick myself: Future cannot be declared as Future<out T> because ... Future<num> f = futureOfInt; (await futureNum.asStream().toList()).add(3.14); creates an List with the same type as the future, and List cannot be declared as covariant. (And out T declaration means that T can only soundly be passed as argument to other out T type parameters.)

That very much makes me worry that declaration-site variance will never be able to apply to even the most obvious of existing types. If it cannot apply to Iterable or Future, the two most covariant types I can think if, what is left?
If we cannot variance-annotate the platform libraries, and have to keep them unsafely covariant, then any soundness other classes build on top of that will be on a shaky foundation.

(At least for List, we can do abstract class List<out E> ... { void add(covariant E element); ... }. That moves the unsoundness from the class itself to only the unsafe methods. Can't do that for Future.asStream since covariant only works for parameters, not return types. Or we can just let list stay unsafely covariant, and allow any type variable to be used in an unsafe position.)

Answer 37 · 2022-04-19T09:43:12.000Z

Future cannot be declared as Future<out T>

This is a known issue: Soundness of the api of a class like Future does not magically extend to soundness of types used in the instance member signatures of said class, or types reachable through two or more such steps.

So if C uses dynamically checked covariance of its type parameter in a member m then an instance of C will potentially throw when m is invoked, because one or more typing requirements fail to be satisfied. In particular, a list may throw when add is invoked, because that method uses the type parameter E as the type of its actual argument, and E has no explicit variance (so it's the old-fashioned kind of covariant where soundness is enforced by dynamic checks), and this is true no matter how we got hold of that list.

Let's assume that List won't be changed to class List<inout E> ... (which is basically a given). In this situation we need to support a use-site mechanism in order to impose sound usage. Use-site invariance is a minimal model which will enable this kind of post-hoc static soundness for classes where dynamically checked covariance is still used:

void main() {
  List<exactly num> xs = [1];
  // xs = <int>[2]; // Compile-time error.
  xs[0] = 1.5; // Statically checked, will never give rise to a type error at run time.
}

So we can keep List and other widely used classes unchanged (to avoid the massive breakage), and we can still introduce statically checked variance at each usage point by using an invariant type.

This is more verbose, and it is less convenient than having a class List<inout E> ... in the first place, but it does allow for the static soundness to be introduced gradually. Note that we can perform tests like if (myList is List<exactly T>) ... for any given T, which will allow us to safely confirm (or disprove) invariance, even in the case where a list has been obtained with the type List<S> for some S (which could be any supertype of T). This is helpful in the situation where the invariant treatment of lists has been introduced in parts of a system, and not in other parts.

Answer 38 · 2022-04-19T15:40:47.000Z

But covariance and contravariance and inout type parameters are things that for some reason I've never seen explained in such a way (and I've certainly never managed to explain it myself in such a way).

Nonetheless, people seem to manage to use Kotlin, Java, C#, Rust and many other languages which have variance.

I do understand the concern, but I do not agree with the conclusion. Variance is hard, but it's a fact of life - it's just a mathematical property of the underlying structures that they behave that way. You can pretend otherwise, but that just pushes the problem off to runtime. So, you can deal with it statically, or you can deal with it at runtime. It's not clear to me that the runtime error there is any easier to explain, and it's certainly harder to debug and fix.

In general, most programmers most of the time won't need to deal with variance, just as they most of the time don't need to now. But the inability to actually express the correct variance in the cases where you need to is a major shortcoming in the language.

I'll also note that this comes up whenever I talk to people about Dart in the context of education.

Answer 39 · 2022-04-21T22:16:51.000Z

In contrast, for example, I can explain subtyping and even generics to someone who understands only imperative programming.

My experience is that you can correctly explain a subset of subtyping and generics to someone who understands only imperative programming, but that many programmers don't fully grok it and yet are able to program profitably most of the time. Many times in my life, I have had to explain the difference between virtually-dispatched overriding versus statically-dispatched overloading in C++/C+/Java to programmers that are otherwise quite sophisticated.

Once your language has function types, you are in a world of contravariance and covariance whether you want to be or not. Many programmers can program just fine using first-class functions and static types without ever realizing that return types are covariant and parameter types are contravariant. By the same token, many programmers can use generic types correctly that have declaration-site variance without fully understanding what those annotations do.

Dart already forces users to deal with variance. Variance is in the language and always has been. All generics are covariant. Users may not know that Dart has covariance, but they will find out the hard way the first time they accidentally write code like:

List<Object> things = <String>[];
things.add(123);

The only thing Dart currently lacks is the ability for a generic class to request anything other than covariance and the ability to ensure that a class's variance is statically correct.

Answer 40 · 2022-04-22T18:31:13.000Z

I think sound declaration-site variance can have an effect on Dart software development which goes considerably beyond actively writing variance modifiers.

First, if someone else knows exactly how to use this mechanism, and they use to improve the type safety properties of a library that I'm importing, then my code can be safer than it would otherwise have been, and I don't have to write any variance modifiers at all.

As @leafpetersen mentioned, the underlying type structure is just a fact of life, but this means that it is possible to benefit from the improved type safety without worrying too much about exactly why the type safety has improved, and then we can proceed to an active use of the mechanism more gradually.

I liked the explanations about variance where the focus is on "input" and "output", but it is also possible to take a different path which is basically relying on feedback from the type checker:

Let's assume that we have a class C with a type parameter X which is legacy (that is, it has no variance modifier).

Try to change X to out X. If there are no compile-time errors then C is already safe, and you're done. This simply means that we're adding an explicit marker for the static type safety which was already there.

On the other hand, if there are compile-time errors then C uses X in certain ways that aren't statically safe. One way ahead is to change X to inout X. This means that X can be used safely anywhere in the class body, but clients of the class will pay, because they will have to live with a more rigid rule for assignability (C<num> c = C<int>(); used to be OK, but now we must use the same type argument, C<num> c = C<num>();).

Finally, the in modifier can be used to improve on the flexibility of the class in the very special case where X is only used for "in"put (that is, mainly, as the type of method parameters).

I think there are many, many ways to approach any complex topic, and we've looked at several of them in this discussion. The point is that the mechanism itself is technically well justified, and we have a bunch of ways to get to know how it works, and then we will develop a community awareness of how to deal with it every day.

Answer 41 · 2022-05-01T14:22:56.000Z

Hixie wrote:

I continue to think that the benefits here (more expressive language) don't outweigh the costs (more complicated language).

leafpetersen wrote:

But the inability to actually express the correct variance in the cases where you need to is a major shortcoming in the language.

I think Hixie makes a valid point. Soundness is great, but even the most expressive language is of no use if it isn't able to attract and keep any new users. The reality is that Python and JS are the most popular languages and not Coq so any move away from Completeness towards Soundness carries the risk of alienating some new and existing users.

One of the core abstractions of Flutter is a 'Widget' where many different types are treated as a Widget and Flutter throws at runtime if misused. Awful Soundness-wise, but a good abstraction for making Flutter extremely easy to learn and attracting a lot of users. I think that it would be a mistake to assume that Soundness is always just 'better'.

I'm very excited about this feature, but It seems like efforts should be made to not overwhelm users of Dart that aren't ready for it so perhaps:

make it clear that it is an optional! and advanced feature that offers users the ability to give the compiler more information for easier maintenance of larger applications.
maybe even explicitly forbid introductory documents on https://dart.dev/guides and https://docs.flutter.dev from using explicit variance annotations unless it is absolutely necessary?

Answer 42 · 2022-05-23T09:24:44.000Z

Would it be hard to restrict dynamic type checking to type casting via as? So that I know that the type system is statically sound in principal, but whenever I tell Dart "here, take this thing, and I promise, it is of type X" I know that the static type checker is no longer responsible to know the actual type of X? Or something like PureScript's unsafePartial?

The reality is that Python and JS are the most popular languages and not Coq so any move away from Completeness towards Soundness carries the risk of alienating some new and existing users.

Maybe I'm the exception here, but I use a lot of JS, yet when I do something more complicated, I want to use a statically typed language. I tried TypeScript but ran into an unsoundness issue on day 1 and didn't like it. So for me it would be very attractive, if some language offers me a statically sound alternative to JS, as I already have JS if I want to use no types and TS if I want to sprinkle some types on my JS code without going fully sound.

Answer 43 · 2022-05-23T10:13:40.000Z

Would it be hard to restrict dynamic type checking to type casting via as?

That is not possible unless we add a more sophisticated way to denote types (we'd basically need an 'existential open' operation). The problem is, by example, that if you have a list xs with static type List<T> and you want to add an element of type S, checking S <: T is not sufficient in order to ensure that there will not be a run-time type error.

void main() {
  List<num> xs = <int>[]; // `T` is `num`.
  xs.add(1.5); // `S` is `double`, so `S <: T` is true. Throws anyway, because `double <: int` is _not_ true.

  // A hypothetical extension supporting an 'existential open' operation could be used.
  final List<final X>() ys = xs; // Binds `X` to the dynamic type argument `int`.
  if (1.5 is X) ys.add(1.5); // Safe, doesn't add anything in this case.
}

If we add support for sound declaration-site variance then the author of a class can mark its type variables as covariant, contravariant, or invariant, and this will ensure soundness. In cases where we can't change a class to do this (say, List), we could use use-site invariance to make each list-typed variable safe (as if the type parameter of List had been invariant).

I believe that it's more helpful to allow program elements to be expressed in a typesafe manner, even in the situation where we improve some program elements and leave others unchanged. A program with explicit casts (if we can even write them) isn't going to be any safer, and it isn't going to be possible for a developer to mentally ascertain that the casts won't fail (because that would involve undecidable questions, which is also the reason why we can't ask a type checker to establish the safety or non-safety of these casts).

Answer 44 · 2022-07-21T20:30:46.000Z

The reality is that Python and JS are the most popular languages and not Coq so any move away from Completeness towards Soundness carries the risk of alienating some new and existing users.

Python and JS aren't great comparisons since they don't have any types at all. If you don't like types to begin with, you aren't gonna like types regardless of how simple or complex they are. Looking at other statically typed languages, C++, C#, Java, Scala, Kotlin, and Swift all have sound variance for generic types as far as I know. (There are a couple of tiny edge cases like the null hole in Java wildcards, but that's essentially a bug.)

Answer 45 · 2023-03-09T15:45:09.000Z

Thanks everyone for working on this proposal. I honestly hope it will go through soon.

I understand the concern that some may have about restricting what you can do, but I think we can all agree how awful it is to have a false sense of correctness. Plus, this is more of an opt-in feature, most of the time you don't even need to know about it, but it does make a difference for those little cases when you need it.

Answer 46 · 2023-03-14T18:21:12.000Z

I think it's worth noting that we can already emulate invariance if we are willing to pay a small extra cost at run time (because the invariant type variable is represented as two type variables), and if we are willing to deal with error messages where the emulation is revealed (which means that they are more complex to read and understand, because they are basically revealing some implementation details).

Assume that we want to model the following class hierarchy (where invariant type variables play an important role because type inference will otherwise very easily create a setup that causes run-time type errors to occur). Here is a version that relies on invariance as a proper language mechanism:

// Library 'result.dart'.

abstract class Result<inout T, inout E> {
  const Result();
  Result<T2, E> andThen<T2>(Result<T2, E> Function(T) f);
  Result<T2, E> map<T2>(T2 Function(T) f);
  Result<T, E2> mapErr<E2>(Result<T, E2> Function(E) f);
}

class ResultOk<inout T, inout E> implements Result<T, E> {
  final T value;
  ResultOk(this.value);
  Result<T2, E> andThen<T2>(Result<T2, E> Function(T) f) => f(value);
  Result<T2, E> map<T2>(T2 Function(T) f) => ResultOk(f(value));
  Result<T, E2> mapErr<E2>(Result<T, E2> Function(E) f) => ResultOk(value);
}

class ResultErr<inout T, inout E> extends Result<T, E> {
  final E error;
  const ResultErr(this.error);
  Result<T2, E> andThen<T2>(Result<T2, E> Function(T) f) => ResultErr(error);
  Result<T2, E> map<T2>(T2 Function(T) f) => ResultErr(error);
  Result<T, E2> mapErr<E2>(Result<T, E2> Function(E) f) => f(error);
}

Here is an example where the Result class hierarchy is used:

void main() {
  final ok = ResultOk<int, String>(10);
  final res = ok.andThen<String>((n) {
    if (n.isEven) return ResultOk(n.toString());
    return ResultErr("not even");
  });
  print('Result type: ${res.runtimeType}');

  // Invariance is enforced statically:
  // Result<num, Object> superTyped = ok; // Compile-time error.
}

However, the same example will also work with the following emulation of invariance. The main point is that every invariant type parameter gets an extra phantom type parameter which is used to maintain the invariance constraint (so class C<inout X> ... is expressed as class C<X, X2> .... The newly added type parameters will always have the value T Function(T) when the original type parameter has the value T (so we will only have C<T, T Function(T)> ..., never something like C<T, S> where S is different from T Function(T)).

Clients will not see the extra type arguments (and they won't get an opportunity to get them wrong), because we use type aliases to access the classes. So class C ... is renamed to class _C ..., and then we add typedef C<X> = _C<X, X Function(X)>;.

In this particular example we have two invariant type parameters, and we're collapsing the extra type parameters such that we can enforce invariance for two type parameters using just one extra type parameter. The extra one will then have the value (T1, T2) Function(T1, T2) in the case where the original type parameters have the value T1 respectively T2.

// Variant of library 'result.dart' where invariance is emulated.

typedef Result<T, E> = _Result<T, E, _Inv<T, E>>;
typedef ResultOk<T, E> = _ResultOk<T, E, _Inv<T, E>>;
typedef ResultErr<T, E> = _ResultErr<T, E, _Inv<T, E>>;

typedef _Inv<T1, T2> = (T1, T2) Function(T1, T2);

abstract class _Result<T, E, Invariance extends _Inv<T, E>> {
  const _Result();
  Result<T2, E> andThen<T2>(Result<T2, E> Function(T) f);
  Result<T2, E> map<T2>(T2 Function(T) f);
  Result<T, E2> mapErr<E2>(Result<T, E2> Function(E) f);
}

class _ResultOk<T, E, Invariance extends _Inv<T, E>> implements
    _Result<T, E, Invariance> {
  final T value;
  _ResultOk(this.value);
  Result<T2, E> andThen<T2>(Result<T2, E> Function(T) f) => f(value);
  Result<T2, E> map<T2>(T2 Function(T) f) => ResultOk(f(value));
  Result<T, E2> mapErr<E2>(Result<T, E2> Function(E) f) => ResultOk(value);
}

class _ResultErr<T, E, Invariance extends _Inv<T, E>> implements
    _Result<T, E, Invariance> {
  final E error;
  const _ResultErr(this.error);
  Result<T2, E> andThen<T2>(Result<T2, E> Function(T) f) => ResultErr(error);
  Result<T2, E> map<T2>(T2 Function(T) f) => ResultErr(error);
  Result<T, E2> mapErr<E2>(Result<T, E2> Function(E) f) => f(error);
}

Thanks to @iazel for coming up with this example!

Answer 47 · 2023-05-16T05:38:30.000Z

But covariance and contravariance and inout type parameters are things that for some reason I've never seen explained in such a way (and I've certainly never managed to explain it myself in such a way).

Nonetheless, people seem to manage to use Kotlin, Java, C#, Rust and many other languages which have variance.

I hope our bar is higher than "manage to use", though. Dart is a significantly easier language to use than Kotlin, and Rust, and APL, and many other languages that have lots of wonderfully powerful yet difficult to explain features, largely because it doesn't have those features. I would hate for us to throw away our advantage because we wanted to add some expressiveness that costs more than it is worth.

I do understand the concern, but I do not agree with the conclusion. Variance is hard, but it's a fact of life - it's just a mathematical property of the underlying structures that they behave that way. You can pretend otherwise, but that just pushes the problem off to runtime.

There's lots of things we push to runtime because it makes the language easier to use. Memory management, for example. That isn't intrinsically bad. It's a trade-off.

So, you can deal with it statically, or you can deal with it at runtime. It's not clear to me that the runtime error there is any easier to explain, and it's certainly harder to debug and fix.

That's not at all obvious to me. I think we should test this assumption with some usability tests.

In general, most programmers most of the time won't need to deal with variance, just as they most of the time don't need to now. But the inability to actually express the correct variance in the cases where you need to is a major shortcoming in the language.

I literally cannot think of a time where I have wished the language had this feature, and I've been writing Dart more or less non-stop for 7 years now. How are we determining that it is a major shortcoming?

Answer 48 · 2023-05-16T07:30:38.000Z

The one case where I have wanted declaration-side contravariance is StreamTransformer. (Which should probably not have been an interface to begin with, it's just a single function type, the cast is only there to make up for the lack of correct variance.)

There are other interfaces which could be contravariant, like Sink or StreamConsumer, but those are rarely a problem, because you never use them in a way where you're likely to up-cast ... I'm guessing, all I know is that they've never really been a problem. The StreamTransformer is used that way, because it also has a covariant type parameter, and you want to connect multiple transformers.

The place where unsound covariance is biting people in practice tends to be up-casting collections, List<num> numList = <int>[1]; numList.add(3.14);. This gives no warnings and causes a runtime error.

If the goal is to prevent runtime type errors, then we must prevent this code.
One way to do so is to make List invariant.

That's hard to sell.
The other option is use-site variance modifiers, which makes invariance easier to work with by allowing "partial up-casts", so you can write List<? extends num> someNumList = <int>[]; but then it prevents you from calling add because the type parameter occurs contravariantly in that signature.

(If we get declaration-site variance, we should be able to make Iterable and Future be covariant. Those are inherently covariant containers, which only provide values. But their APIs have pitfalls that make it hard to be soundly covariant.)

Answer 49 · 2023-05-16T08:46:37.000Z

I think it's fair to mention that use-site variance is a rather complex mechanism. With Dart, I believe that we can use a much simpler mechanism, and get the same improvement of type safety in the vast majority of situations where we would otherwise have used the more complex use-site variance mechanism.

We have a proposal for use-site invariance, dart-lang/sdk#229, which is a lot simpler than use-site variance. The point is that you can declare in a type annotation (say, the type of a variable or a parameter) that the type argument is exactly as stated, and not a subtype:

void main() {
  List<exactly int> xs = [1, 2, 3]; // `xs.add` is safe.
  xs.add(1); // OK.
  xs.add(3.14); // Compile-time error.

  List<exactly num> ys = xs; // Compile-time error.
  ys = xs as dynamic; // Run-time error, `exactly` is enforced at run time.

  // Here's the reason why we'd use `exactly` in the first place.
  List<num> ys = xs; // Allowed, but `ys.add` is now unsafe.
  ys.add(3.14); // OK at compile time, but throws at run time.
}

Answer 50 · 2023-05-16T11:53:05.000Z

tl;dr Main points use a bold font rd;lt

@Hixie wrote:

How are we determining that it is a major shortcoming?

I'm genuinely surprised that we have so few reports about these run-time type errors that could have been detected at compile-time: It has been known since 1980 or so which kinds of variance are statically safe (and this paper from 1989 is all about that question).

Still, Dart was designed to allow some unsafe declarations and expressions to be compiled (with a type check at run time), explicitly based on the rationale that this would keep the type system simpler.

So we could just conclude that it's not a major shortcoming, developers are happy, and users don't encounter a huge number of apps crashing because of run-time type errors. Done!

However, I tend to conclude that the current design is useful, and in particular that it is a useful trade-off in a large majority of the Dart software which is actually being written.

Some specific kinds of software tend to have more serious conflicts with this trade-off. In particular, code which is written in a more functional style may give rise to type parameters that need to be treated as contravariant or invariant, because the default (and currently unavoidable) covariant treatment is wrong all the time.

A very typical example comes up when a first class function is stored in an object, that is, when there is an instance variable whose type is a function type, and that function type contains type variables declared by the class.

Some examples: dart-lang/sdk#36800, dart-lang/sdk#48108, dart-lang/sdk#48738, dart-lang/sdk#49419, #3007, dart-lang/sdk#52168.

Here is the core structure some of those examples:

class A<X> {
  final void Function(X) f;
  A(this.f);
}

void main() {
  A<num> a = A<int>((i) {});
  a.f(1); // Throws.
}

In the class A, X should be contravariant. This means that we should have A<num> <: A<int>, but we actually have A<int> <: A<num>, and there's an imminent danger of a run-time error if we ever use it. We miss out on the safe flexibility (it would be safe to allow A<int> a = A<num>((n) {});), but we are given the flexibility in the opposite direction (where every use of a.f will throw at run time). So we can do the wrong thing, but we're not allowed to do the correct thing.

I think sound variance can be used by designers of highly visible and widely shared code to give other developers the correct kind of flexibility (and avoid the unsafe kind), especially if they wish to use a style where first-class functions play an important role.

The rest of the world can continue to use dynamically checked covariance because it works quite well, and one generally doesn't even have to notice why certain constructs involving a class with a different variance is accepted or causes a compile-time error, we will just enjoy the fact that our code is now safer, even though we may not ever have written inout/in/out on any type parameter.

I believe that sound variance can help the community write safer code, and this can happen even if most developers haven't been thinking about sound variance at all. Over time we'll see how widespread the use of sound variance is; the main point is that it isn't a problem if it turns out to be used by somewhat specialized developers, and most of the Dart software out there continues to use dynamically checked covariance.

Answer 51 · 2023-05-17T06:28:25.000Z

I think it would be worth going back to a point I made in an earlier comment: can you fully explain this feature to someone who has no understanding of type theory? I don't think I can. In fact I'm not really convinced I can explain the feature completely at all. (Maybe because I don't have a background in type theory!)

FWIW, I strongly disagree with the argument that we can add a feature that people don't use and don't understand. Everyone who writes Dart has to understand every feature of Dart because they will come across code that uses those features. We've done a great job so far of keeping things easy to explain.

Answer 52 · 2023-05-17T09:47:03.000Z

As a developer, I honestly disagree with this statement:

Everyone who writes Dart has to understand every feature of Dart because they will come across code that uses those features.

I think it is quite common to learn of a language only the subset of features you need, and then expand your knowledge while you keep using it. I mean, you don't need to know all the various way you can declare a constructor in Dart to write your first "hello world" program.

More importantly, this feature would be transparent to most people, if you encounter it, it's because you actually need it. Right now you can end up in the same place, but have no easy solution out of it.

The worst part is that we can write programs that are correct from the type system point of view, and yet they will fail hard at runtime. We are telling users of the language that they can't trust the type system.

Answer 53 · 2023-05-17T10:01:46.000Z

@Hixie wrote:

can you fully explain this feature to someone who has no understanding of type theory?

Seems like it would not be possible to fully explain int i = 5; with no understanding of type theory, so where do we start?

I think we have a similar situation with many other features. For example, promotion of local variables relies on a detailed control flow analysis, and I'm pretty sure we are all writing code where that feature is used heavily, but nobody (other than, say, @stereotype441 and @leafpetersen ;-) would be able to explain fully why a specific variable has a specific type when it is used at a specific location.

My point is that we can have the following situation, where the maintainers of 'widely_used_thing.dart' must understand declaration-site variance in order to see that it is helpful to declare X as a contravariant type variable. When that's done, lots of clients of that widely used thing would get the benefit that there is a compile-time error when they try to do something that introduces potential dynamic errors.

// Library 'widely_used_thing.dart'.

class A<in X> {
  final void Function(X) f;
  A(this.f);
}

// Library 'my_app.dart'.
import 'widely_used_thing.dart';

void main() {
  A<int> a = A<num>((n) {}); // OK.
  // A<num> a2 = A<int>((i) {}); // But this is a compile-time error.
  a.f(1); // OK at compile time and safe at run time.
}

I believe the maintainers of 'my_app.dart' can benefit from having safer code, and they never need to dive into the details of variance, they just react to the compile-time errors. The point is that the compile-time errors are now correct, which is better than wrong (as they are today, when X is covariant).

That said, here is an attempt to summarize statically checked variance, intended to make sense for a person who knows something about subtyping and type parameters, but not variance:

If C is a generic class taking one type argument then C is a subtype of C<Super> whenever Sub is a subtype of Super. This is quite natural, and also safe, in many cases. We say that C's type parameter is covariant (because the types "co-vary": when the type argument gets smaller, the whole type gets smaller).

However, this rule is unsafe if the type variable is used in certain locations in the body of C. The safe locations are known as covariant locations, and they include return types and the types of final instance variables.

Locations like the type of a parameter of a method are contravariant. For those locations, the opposite rule is the safe rule: When Sub is a subtype of Super, C<Super> is a subtype of C (when the type argument gets smaller, the whole type gets bigger).

If the type variable is used in both covariant positions and contravariant positions, there is no safe rule: Even if Sub is a subtype of Super, we don't know anything about C in relation to C<Super>, they are just unrelated types. This is known as an invariant type variable.

Statically checked variance is expressed by putting the keyword out on a covariant type variable, in on a contravariant type variable, and inout on an invariant type variable. This roughly matches the usage: A covariant type variable can be the return type of a method, which is something we get "out" of the interaction with an instance of that type; a contravariant type variable can be a parameter type, which is a place where we put something "into" the method call. Finally, inout is allowed to be used in both kinds of locations.

If we do this, there will be compile-time errors whenever the type variable is used in a wrong position (that is, a covariant type variable is used in a contravariant position, and so on):

class C<out X, in Y, inout Z> {
  final X x; // OK.
  void m(Y y) {} // OK.
  Z z; // OK.

  void m2(X x) {} // Error, `X` occurs in a contravariant position.
  Y m3() {...} // Error, `Y` occurs in a covariant position.
  Z m4(Z z) => z; // OK, `Z` can occur anywhere.
}

In return for the extra work done to avoid these compile-time errors, the code is safer: The corresponding type errors that could occur at run time without statically checked variance are now completely eliminated.

Answer 54 · 2023-05-17T17:45:20.000Z

you don't need to know all the various way you can declare a constructor in Dart to write your first "hello world" program.

But you do need it to debug your program, if that involves stepping into other people's code. You don't run into features because you need them. You run into them because someone else used them.

Seems like it would not be possible to fully explain int i = 5; with no understanding of type theory, so where do we start?

I've explained int i = 5 to 4 year olds who have taken that knowledge and used it without any trouble. I've explained class hierarchies and inheritance and function pointers and abstract methods and interfaces and even mixins by building on simple principles. In my experience most people interested in programming with no theoretical background can more or less reach generics without trouble and only start to maybe have difficulty when you have to explain the implications of recursive generics (e.g. Foo<T extends Foo<T>>), and the implications of the covariant modifier on parameters.

here is an attempt

I beg you to take this explanation, and test people with it. Give people that explanation and a quiz with various different cases (both simple and complicated) and have them say where the analyzer will report issues, have them say what types are allowed where, that kind of thing. Basically, test if they can reason about the code.

We can have a control; give a similar explanation for, say, constructor syntax, or Isolates, and give a similar quiz.

If people in general score roughly as well for declaration-site variance as for the control, then I would be happy to admit my error and concede that this feature is worth it.

Answer 55 · 2023-05-17T23:12:25.000Z

I beg you to take this explanation, and test people with it. Give people that explanation and a quiz with various different cases (both simple and complicated) and have them say where the analyzer will report issues, have them say what types are allowed where, that kind of thing. Basically, test if they can reason about the code.

@Hixie I would propose the same to you for the example code that @eernstg wrote above.

class A<X> {
  final void Function(X) f;
  A(this.f);
}

void main() {
  A<num> a = A<int>((i) {});
  a.f(1); // Throws.
}

This is a sharp edge that our users have hit repeatedly (see the links above). Personally, I would vastly prefer the compiler telling me "don't do this", to an inexplicable runtime error. But I do understand that there is some value in making the world simpler in the common case even if it means the uncommon case becomes substantially worse.

This is another way of saying "there is inherent complexity here". Variance is not an opinion, it's a fact about the world, like gravity. You can surface it statically, or at runtime. Both of those have tradeoffs.

Answer 56 · 2023-05-17T23:34:50.000Z

I beg you to take this explanation and test people with it.

As an intermediate user of Dart, I think the idea of variance itself isn't too complicated to understand -- at least not compared to mixins, class modifiers, and pattern matching, which usually trip me up at least once -- but IMO, the fact that this is a repo for formal specs might be making things sound complicated. We should try something more user-friendly, something you'd see on dart.dev. Here's my attempt at a rough draft, but I'd like to point out that this Medium article by @kallentu really helped me understand this topic when I was first learning it. The idea is to avoid "complicated type theory" by presenting code that looks like it won't compile but does anyway.

In object-oriented programming, we use extends to demonstrate an "is a" relationship:

class Animal { void digestFood() { } } 
class Dog extends Animal { void bark() { } }
class Cat extends Animal { void meow() { } }

In the above example, a Dog "is an" Animal and a Cat "is an" Animal, but an Animal isn't necessarily a Dog; likewise, a Dog certainly isn't a Cat. The idea is that anywhere your code expects an Animal, you could use a Dog or a Cat instead, since they are still animals, just more specific types. If you had a function Animal.digestFood, you would expect Dog and Cat to inherit it. However, that relationship doesn't work the other way around. If you have Dog.bark(), you can't expect Animal.bark() or Cat.bark() to exist, because that's unique to Dogs.

When writing code, you have to keep in mind the difference between a "static type" and a "runtime type". Consider the following:

Dog russle = Dog();
Animal someAnimal = Dog();

russle.bark();  // Ok
someAnimal.bark();  // The method 'bark' isn't defined for the type 'Animal'.

The russle object is a Dog(), so its runtime type is Dog. It's also declared as a Dog, so its static type is Dog as well. While someAnimal also happens to be a Dog, it's declared as any Animal -- it just happens to be a Dog -- so its static type is Animal. We could as well write someAnimal = Cat(). Because we don't have a guarantee that someAnimal is actually a Dog (ie, its static type is not Dog), we can't call someAnimal.bark(). After all, what if it were a Cat?

Dart already handles this subtlety in the case of individual variables, but not for generics. For example:

class AnimalShelter<T extends Animal> {
  List<T> animals = [];
  void add(T animal) => animals.add(animal);
  T adopt() => animals.removeLast();
}

void breakGenerics() {
  AnimalShelter<Dog> dogShelter = AnimalShelter<Dog>();
  dogShelter.add(Dog());  // Ok
  
  AnimalShelter<Animal> generalShelter = AnimalShelter<Dog>();
  generalShelter.add(Cat());  // TypeError: Instance of 'Cat': type 'Cat' is not a subtype of type 'Dog'
}

Looking carefully, we can see that the T in dogShelter has a static and runtime type of Dog, but the T in generalShelter has a static type of Animal and runtime type of... still Dog. This means that it is an error to add a Cat to the generalShelter, despite its static type of AnimalShelter<Animal>. However, when Dart analyzes your code, it can only check static types, so this error happens at runtime, despite being detectable at compile-time.

To catch these errors at compile-time, use the new variance modifiers in your AnimalShelter class:

class AnimalShelter<inout T extends Animal> {

There are three modifiers to choose from: in for type arguments used as inputs, out for type arguments used as outputs, and inout for those used as both. Since AnimalShelter.add() uses T as an input and AnimalShelter.adopt() uses T as an output, we need to use inout. This will make Dart flag the following line as an error:

AnimalShelter<Animal> generalShelter = AnimalShelter<Dog>();  // Error: `Dog` is not exactly `Animal`

Answer 57 · 2023-05-18T20:33:51.000Z

@Hixie I would propose the same to you for the example code that @eernstg wrote above.

Yes! We should definitely test this too. Or the AnimalShelter case. Indeed I would go further and also test whether things get clearer or not when we add the annotations. Can people correctly determine which annotation to add in which case?

(We could also examine how common this pattern is by scanning pub.dev and google's codebase.)

This is a case where UXR testing is really well-suited, and how to test these things is well understood, and it would resolve the debate of whether it's too complex to add or not and move us from opinions to facts.

However, when Dart analyzes your code, it can only check static types, so this error happens at runtime, despite being detectable at compile-time.

There's tons of errors that we don't catch at compilation time which, given sufficient annotations, we could catch. I don't really understand why this particular one is more important than the many others which we've decided add too much complexity to be worth supporting.

Answer 58 · 2023-05-19T01:10:24.000Z

This is a case where UXR testing is really well-suited, and how to test these things is well understood, and it would resolve the debate of whether it's too complex to add or not and move us from opinions to facts.

I'm less optimistic about this than you, but I'm happy to give it a try. @InMatrix, thoughts on this?

There's tons of errors that we don't catch at compilation time which, given sufficient annotations, we could catch. I don't really understand why this particular one is more important than the many others which we've decided add too much complexity to be worth supporting.

Yeah, this is worth talking through a bit. Here's a few thoughts in this direction.

First, because we can.

Obviously, this is necessary but not sufficient. But a key starting point here is that there is a fairly straightforward (technologically speaking) static analysis with low implementation cost and low syntactic overhead which can prevent all of these errors. This static analysis has been around for decades, and is used in one form or another (i.e. either invariance, use site variance, or declaration site variance) by almost all modern statically typed languages (C#, Java, Swift, Kotlin, F#, OCaml, Rust, and I believe C++ in it's own way). Even Typescript has variance now!

You mentioned "tons of errors that we don't catch at compilation time". I don't know which specific ones you had in mind, but variance is solidly in the category of things that we both can catch, and can catch very easily. Some things we can't catch statically at all in general (e.g. stack overflows). Some things we can catch, but only via a tremendously syntactically heavy and/or hard to use system (e.g. dependent types for preventing index out of bound errors or ownership types for managing resources). Syntactically, variance is somewhere between zero and six extra characters per class. Complexity wise, well...

Second, because there's lots of UX prior art.

This isn't new. C# has had declaration site variance for a long time now. Java has use site variance (and it's pretty widely disliked, as I understand it). Kotlin has both. Swift is largely invariant. Typescript is adding declaration site variance.
These systems have been used by an inordinate number of developers now. There's a pretty strong developed consensus that invariance and/or declaration site variance are, in general, not that hard for programmers to deal with. With declaration site variance, you can always choose to make your class invariant, and if you try to make it covariant/contra-variant, you'll either get told "nope, can't do that", or it will just work. Now, it's true that variance is non-trivial to think about especially contra-variance. But we already always have to think about covariance in Dart! And we already have to think about contra-variance for function types. So there are no deep new concepts here. And of course, as proposed, you can always just continue not to think about it for your own classes (though it does, of course, affect what you can do with instances of other people's classes - no free lunches in life!)

Third, because it makes things faster.

Covariance checks on classes have a cost (sometimes significant). This is one of the reasons dart2js has an unsafe compile mode (cc @rakudrama ). On the VM, we do substantial whole program analysis to reduce the cost of these checks, but that's not always successful, and if we want to do non-closed world compilation our ability to optimize this will get much weaker. @mraleph or @alexmarkov might have more to say here.

Fourth, because it makes errors local.

This is actually a really key bit. Let's look at a variant of the example above:

class A<X> {
  final void Function(X) f;
  A(this.f);
}

void main() {
  A<num> a = A<int>((i) => i.isEven);
  var f = a.f;
   f(1.5); 
}

If we look at programming languages, there are a bunch of different ways this program can be interpreted.

In a dynamic typing world, we delay errors as long as possible. If it walks like a duck and quacks like a duck, it's duck. And here, everything walks and quacks just fine until we finally try to call isEven on a float, and then we discover we had a wolf in duck's clothing. It might have worked out! We gave it every chance! But sadly, this time it didn't. And the problem we now have is that the site of the error is potentially arbitrarily disconnected from the place that we went wrong. I've written this straightline, but of course I can pass f around arbitrarily before calling it. Where did we go wrong? Is it even in code we control? How do we track down where the "bad" value came from? It's... hard.

In a gradual typing world, we might do something like wrap the tearoff of f in a wrapper function which checks the type. Here we get the error when we call f, even if f would work with a float. So it's kind of a "trust but verify" for ducks. This does move the error slightly earlier. In this world, if you say something is an int, it really will be. So you'll never get a noSuchMethodError when you call isEven on an int typed variable. But you will get an error as soon as you try to pass a mistyped concrete value. And again, this error can happen a long way from the place where things went "wrong". The gradual typing world has a whole field of research around assigning "blame" for these kind of errors to try to trace the problem back to its origin (search for "gradual typing blame" if you're interested and you'll get more than you could possibly want).

In Dart 2, we move the checks even a little earlier than in the classic gradual typing world. Here we'd give an error as soon as you try to tear off a.f at the "incorrect" type. From a performance standpoint, this is a lot more efficient than wrapping the function (and avoids issues of identity), and it also moves the error even earlier in the program, closer to the place that we went "wrong". But... it's still potentially arbitrarily far from the place where things went off the rails. At the same time, it prevents a lot of perfectly useful programs! You might literally be in the process of passing an int to this function, which should be fine, but we'll still give you an error.

In a use site variance world there are various ways this program might change, but one of them might be that you would get an error on the assignment to a. The type system would warn you that this assignment violates the variance assumptions of A, and stops you from proceeding. This might be annoying! You might happen to know that you will never actually call f with anything but an int. But... no free lunch. If you want to do the assignment, you'd have to do something like replace A<num> with its covariant projection (maybe A<out num>). And since f has a contra-variant occurrence of the generic parameter, access to f would be blocked on a variable of type A<out num>. This is nice! The error is precisely at the location where things went wrong! But... use site variance gets pretty complicated. We've explored it, and we could pursue it, but there's a lot of both implementation and user facing complexity involved. Use site invariance maybe not so much, and there's an argument that the latter would fit well with Dart's runtime checked covariance. But still, complicated.

In a declaration site variance world, well.... it's depends on what the defaults are. As currently proposed, you wouldn't get any static errors at all with this program - you'd get the current Dart 3 behavior unless you explicitly opted into using variance on a class. But assuming you did choose to put a variance annotation on A, you might decide to make it a covariant class, and you'd change the declaration to class A<out X>. And you'd get a static error saying something like "Type parameter X can't be marked out, because it is used contra-variantly in the type of f. Try marking X inout or in or removing the variance annotation". And then, assuming you changed the declaration to class A<inout X>, you'd get another error on the initialization of a saying something like "A value of type A cannot be assigned to a location of type A because A's generic parameter is marked as invariant". It's always annoying to get static errors in a program, but these are very actionable, and very local errors. The error you get is fixable at the location of the error, and the error message can be pretty helpful.

So the point is that explicit variance (of one kind or another) moves the location of errors back to the point at which things went wrong. Which is good! It's makes fixing things much easier. In the process, we have, of course, cut out a bunch of programs which wouldn't have gone wrong. That's a real cost. And we've forced the user to think about something things that they might not have had to think about otherwise (like variance). Also a real cost.

So returning to your question about why choose to catch this one at compile time as opposed to others, I think the above is at least an attempt at an explanation of why I think these errors are good ones to try to catch. I don't know what other errors you had in mind, but if there are other categories of errors for which there are straightforward, well known, low overhead static analyses that can catch them, and they have lots of prior art in other programming languages, and they would make code faster, and they would better localize the errors.... I would most definitely consider that a strong argument in favor of doing those as well!

Answer 59 · 2023-05-19T02:51:15.000Z

Let's do user testing. It is the most powerful tool we have.

FWIW, I think there's a real danger in the line of reasoning of "other languages do it, so we can/should do it". A language is at least as much about the features it has, as it is about the features it doesn't have. It is ironically much harder to not add something than it is to add something, but it can often be much more important to the long-term health of the language. Antoine de Saint-Exupéry said it best, "perfection is attained not when there is nothing more to add, but when there is nothing more to remove".

Answer 60 · 2023-05-19T07:24:04.000Z

Erik Meijer and Gilad Bracha: Dart, Monads, Continuations, and More

[Gilad Bracha]: ... dare I say it, no I shouldn't say it, it's based on the Future Monad, but 
of course, no one needs to know that, and in fact, the more I say it, the less likely ...
[Erik Meijer]: ... exactly, now we kind of killed the idea, beautiful flowers, cut off at the roots ...

Maybe this issue could benefit from more accessible branding? Here's a different view that might not be as intimidating:

~~variance annotation~~ type parameter position (i.e. in, out, inout) -> the allowed position of a reference to a type parameter.
~~invariant~~ flexible type parameter position (e.g. <inout T>) -> T can be anywhere. T foo(int a, T b);
~~covariant~~ output type parameter position (e.g. <out T>) -> T can only be a return type. T foo(int a);
~~contravariant~~ input type parameter position (e.g. <in T>) -> T can only be a value parameter type. int foo(T a);
~~no explicit variance~~ unsafe type parameter position (e.g. <T>) -> you can do whatever you want, the analyzer and compiler won't help you avoid type parameter position related errors.
~~soundness~~ no accidental runtime errors -> The compiler can guarantee that code won't crash in unexpected ways.

I feel like the analyzer will be able to teach users everything else by surfacing misuse as errors. And I think that user testing of how the analyzer helps users debug variance related issues would be very helpful, but deciding whether this feature should be included in the language by evaluating how a random group of people deals with a basic implementation of it feels like user testing the concept of health insurance.

I think that the language team shouldn’t be coerced into feeling compelled to be less formal and precise, but the rigor in their specification also shouldn't be used as strong evidence that there's no user friendly implementation of that specification.

Answer 61 · 2023-05-20T14:20:32.000Z

It is ironically much harder to not add something than it is to add something, but it can often be much more important to the long-term health of the language. Antoine de Saint-Exupéry said it best, "perfection is attained not when there is nothing more to add, but when there is nothing more to remove".

@Hixie

The point is that variance is something we need to account for, but right now we do it at runtime rather than compile-time. We could argue that the whole type system is adding complexity, but in truth we are better modelling the intrinsic complexity of writing a computer program, we are focusing on the information that matters, reaching a more optimal abstraction for the job we need to do.

I understand your concern about keeping the language lean and useful, and I share your sentiment, I also don't want Dart to become like Scala or brainfuck.

As I see it, we are not adding yet-another-syntax for constructors, but rather we are fixing a bug in the type system. I see this at the same level as nullability, it add some extra syntax, but help programmers feeling safer within the boundaries of the type system.

Answer 62 · 2023-05-21T13:49:24.000Z

Reading this discussion made me reallize that there is a mistake which I made probably several times in the past, that could have been prevented by sound declaration-site variance but also made in a way more anoying.

Say for example I write an immutable list with generic parameter T which I would want to be covariant, because the problems with add and so on can not happen if the list is immutable and we have no add.

So I could begin with

class ImmuatableList<out T> {
  ...

  T operator [](int i) => ...;
}

Having sound variance here is nice because it prevents me from adding problematic methods like add.

However it also prevents me from adding completely reasonable methods like bool contains(T element) => ....
So far this is what I would have written without paying too much attention to it.

Now it is completely correct that this is also problematic in a scenario like:

ImmuatableList<num> l = ImmuatableList<int>(...);
l.contains(1.5);

Now to be fair these cases are quite rare, because most of the time genereric parameters are used in the same way you would with an invariant type.
But it is nice that the compiler would warn me from implementing contains this way because I want to write good, save and reusable code. However the alternative is to declare contains like this: bool contains(Object? element) => ..., which is also what the SDK does.

There are two problems with this though:

Firstly this prevents errors in common scenarios like this to be statically caught:

final l = ImmuatableList<int>(...);
l.contains('Some String');

And secondly it can make it unclear what the parameter should actually be. Now in the contains case it might not be quite as bad because everyone knows what this method does, but I can imagine more complex cases where this is no longer clear and would have to be specified in the docs.

There is a solution that I came up with, but admittedly it does have some flaws.

My Idea is to be able to mark the type in contains with some keyword. For example like:

  bool contains(in T element) => ...

This would make it such that the type for the caller is T but for the implementer it would behave like the top most possible type (in this case Object?. If we would have a class NumList<T extends num>, it would be num.)

This way it would be an error for a user to do something like

final l = ImmuatableList<int>(...);
l.contains('Some String'); // error

but not

ImmuatableList<num> l = ImmuatableList<int>(...);
l.contains(1.5); // ok

.
And the implementer would be forced to implement a case for non T arguments.

I know it would be confusing for a programmer that if they declare bool contains(in T element) =>, the argument element is not of type T but I can't think of a better solution. Good compiler erros would probably help a lot. Also the prefix in is debatable of course.

An analogous problem likely also exists for contravariant types, but I don't want to think about that right now :)

Sorry if this this problem has already been discussed. I didn't see anything on it so far.

Answer 63 · 2023-05-21T22:07:43.000Z

One option for contains is to declare it as

  bool contains<S super E>(S value) { 
    for (var e in this) if (e == value) return true; 
    return false; 
  }

This is basically the same as Object? value, since you can always write <Object?> anyway.

Another option is to make contains non-virtual (like an extension method), meaning that it uses the static type of the collection. That solves the problem, but also makes it impossible to override with a more efficient implementation.

A third option would be to allow capturing the static type at the call site in instance method invocations.
Say (complete strawman syntax):

  bool contains<T extends static E>(T value) {
    for (E e in this) if (e == value) return true;
    return false;
  }

which can be instantiated with any type argument which satisfies the static type of E at the call point, which may be a supertype of the actual type of E at runtime.
And if the value argument is already a subtype of that, there is no problem just passing the static type as the argument.
Or something along those lines.

The problem then becomes how we can migrate the platform libraries to exposing the new definition, without breaking existing classes implementing Iterable.

Answer 64 · 2023-05-22T22:12:27.000Z

@Hixie

Let's do user testing. It is the most powerful tool we have.

I'm happy to hear from @InMatrix here, but I am, as I said, pretty skeptical about this based in part on conversations I've had with him in the past. UX testing here would give us great insights into questions like "does the syntax inout T communicate more clearly to users the intent than the syntax = T", or "how easily can a user find an actionable response to this specific error message". I'm highly skeptical that it can give us meaningful data to the question of "Does declaration site variance, on balance, provide sufficient benefit to users over time (integrating over the entire learning and development cycle) in terms of more reliable, robust, and statically checked code to make up for the up front learning cost, barriers to entry, and the cost to users in terms of correct programs rejected by the conservative analysis".

FWIW, I think there's a real danger in the line of reasoning of "other languages do it, so we can/should do it".

That's not my reasoning. I do think that "many other languages do X" is a good reason to ask "why do many other languages do X, and should we consider doing X", but my main point above was that the experiences of users of other languages with a technology Y is probably by far the best UX data we can get for the cost/benefit tradeoffs of technology Y.

Answer 65 · 2023-05-23T01:28:27.000Z

I haven't read all of the 64 comments above this, but hopefully my responses related to user testing and research are on point.

@leafpetersen wrote:

I'm highly skeptical that it can give us meaningful data to the question of "Does declaration site variance, on balance, provide sufficient benefit to users over time (integrating over the entire learning and development cycle) in terms of more reliable, robust, and statically checked code to make up for the up front learning cost, barriers to entry, and the cost to users in terms of correct programs rejected by the conservative analysis".

We obviously can't answer this question directly in the context of Dart, since that requires us to predict the future. However, we might be able to get some directional signals for how similar features have been received by users of other languages. The data will be in the nature of user preferences and perception. For example, we can have Java/C#/Kotlin users rank the value/usability of the equivalent feature against other language features.

Taking a step back, if we want to spend any UX research resource on this proposal, we should probably start with measuring the size of the problem this can solve for our users, e.g., how would they rank it against other things they wish Dart had implemented or problems eliminated? Are they excited about having this class of error detected in compile time? If they ran into this kind of error at runtime, how did they feel about it? This is something we might have room for in the Q3 Flutter user survey. If I've missed such discussion in this thread, please point me to the relevant comments.

Once we establish the priority of the problem, UX research can help selecting competing design proposals (e.g., which syntax is easier to learn and more intuitive to understand) through usability studies.

Answer 66 · 2023-05-24T04:15:36.000Z

@InMatrix @Hixie

owever, we might be able to get some directional signals for how similar features have been received by users of other languages. The data will be in the nature of user preferences and perception. For example, we can have Java/C#/Kotlin users rank the value/usability of the equivalent feature against other language features.

This seems like it could be very useful.

Taking a step back, if we want to spend any UX research resource on this proposal, we should probably start with measuring the size of the problem this can solve for our users, e.g., how would they rank it against other things they wish Dart had implemented or problems eliminated? Are they excited about having this class of error detected in compile time? If they ran into this kind of error at runtime, how did they feel about it? This is something we might have room for in the Q3 Flutter user survey. If I've missed such discussion in this thread, please point me to the relevant comments.

Maybe. I'm certainly always happy to have more data, but I'm not sure how much we can really get here. I strongly suspect that very few users, when they encounter the runtime error "type 'String' is not a subtype of type 'int' of 'key'", immediately connect this up and say "Oh, darn that runtime checked covariance, I wish I had statically checked variance". So sure, we can ask questions about "have you seen this kind of error" (though unfortunately there are multiple paths to getting such an error, some of which don't really involve variance), but I'm worried it will be hard to draw any actionable conclusions from the responses there.

I do think it could be useful to gauge interest in the feature itself, just to get a sense of, for users who are already familiar with the feature from other languages, whether or not it's something they miss when coming to Dart.

Answer 67 · 2023-05-26T18:28:53.000Z

Sorry for the naive question, but if sound declaration-site variance is implemented, how more is missing to make Dart fully sound? Also Dart 3 is advertised as "100% sound null safety", does this mean something different than being sound?

Answer 68 · 2023-05-26T18:40:30.000Z

but if sound declaration-site variance is implemented, how more is missing to make Dart fully sound?

I've tried to school myself to use different terminology here, but we tend to slip. Properly speaking, this feature should probably be called "statically checked variance". Currently in Dart, we enforce soundness, but for variance, we do so via runtime checks (Java and Swift do similarly for arrays, I believe). So Dart is currently designed to be sound, in the formal sense ("an expression of static type T, if it evaluates to a value, evaluates to a value of some time S which is a subtype of T"). This feature is intended to move more of the checking that enforces soundness from runtime to compile time (much as null safety moved (some of the) null checking from runtime to compile time).

Answer 69 · 2023-05-26T21:07:56.000Z

I do think it could be useful to gauge interest in the feature itself, just to get a sense of, for users who are already familiar with the feature from other languages, whether or not it's something they miss when coming to Dart.

@leafpetersen I've filed b/284511831 internally to get this on our backlog. It's most likely that we can get to this in Q3 through the quarterly user survey, and if the timeline works for you, it would be good to include a few other features under consideration. If it's too late, it might be possible to get data from a smaller sample (e.g., fewer than 100 respondents) sooner.

Answer 70 · 2023-05-29T13:58:22.000Z

It is not simple to endusers because it is really not a simple thing, however, it is a must-have as a default language feature.

Answer 71 · 2023-06-16T03:27:45.000Z

We just recently migrated to null safety (dart 2.18), and we ran into some generic issues after the migration.

Map<String, String> map1 = {};
Map<String, String?> map2 = map1;
map2.putIfAbsent('key', () => null);  // compile ok, but runtime error

List<String> list1 = [];
List<String?> list2 = list1;
list2.add(null); // compile ok, but runtime error

The above is just simplified code, the actual business code is complex, for example, a method accepts a List<int?> argument, and then there are many callers, maybe one of the callers passing in a List<int> will cause an error. (By the way, is there any lint that can find this code?)

Our code was migrated to null safe, but because of this issue, we had to compile to a non-null safe version at compile time using the //@dart=2.9 flag to ensure online stability (We compiled to null safe in the test environment to expose this issue as much as possible, but there was no guarantee that all potential problems would be found. By the way, after moving to dart3, will the dart=2.9 mark be disabled?)

I hope to find this kind of problem at compile time.

Answer 72 · 2023-07-21T08:28:25.000Z

About 'unsound': I've used the phrases 'dynamically checked covariance' and 'statically checked variance' (any kind) for a long time, and I just noticed @leafpetersen's comment here, so I just changed the title and the text in the original posting to use that terminology.

Answer 73 · 2023-07-21T08:30:27.000Z

Note that run-time errors arising from failed dynamic covariance checks do come up now and then in actual code. Here are some recent examples:

Answer 74 · 2023-08-18T00:35:28.000Z

i'd like to 👍 this feature as well! it'd allow me to design safer APIs, which would prevent extremely difficult debugging sessions where runtime errors happen at very distant places in the codebase from where the actual programming happened. a single one of these debugging sessions is more frustrating and confusing than learning how variance works was for me, and i've had to do more than one of those sessions (and counting!), and i don't even learn anything from them!

Answer 75 · 2023-08-21T11:00:09.000Z

I use riverpod in most of my projects. Is the following problem related to this issue?
I always need to provide the exact generic type which seems to be redundant.

// inferred as NotifierProviderImpl<SettingsNotifier, SettingsState>
final settingsProvider = NotifierProvider<SettingsNotifier, SettingsState>(() {
  return SettingsNotifier();
});

// inferred as NotifierProviderImpl<SettingsNotifier, dynamic>
final settingsProvider2 = NotifierProvider(() {
  return SettingsNotifier();
});

class SettingsNotifier extends Notifier<SettingsState> {
  // ...
}

Answer 76 · 2023-08-21T12:54:53.000Z

@Tienisto I think yours is related to dart-lang/sdk#620.

Answer 77 · 2023-09-02T18:53:11.000Z

I think I ran into this today with Streams. If so, the lack of variance modifiers does lead to very confusing behavior.

final StreamController<num> numbers = StreamController();
final StreamController<double> doubles = StreamController();
doubles.stream.pipe(numbers);
doubles.add(123);

Intuition would lead me to think this should work. Since double is a subtype of num, a StreamConsumer<num> should be able to consume a Stream<double>. But the analyzer tells us:

The argument type 'StreamController<num>' can't be assigned to the parameter type 'StreamConsumer<double>'.

Further, the behavior I know to be incorrect is reversing the source and sink. Not every num is a double so piping a stream of num to a StreamConsumer<double> will inevitably fail.

final StreamController<num> numbers = StreamController();
final StreamController<double> doubles = StreamController();
numbers.stream.pipe(doubles);
numbers.add(123);

But the analyzer is appeased, and I don't find out til runtime that this is incorrect code.

TypeError: Instance of '_ControllerStream<num>': type '_ControllerStream<num>' is not a subtype of type 'Stream<double>'Error: TypeError: Instance of '_ControllerStream<num>': type '_ControllerStream<num>' is not a subtype of type 'Stream<double>'

Would love to see this feature so that both of these situations can be avoided!

Answer 78 · 2023-09-04T10:43:38.000Z

That's a very good example because StreamController (and several supertypes including StreamSink) are almost exclusively using their type parameter in contravariant positions, which means that the (unavoidable, standard) dynamically checked covariance is flat wrong almost every time.

However, we do have some covariant occurrences of the relevant type variables as well, so we'd need to make some classes invariant in order to get the statically safe typing, or we can keep them unchanged (using dynamically checked covariance) because they are already "mostly OK", and they are difficult to change (like Stream).

Here is an example (using some fake classes just to make the example small):

// Will probably continue to use dynamically checked covariance.
// Could also be invariant, but that's a massively breaking change.
class Stream<T> {
  Future pipe(StreamConsumer<T> _) => Future<Null>.value();
}

// Can be soundly contravariant (if `Stream` is dyn-covariant).
class StreamConsumer<in T> {
  Future addStream(Stream<T> stream) => Future<Null>.value();
}

// Needs to be invariant (to get a safe `add` and a safe `stream`).
class StreamController<inout T> extends StreamConsumer<T> {
  Stream<T> get stream => Stream<T>();
  void add(T t) {}
}

void main() {
  final StreamController<num> numbers = StreamController();
  final StreamController<double> doubles = StreamController();
  doubles.stream.pipe(numbers); // Sure, no problem! ;-)
  doubles.add(123);
}

With these changes, doubles.stream.pipe(numbers) type checks without errors, as it should.

As usual, we can emulate invariance (which is not quite as flexible as the real declaration-site variance feature where we can use contravariance where that's sufficient and invariance only where nothing else will do).

Here's a minimal example, where we only declare those few methods that we're actually calling here:

import 'dart:async' as async;

// Provide invariant types for some stream related classes.

typedef Inv<X> = X Function(X);
typedef Stream<X> = _Stream<X, Inv<X>>;
typedef StreamConsumer<X> = _StreamConsumer<X, Inv<X>>;
typedef StreamController<X> = _StreamController<X, Inv<X>>;

extension type _Stream<T, Invariance extends Inv<T>>(async.Stream<T> it) {
  Future pipe(StreamConsumer<T> consumer) => it.pipe(consumer.it);
  Stream<R> cast<R>() => Stream(it.cast<R>());
}

extension type _StreamConsumer<T, Invariance extends Inv<T>>(
  async.StreamConsumer<T> it
) {
  Future addStream(Stream<T> stream) => it.addStream(stream.it);
}

extension type _StreamController<T, Invariance extends Inv<T>>._(
  async.StreamController<T> it
) implements _StreamConsumer<T, Invariance> {
  _StreamController() : this._(async.StreamController<T>());
  Stream<T> get stream => Stream(it.stream);
  void add(T t) => it.add(t);
}

// Usage, in some other library (assume `import 'something';)

void main() {
  final StreamController<num> numbers = StreamController();
  final StreamController<double> doubles = StreamController();

  // doubles.stream.pipe(numbers); // Compile-time error.
  doubles.stream.cast<num>().pipe(numbers); // OK.
  doubles.add(123);
}

Note that cast on a stream is generally not safe (each data event on the stream has its type checked dynamically), but in this particular case it is an upcast, which implies that it is safe.

Finally, we should be able to use extension types together with statically checked declaration-site variance in order to be able to access the Stream related classes safely even without changing those classes themselves. However, that doesn't work at this time. Here's a hint:

// Provide soundly variant types for some stream related classes.

extension type StreamConsumer<in T>(async.StreamConsumer<T> it)
    implements async.StreamConsumer<T>;

extension type StreamController<inout T>._(
  async.StreamController<T> it
) implements async.StreamController<T> {
  StreamController() : this._(async.StreamController<T>());
}

[Edit, Sep 19, 2023: Add the bound Inv<T> on the Invariance type parameters, for improved typing of this.]

Answer 79 · 2023-09-05T18:26:24.000Z

That's very cool, thanks so much for sharing! Are any of the same tricks possible for replicating contravariance and static covariance in current Dart?

Answer 80 · 2023-09-05T18:48:02.000Z

I've been playing around with the experimental feature and it's awesome for these situations. I do, however, keep forgetting what in, out, and inout mean. I understand the argument that they're tied to the position of the type parameter in its usage, but the way I've been thinking about them is more about which types I want to allow T to be.

FWIW I would cast my vote for modifiers (as mentioned above) which make this explicit:

exact T -> invariant (inout)
super T/sup T -> T or any of its supertypes - contravariant (in)
sub T -> T or any of its subtypes - statically covariant (out)
dyn T -> dynamic covariance, same as T today but explicit

I have also stumbled into some situations where an in parameter can be used in a getter or method return. And, in general, there seems to maybe be some gray areas where the modifiers do not precisely map to the position of the type parameter's usage. Although, I could just be missing something here.

Answer 81 · 2023-09-06T10:57:02.000Z

@dnys1 wrote:

Are any of the same tricks possible for replicating contravariance and static covariance in current Dart?

The invariance emulation won't work out of the box: Contravariance should allow subtyping relationships like MyConsumer<num> <: MyConsumer<int>, but the regular type parameter (which is needed because we need to use that regular type parameter in the body of the class) will remain covariant, so we just get invariance again:

typedef _Ctv<X> = void Function(X); // `_Ctv<num> <: _Ctv<int>`.
typedef MyConsumer<X> = _MyConsumer<X, _Ctv<X>>;
  
class _MyConsumer<X, Contravariance> {
  void m(X x) {}
}

void main() {
  MyConsumer<int> intConsumer = MyConsumer<num>(); // Error, should have been OK.
}

We can't express statically checked covariance, either, because it's just going to be dynamically checked covariance as usual, and there will not be any errors for things like class C</*out*/ X> { void m(X x) {}}.

It's possible that there is some other technique, but someone would have to invent it, and tells us about it. ;-)

I do, however, keep forgetting what in, out, and inout mean.
I understand the argument that they're tied to the position of the type parameter in its usage, but ...

Right, that's always a difficult point. inout is easy because it includes all constraints associated with in as well as the ones for out. The usage perspective would be that in is only used for things that we "put into the receiver", such as method parameters, and out is only for things that "the receiver gives us", such as returned values.

With respect to the subtype relationships, I tend to think that in is used with "inverse" subtyping, that is MyConsumer<SuperType> <: MyConsumer<SubType>, and then it follows that out is the other case.

same as T today but explicit

We've considered using covariant for that: It is already associated with a typing which is best described as "yes, this is not safe, but I know what I'm doing, and it's so convenient!". ;-)

an in parameter can be used in a getter or method return

Sure, the covariant/contravariant/invariant positions in a type can occur in a return type, because that return type can have enough structure on its own to have both covariant and contravariant positions:

class C<in X> {
  void Function(X) get myGetter => (X x) {...};
}

You could say that X is still used for things that we "put into the receiver" because we invoke myGetter, and then we put an X into the function that the getter returned, which is in some sense "a function which is owned by the receiver".

In general, the variance of a position is defined recursively (PDF, see section 'Variance'), so you can always stack enough function types on top of each other to obtain a covariant/contravariant position anywhere in a function signature.

So it's not really a grey area, it's just a situation where the "inversion" that occurs whenever we switch from a function type to one of its parameter types occurs in a type that occurs in a member declaration signature, rather than at the top level of the signature. We can also invert twice and get back to covariance:

class D<out X> {
  final X x;
  D(this.x);
  X Function() myMethod(void Function(X) arg) => () => x;
}

Answer 82 · 2023-09-18T18:40:35.000Z

@eernstg

Regarding your proposed workaround, the following case won't work:

typedef Invariance<T> = T Function (T);

typedef Foo<T> = _Foo<T, Invariance<T>>;

final class _Foo<T, I> {
  const _Foo(this.bar);
  
  final _Bar<T> bar;
  
  void baz() => bar.baz(this);
}

final class _Bar<T> {
  void baz(Foo<T> foo) => print(foo);
}

I get the following error: The argument type '_Foo<T, I>' can't be assigned to the parameter type '_Foo<T, T Function(T)>'.

I could fix it by changing I so it extends from Invariance<T>:

-final class _Foo<T, I> {
+final class _Foo<T, I extends Invariance<T>> {

The error disappeared and I still get a static error if I try to pass covariant values to Foo.

Do you think this change is valid or can it introduce some other unexpected behavior?

Answer 83 · 2023-09-19T09:34:29.000Z

@mateusfccp, you are right. I actually used that approach a while ago (March 2023: dart-lang/sdk#51680 (comment)), I just got lazy in the meantime because the actual examples did not require this extra bit of type information. ;-)

But we can do it: It is benign for clients to have the bound, because they should always be using actual type arguments of the form <T, Invariance<T>> (and the use of a type alias would enforce this for clients outside the declaring library). It is also benign for a class like _Foo itself, because the value of I will indeed be a subtype of Invariance<T> in every case where the desired invariant is maintained.

It would still be possible to violate that invariant inside the declaring library, which means that it isn't precisely equivalent to the real language mechanism, but the emulation is better when we include this bound.

I changed the comment where the idiom is introduced accordingly.

Thanks!

Answer 84 · 2023-09-19T12:21:38.000Z

@eernstg

I am sorry if I am being annoying with this workaround, but I got to another problem while using it in my project.

Consider the following:

typedef Invariance<T> = T Function (T);

typedef Foo<T> = _Foo<T, Invariance<T>>;

abstract interface class _Foo<T, I extends Invariance<T>> {
  void baz();
}

mixin DefaultFoo<T> on Foo<T> {
  @override
  void baz() {}
}

final class Bar with DefaultFoo<Object> implements Foo<Object> {}

This code shows me two compile-time errors:

line 9: 'T' can't be used contravariantly or invariantly in '_Foo<T, T Function(T)>'.
line 14: 'DefaultFoo<Object>' can't be mixed onto 'Object' because 'Object' doesn't implement '_Foo<Object, Object Function(Object)>'.

I managed to fix the first error by also redirecting the mixin:

+typedef DefaultFoo<T> = _DefaultFoo<T, Invariance<T>>;
+
-mixin DefaultFoo<T> on Foo<T> {
+mixin _DefaultFoo<T, I extends Invariance<T>> on _Foo<T, I> {

However, the second error persists, and I couldn't find a way to circumvent this... I feel like Bar actually implements _Foo<Object, Object Function(Object)> because we explicitly use implement Foo<Object>, and Foo<Object> is the same as _Foo<Object, Object Function(Object)>, so I am not sure why the static analysis is talking about.

Is there a way or is it impossible to use this workaround with mixins?

Answer 85 · 2023-09-19T13:51:47.000Z

I am sorry if I am being annoying with this workaround,

No problem, of course! Try this:

typedef Invariance<T> = T Function (T);

typedef Foo<T> = _Foo<T, Invariance<T>>;

abstract interface class _Foo<T, I extends Invariance<T>> {
  void baz();
}

typedef DefaultFoo<T> = _DefaultFoo<T, Invariance<T>>;

mixin _DefaultFoo<T, I extends Invariance<T>> on _Foo<T, I> {
  @override
  void baz() {}
}

final class Bar extends Foo<Object> with DefaultFoo<Object> {}

Perhaps you didn't actually mean that the mixin should be on Foo<T>, because an on type of a mixin is useless if you never call super.something(...). If implements Foo<T> is indeed sufficient for your purpose then you can proceed as follows:

typedef Invariance<T> = T Function (T);

typedef Foo<T> = _Foo<T, Invariance<T>>;

abstract interface class _Foo<T, I extends Invariance<T>> {
  void baz();
}

typedef DefaultFoo<T> = _DefaultFoo<T, Invariance<T>>;

mixin _DefaultFoo<T, I extends Invariance<T>> implements _Foo<T, I> {
  @override
  void baz() {}
}

final class Bar with DefaultFoo<Object> implements Foo<Object> {}

It's worth noting that we must use _Foo as a superinterface of _DefaultFoo because we need to pass on the actual type argument I rather than just use Foo<T> directly. This implies that the idiom won't work with private classes if you wish to create a hierarchy of classes in multiple libraries (e.g., if _Foo is declared in one library and _DefaultFoo is declared in a different library).

In that case you'll have to admit to the world that the phantom type parameter (I) exists, such that they can pass it on correctly. (So you'd have typedef Foo<T> = FooWithPhantom<T, Invariance<T>>; for clients who just want to use the type, and FooWithPhantom<T, I extends Invariance<T>> for clients who wish to use it as a superinterface).

However, that just illustrates that we do need declaration site variance. ;-)

Answer 86 · 2023-10-18T15:42:36.000Z

@leafpetersen wrote:

Third, because it makes things faster.

Covariance checks on classes have a cost (sometimes significant). [...]

In order to demonstrate this point:

Micro-benchmark which measures the cost of covariant parameter type checks in Dart VM/AOT.

import 'package:benchmark_harness/benchmark_harness.dart';

const int N = 1000000;

class A {}
class B extends A {}

class C1<T> {
  @pragma('vm:never-inline')
  void foo(List<T> x) {
  }
}

class C2<T> {
  @pragma('vm:never-inline')
  void foo(List<T> x) {
  }
}

class TypeCheckBench extends BenchmarkBase {
  final C1<A> obj = int.parse('1') == 1 ? C1<B>() : C1<A>();
  final List<A> arg = int.parse('1') == 1 ? <B>[] : <A>[];

  TypeCheckBench() : super('TypeCheck');

  @override
  void run() {
    for (int i = 0; i < N; ++i) {
      obj.foo(arg);
    }
  }
}

class NoTypeCheckBench extends BenchmarkBase {
  final C2<A> obj = C2<A>();
  final List<A> arg = <A>[];

  NoTypeCheckBench() : super('NoTypeCheck');

  @override
  void run() {
    for (int i = 0; i < N; ++i) {
      obj.foo(arg);
    }
  }
}

void main() {
  final benchmarks = [TypeCheckBench(), NoTypeCheckBench()];

  for (var benchmark in benchmarks) {
    benchmark.report();
  }
}

In this benchmark, an empty instance method with one List<T> parameter is called in a loop. The inlining of the method is disabled so it would not be optimized out.

In one case (TypeCheck) the compiler cannot prove that a covariant parameter type check always succeeds, so the check is performed at run time. In another case (NoTypeCheck) the check is eliminated.

Results:

JIT, x64:

TypeCheck(RunTime): 50392.05128205128 us.
NoTypeCheck(RunTime): 25101.63953488372 us.

AOT, x64:

TypeCheck(RunTime): 41870.67796610169 us.
NoTypeCheck(RunTime): 17317.603603603602 us.

JIT, arm64:

TypeCheck(RunTime): 37042.545454545456 us.
NoTypeCheck(RunTime): 33444.29508196721 us.

AOT, arm64:

TypeCheck(RunTime): 21953.68085106383 us.
NoTypeCheck(RunTime): 9342.937219730942 us.

With statically checked declaration-site variance, if type parameter T of a class is declared as invariant, then runtime checks of the parameters involving T are not needed (because soundness is enforced statically). That would guarantee that parameters of generic types don't have any additional hidden performance overhead. I think this is also a very strong point towards making type parameters invariant by default.

Answer 87 · 2024-03-21T09:39:14.000Z

Any updates on the status of this feature for release?

Answer 88 · 2024-03-21T10:03:41.000Z

@SandroMaglione

I don't think the team has any ETA, but the feature is behind an experimental flag (--enable-experiment=variance).

It doesn't seems to be something the team is prioritizing, tho.

Answer 89 · 2024-08-01T08:34:38.000Z

First draft of feature specification at PR dart-lang/sdk#557.

I think the link should be #557 (dart-lang/language).

Same for the other links in the issue description.

Answer 90 · 2024-08-07T07:04:32.000Z

I don't think the team has any ETA, but the feature is behind an experimental flag (--enable-experiment=variance).

Unfortunately the experiment is also unusable... The following basic snippet fails:

// dummy class with contravariant type parameter
class Consumer<in T> {
  void consume(T item) {}
}

final List<Consumer<String>> consumers = [];
Consumer<String> consumer = Consumer<Object>();  // OK, both static analysis and runtime
consumer.consume("hello");  // OK, both static analysis and runtime
print(consumer is Consumer<String>);   // static analysis says this is always true, runtime prints false
consumers.add(consumer);  // runtime error: "Consumer<Object> is not a Consumer<String>" (but it is)

Answer 91 · 2024-08-07T09:10:46.000Z

Consumer<String> consumer = Consumer<Object>();  // OK, both static analysis and runtime

this feels more like a bug than a new feature

Answer 92 · 2024-08-07T09:26:08.000Z

Consumer<String> consumer = Consumer<Object>(); // OK, both static analysis and runtime
this feels more like a bug than a new feature

The bug is the current behaviour:

// no variance annotation, defaults to covariant
class Consumer<T> {
  void consume(T item) {}
}

Consumer<Object> consumer = Consumer<String>();  // OK, both static analysis and runtime
consumer.consume(123);  // static analysis ok, fails at runtime: we passed an int to a method which expects a String

It might feel strange because our intuition tells us that generics are covariant, and in some cases this is correct (if the type is an "output" type, as a rule of thumb). But some generics really should be contravariant (if the type is used for input only) or invariant (if the type is used for both input and output).

For instance, a consumer that accepts strings is not a consumer of objects, because it doesn't accept objects that are not strings. Conversely, a consumer that accepts any kind of object is certainly a consumer of strings, because you can pass it a string and it will deal with it no problem.

Unfortunately, Dart not only lacks variance annotation but defaults to covariant parameters, which leads to type unsoundess like the above example. A better default in terms of type safety would have been invariant.

Answer 93 · 2024-08-21T19:06:37.000Z

Why these have be keywords? Why not go with typescript way? It allows much more ways to expand. Like for example doing things like ReturnType. Maybe doing something like All objects except num and string.