CSharpSourceGeneratorTest<T> should not care about GeneratedSources input order.

[grosch-intl]

Using version 1.1.2-beta1.23509.1 of the code generation testing assemblies, it appears that the GeneratedSources is being compared in order, as opposed to just having the same contents. That's a problem as my test cases shouldn't have to care about the order in which files are generated, just that they are generated.

The below code results in a failure. If, instead, I move the ScopedInterfaceGeneratorExtensions.g.cs into the TestAsync method, and add it after the expected class file is added, then I don't get a failure about generated sources not matching.

The code generator is first adding LampGeneratorAttributes.g.cs, then the class specific file, and then the ScopedInterfaceGeneratorExtensions.g.cs file.

class SourceGeneratorTest : CSharpSourceGeneratorTest<LampCodeGenerator, XUnitVerifier> {
    private readonly string _generatedAttributes = ...";

    public SourceGeneratorTest() {
        TestState.ReferenceAssemblies = ReferenceAssemblies.Net.Net70;

        TestState.GeneratedSources.Add((Path.Combine(_pathPrefix, "LampGeneratorAttributes.g.cs"), _generatedAttributes));
        TestState.GeneratedSources.Add((Path.Combine(_pathPrefix, "ScopedInterfaceGeneratorExtensions.g.cs"), "..."));
    }

    public static Task TestAsync(string input, string className, string expected) {
        var test = new SourceGeneratorTest {
            TestCode = input
        };
        test.TestState.GeneratedSources.Add((Path.Combine(_pathPrefix, $"{className}.g.cs"), expected));
    
        return test.RunAsync();
    }
}

[sharwell]

Generator outputs are not order-sensitive starting with #988, which is definitely included in the version you mentioned. It seems likely that there was no order of files which allowed the result to match.

Can you provide a more complete repro case here, along with the exact failure message?

[grosch-intl]

@sharwell It's not working properly then. If I do nothing else related to my test other than move the order of the GeneratedSources line, it fails or works appropriately.

This appears in the output:

LaMP.CodeGenerator\LaMP.CodeGenerator.LampCodeGenerator\LampGeneratorAttributes.g.cs
+LaMP.CodeGenerator\LaMP.CodeGenerator.LampCodeGenerator\Foo.g.cs
LaMP.CodeGenerator\LaMP.CodeGenerator.LampCodeGenerator\ScopedInterfaceGeneratorExtensions.g.cs
-LaMP.CodeGenerator\LaMP.CodeGenerator.LampCodeGenerator\Foo.g.cs

So you can see that it knows the Foo.g.cs file is there, it just doesn't like "where" it is.

[sharwell]

Can you provide a more complete repro case? We have tests for this feature and multiple repositories that depend on it allowing reordered inputs.

This appears in the output:

This doesn't necessarily mean that reordering was the only problem. Whenever it can't find an exact match from expected to actual, it tries to find the "closest" match according to some arbitrary comparisons and weights. For some reason, it thinks the content your test provides for ScopedInterfaceGeneratorExtensions.g.cs is a better match for Foo.g.cs, and vice versa.

[grosch-intl]

Ah, OK, that makes sense then as to why it'd be failing. Is there a way to tell it, "You're going to see files X, Y and Z generated, but I don't want you to look at X or Z, just Y"?

[sharwell]

Not currently. The options are telling it to ignore all of the generated files, or providing the full list of generated files.

[grosch-intl]

Is it in plan as an enhancement? That would make things quite a bit simpler. In my case, I have three separate files that are auto-generated just as a side-effect of having the code generator, and then the rest are done via attributes.

Sure, I could make my "automatic" stuff all go to a single file, but that just feels wrong :)