jeffcampbellmakesgames/Entitas-Redux

[RESEARCH] Investigate alternative object cloning libraries for copying components, blueprints

jeffcampbellmakesgames opened this issue · 4 comments

jeffcampbellmakesgames commented

Is your feature request related to a problem? Please describe.

I'd like for copying components, blueprints to be simpler and easier for developer to solve without needing to implement ICloneable for custom reference types to determine how a deep clone should occur (or at least make that optional for potential performance improvements).

Describe the solution you'd like

There are some third-party libraries I could add as a dependency to simplify deep clones of component members, but they would likely need:

  • Potentially may not compile for Unity and/or need additional work to be able to compile for Unity.
  • May not work on specific platforms with constrained resources and functionality like WebGL.
  • Constraints or filtering added to be able to handle/exclude native Unity ref types derived from UnityEngine.Object like MonoBehavior, ScriptableObject, Texture2D, etc...

Describe alternatives you've considered
I'd rather not duplicate the efforts at writing another homebrew deep-cloning solution for C# if reasonable alternative exist that are either maintained or that I could fork and maintain for Unity/C#.

Additional context
This is something worthwhile to look into, but nothing I would hold up the blueprints feature for on a release. On second thought I would like to add this to blueprints on the first release if possible because it would remove many of the cloning edge cases a developer would need to take into account and simplify the implementation.

jeffcampbellmakesgames commented

As I've investigated this further, DeepCopy seems to me to be one of the more robust, performant options for adding this capability for component copying. There are a few challenges to overcome, particularly for il2cpp platforms.

Preventing Code Stripping
This is one of the easier areas as adding the [Preserve] attribute throughout the forked library and adding a link.xml file to prevent aggressive code stripping from removing required functionality have resolved this.

Ahead-Of-Time (AOT) Support
This is a challenging area; the crux of it is that for il2cpp to support generic methods it needs to know what typed variants of these there are prior to generation so that it can create the corresponding c++ code to support them. Because DeepCopy largely uses expression trees to generate the copy code and generic methods are not called directly, these variants need to be discovered and created in a way where il2cpp can discover them, even if the generated code used for il2cpp discovery is never used. An example of this might be something like this.

[Preserve]
public class AOTSupport
{
    private void Foo()
    {
        GenericMethodOne<int>();
        GenericMethodTwo<int[]>();
        GenericMethodOne<ExampleStruct>();
        GenericMethodOne<ExampleClass>();
    }
}

At the moment, my thought for how to accomplish AOT support would be to execute a pre-build processor for discovery and code-gen with the following steps.

  • Use Roslyn code-analysis to discover all places where ShallowClone, DeepClone are being called in all scripts, assemblies and collect the type information of the object where those methods are called.
    • For all types, filter into groups for:
      • Get all array types, including multidimensional arrays or arrays of arrays.
      • Get all array element types.
      • Get all interface types
      • Get all closed types, including array element types.
  • For those types and any relevant fields on them, create the appropriate generic method variant code snippets and add to an AOTSupport class.
    • For all array types, create generic array copy snippet based on array rank
    • For all closed types, create generic clone struct or class variant snippet.
    • For all interface types, either:
      • If contained in whitelist for code-gen, find all types implementing interface and create generic snippets appropriate for it and all field types.
      • If not in whitelist, log to the console a warning about that type and that it may not have AOT support for il2cpp builds.
  • Generate and import the AOTSupport class.
  • After the build, delete the AOTSupport class from the project.
jeffcampbellmakesgames commented

Unfortunately, I am going to place this on the backburner until new avenues of AOT support open up for generics or I get some fresh inspiration on how to attack this. I've gotten this fairly close to working in all circumstances, but there are a few areas I don't see a way to work around for now.

Providing AOT support for generic usage of private types
To be able to call generic methods via expression trees in an il2cpp Unity app, you need to have AOT support such that Unity knows to compile that specific generic variant ahead of time. I was able to add a level of support for this using Roslyn to discover 99% of all types, including member types and generate the appropriate generic variants like so:

// Class Generic Calls
JCMG.DeepCopyForUnity.DeepClonerGenerator.GetClonerForValueType<JCMG.DeepCopyForUnity.PlayModeTests.ArrayTests.AC>();
JCMG.DeepCopyForUnity.DeepClonerGenerator.GetClonerForValueType<JCMG.DeepCopyForUnity.PlayModeTests.ConstructorsTests.C3>();
JCMG.DeepCopyForUnity.DeepClonerGenerator.GetClonerForValueType<JCMG.DeepCopyForUnity.PlayModeTests.ConstructorsTests.C4>();
JCMG.DeepCopyForUnity.DeepClonerGenerator.GetClonerForValueType<JCMG.DeepCopyForUnity.PlayModeTests.ConstructorsTests.ClonableClass>();
JCMG.DeepCopyForUnity.DeepClonerGenerator.GetClonerForValueType<JCMG.DeepCopyForUnity.PlayModeTests.ConstructorsTests.ExClass>();
JCMG.DeepCopyForUnity.DeepClonerGenerator.GetClonerForValueType<JCMG.DeepCopyForUnity.PlayModeTests.ConstructorsTests.T1>();
JCMG.DeepCopyForUnity.DeepClonerGenerator.GetClonerForValueType<JCMG.DeepCopyForUnity.PlayModeTests.ConstructorsTests.T2>();

However, one area that seems very difficult if not impossible to overcome is when a member type that might need to be copied is a nested private type like so:

public class Foo
{
    private class Bar    {    }

    private Bar _bar;
}

Generic AOT support can't be provided in this same way alone as there isn't access to that private type and thus when an expression tree is generated that would create a generic method variant to copy this field it will result in an AOT exception. This example comes from the private Dictionary<T, TV>.Entry struct.

System.ExecutionEngineException : Attempting to call method 'JCMG.DeepCopyForUnity.DeepClonerGenerator::Clone1DimArraySafeInternal<System.Collections.Generic.Dictionary`2+Entry[[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089],[System.String, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]>' for which no ahead of time (AOT) code was generated.

Generic Function Generation at Runtime
Another issue with this particular library is that internally it makes use of cached expression trees compiled as generic methods. While issues don't seem to exist for classes, which internally all share a signature of Func<object, DeepCloneState, object> and make use of typed casting when passing in parameters, getting return values, value types like structs use specific typed variants like Func<StructOne, DeepCloneState, StructOne> which because this method is generated at runtime there isn't a way to provide AOT support for beforehand. I'd be curious to learn more about why this works for classes and not for structs; I believe it may have something to do with the class variants sharing a method signature, but I'm not sure (here for more details).

Summary
Overall this is a worth goal to pursue and with additional Unity Mono/il2cpp upgrades this may become more possible to do without so much AOT support, but for now I am tabling this in pursuit of other features, improvements.

jeffcampbellmakesgames commented

Closing this for now until new developments arise to solve some of these issues.