Update the `UnityFileSystem` library checked in the repo

Question

Update the `UnityFileSystem` library checked in the repo

Opened this issue a year ago · 0 comments

The version of UnityFileSystem.{so, dylib, dll} checked into the repo should be updated to match newer versions of Unity. Since the library rejects archives built with newer versions of the Unity Editor.

For example, when running UnityDatatool locally, I get the following errors. (Which seems reasonable and more or less expected.)

Expected version: 2023.1.0a16. Actual version: 2022.3.4f1.
System.NotSupportedException: SerializedFile version not supported.

Full error message:

% ./UnityDataTool analyze "/unity-assets/"
objc[18223]: Class PlaceholderObject is implemented in both /src/github.com/Unity-Technologies/UnityDataTools/UnityDataTool/bin/Release/net6.0/osx-x64/UnityDataTool (0x100922ef0) and /src/github.com/Unity-Technologies/UnityDataTools/UnityDataTool/bin/Release/net6.0/osx-x64/libSystem.Native.dylib (0x109efd638). One of the two will be used. Which one is undefined.
Processing 3% (1/26) /unity-assets/Data/globalgamemanagersInvalid serialized file version. File: "/unity-assets/Data/globalgamemanagers". Expected version: 2023.1.0a16. Actual version: 2022.3.4f1.
Unknown error occurred while loading '/unity-assets/Data/globalgamemanagers'.

... snip ...

Error processing file /unity-assets/Data/Managed/Metadata/global-metadata.dat!
System.NotSupportedException: SerializedFile version not supported.
   at UnityDataTools.FileSystem.UnityFileSystem.HandleErrors(ReturnCode returnCode, String filename) in /src/github.com/Unity-Technologies/UnityDataTools/UnityFileSystem/UnityFileSystem.cs:line 75
   at UnityDataTools.FileSystem.UnityFileSystem.OpenSerializedFile(String path) in /src/github.com/Unity-Technologies/UnityDataTools/UnityFileSystem/UnityFileSystem.cs:line 49
   at UnityDataTools.Analyzer.SQLite.SQLiteWriter.WriteSerializedFile(String filename, String folder) in /src/github.com/Unity-Technologies/UnityDataTools/Analyzer/SQLite/SQLiteWriter.cs:line 168
   at UnityDataTools.Analyzer.AnalyzerTool.Analyze(String path, String databaseName, String searchPattern, Boolean skipReferences) in /src/github.com/Unity-Technologies/UnityDataTools/Analyzer/AnalyzerTool.cs:line 49
Processing 73% (19/26) /unity-assets/Data/Raw/AB/cap0007_f00_s0.abiFailed to load '/unity-assets/Data/Raw/AB/cap0007_f00_s0.abi'. File may be corrupted or was serialized with a newer version of Unity.

Note the warning about "Class PlaceholderObject" being implemented in two libraries. I assume that is benign because of the static linking when building with -p:PublishSingleFile=true -p:UseAppHost=true?

Errors When Updating

From the README.md, it sounded like fixing this should just be a matter of referencing newer versions of the UnityFileSystemApi library. However, this doesn't pan out.

I downloaded the latest version of the Unity Editor. And confirmed there was a UnityFileSystemApi.dylib file in the /Tools directory. (But no .dll or .so, which makes sense since I'm on a Mac.)

After copying the new UnityFileSystemApi.dylib into the same folder as UnityDataTool, or setting DYLD_LIBRARY_PATH="/Applications/Unity/Unity.app/Contents/Tools" it appears that the library can no longer be loaded.

There is the same error about Class PlaceholderObject, but a new one Unable to load shared library 'UnityFileSystemApi' or one of its dependencies..

./UnityDataTool analyze "/unity-assets/Data/"
objc[17153]: Class PlaceholderObject is implemented in both /src/github.com/Unity-Technologies/UnityDataTools/UnityDataTool/bin/Release/net6.0/osx-x64/UnityDataTool (0x1009dfef0) and /src/github.com/Unity-Technologies/UnityDataTools/UnityDataTool/bin/Release/net6.0/osx-x64/libSystem.Native.dylib (0x109ffe638). One of the two will be used. Which one is undefined.
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'UnityFileSystemApi' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libUnityFileSystemApi, 0x0001): tried: 'libUnityFileSystemApi' (no such file), '/System/Volumes/Preboot/Cryptexes/OSlibUnityFileSystemApi' (no such file), '/usr/lib/libUnityFileSystemApi' (no such file, not in dyld cache), 'libUnityFileSystemApi' (no such file), '/usr/local/lib/libUnityFileSystemApi' (no such file), '/usr/lib/libUnityFileSystemApi' (no such file, not in dyld cache)
   at UnityDataTools.FileSystem.DllWrapper.Init()
   at UnityDataTools.FileSystem.UnityFileSystem.Init() in /src/github.com/Unity-Technologies/UnityDataTools/UnityFileSystem/UnityFileSystem.cs:line 12
   at UnityDataTools.UnityDataTool.Program.Main(String[] args) in /src/github.com/Unity-Technologies/UnityDataTools/UnityDataTool/Program.cs:line 16
   at UnityDataTools.UnityDataTool.Program.<Main>(String[] args)
zsh: abort      ./UnityDataTool analyze

Since UnityFileSystemApi.dylib clearly exists, I'm guessing the error is actually because of the "or one of its dependencies" part? However, it's unclear what dependency it is looking for.

In case it was related, I did check that Terminal.app can "run software locally that does not meet the system's security policy". (In case the error is from the .dylib file not being signed or something.) But that didn't seem to change anything.

Summary

The UnityFileSystemApi libraries checked into this repo are now a little dated, meaning that out of the box you cannot build and run the tool to inspect newer asset files.

In theory, it's just a matter of checking in a newer build of those libraries into the repo. However, at least with my understanding of things, copying over newer versions of the files isn't quite so easy.