Physis

Physis is a library for reading and writing FFXIV data.

Even though this library was written with and for Rust in mind, Physis has bindings for other languages:

PhysisSharp can be used in any C# application, and is built on top of libphysis.
libphysis can be used for anything that can interface with the C FFI. Novus and Astra is built on top of libphysis, for example.

Goals

Physis should:

... Make it easy for developers to tinker around with game data.
... Document file formats along the way, to make writing future and applications libraries easier.
... Make parsing data safe, and create automated tests when possible.
... Aim to be have minimal dependencies, and easy to use regardless of platform.

Features

Here is a list of supported formats and their status:

File Format	Read	Write	Note
Configuration files	✅	✅
Saved character data	✅	❌	Only some versions are currently supported.
Chara make params	✅	❌
Dictionaries	✅	❌
Excel data	✅	❌
File infos	✅	✅
Map layers	✅	❌	Layer support isn't well tested yet.
Chat logs	✅	❌	Not all chat categories are discovered yet.
Models	✅	✅	Adding custom shape keys aren't fully supported yet.
Materials	✅	❌
Patch files	✅	❌
Pre bone deformers	✅	❌
Shader packages	✅	❌
Skeletons	✅	❌
Terrain	✅	❌
Textures	✅	❌	Only some formats are supported.

Physis also supports doing some other useful things other than reading and writing file formats:

Extract game data from SqPack files, and list file hashes from index/index2.
Apply game patches. Indexed ZiPatch is not yet supported, though.
Blockfish ciphers for encrypting and decrypting SqexArg.
Extract retail installer contents, useful on Linux for avoiding having to run the InstallShield installer.
Construct paths to equipment, items, faces, and other useful models.
Extract strings from executables.

Supported Game Versions

Version	Works	Note
A Realm Reborn (2.x)	???	Have yet to test.
Heavensward (3.x)	???	Have yet to test.
Stormblood (4.x)	???	Have yet to test.
Shadowbringers (5.x)	~	Not tested in a while but shuld work..
Endwalker (6.x)	✅
Dawntrail (7.x)	~	Limited support for the benchmark, awaiting retail release.

Note that only Windows versions of the game is supported at the moment.

Usage

If you want to use Physis in your Rust project, you can simply add it as a dependency in Cargo.toml:

[dependencies]
physis = "0.2"

Documentation is available online at docs.xiv.zone. It's automatically updated as new commits are pushed to the main branch.

C# projects can use PhysisSharp which exposes Physis in C#.

C/C++ projects (or anything that can interface with C libraries) can use libphysis.

Building

Physis only has a few dependencies, and very little if nothing is turned on by default. You need to set up Rust and then run cargo build.

If you want to build the game_install feature, you also need to install unshield.

Development

If you're interested to read how these formats work in more detail, see xiv.dev and docs.xiv.zone.

Testing

One of the main goals of Physis is to avoid accidental regressions, this is especially important when handling game data that might take hours to download.

Unit Testing

There are a set of basic unit tests you can run via cargo test. You can also find the relevant test resources in resources/tests. This does NOT contain copyrighted material, but actually fake game data created by physis itself. These tests are run automatically by the CI.

Retail Testing

There are some tests and benchmarks require the environment variable FFXIV_GAME_DIR to be set. By default, these are disabled since they require a legitimate copy of the retail game data. These tests can be turned on via the retail_game_testing feature.

Patch Testing

Patching is an extremely sensitive operation since it is not easily reversible if done wrong. Repairing the game files is an option, but it's time-consuming and not yet implemented in physis. To prevent regressions in patching the game, I have set up a testing bed for cross-checking our implementation with others. Currently, this is limited to XIVLauncher's implementation, but I will eventually adopt a way to test the retail patch installer as well.

Enable the patch_testing feature.
Set a couple of environment variables:
- FFXIV_PATCH_DIR is the directory of patches to install. It should be structured as $FFXIV_PATCH_DIR/game/D2017.07.11.0000.0001.patch.
- FFXIV_XIV_LAUNCHER_PATCHER should be the path to the XIVLauncher patcher executable. If you're running on Linux, we will handle running Wine for you.
- FFXIV_INSTALLER is the path to the installer executable. This will be installed using the usual InstallShield emulation physis already includes.

As you can see, you must have the previous patches downloaded first as well as the installer before running the tests. This is left up to the developer to figure out how to download them legally.

Note: These tests create the game_test and game_test_xivlauncher folders in $HOME and does not delete them on exit, in case you want to check on the results. You may want to remove these folders as they are full game installations and take up a considerable amount of space.

Semver and Dependency Checks

Even though package management in Rust is easier, it's a double-edged sword. I try to prevent getting carried away from including crates - but the ones we do include, have to get checked often. I use cargo deny to check my dependencies for mismatched versions, deprecation warnings, updates and more. This is also run on the CI!

Making sure that the library is semver-compliant is also important, and I use cargo semver for this task. This is to ensure the API does not break when moving between patch versions.

Contributing & Support

The best way you can help is by monetarily supporting me or by submitting patches to help fix bugs or add functionality. Filing issues is appreciated, but I do this in my free time so please don't expect professional support.

Credits & Thank You

goatcorp (XIVQuickLauncher, docs.xiv.dev, and even more)
Ioncannon (FFXIV Data Explorer) for the first documenting the file formats
binrw team for an awesome Rust library!
sha1-smol for a dependency-free SHA1 implementation

And everyone else who writes FFXIV tools!

License

This project is licensed under the GNU General Public License 3. Some parts of the code or assets may be licensed differently, refer to the REUSE metadata.

redstrate/Physis