Dearchiver Improvements

[TheMostDiligent]

In current implementation, all resources required by a pipeline state (shaders, render passes, resource signatures) must be packed into the same archive. A packager will fail to process the pipeline if some information is missing. This setup has a downside: shared resources may be duplicated in multiple archives and may require extra processing at run time. For example, if multiple pipeline states share the same resource signature, this signature must be packed into each archive. It will also be duplicated multiple times when PSOs are unpacked from the archive at run time.

Proposed API changes

• Add PSO_ARCHIVE_FLAG_ALLOW_OPEN_DEPENDENCIES flag that will indicate that a pipeline state may be packed with open dependencies
• Replace IEngineFactory::GetDearchiver with IEngineFactory::CreateDearchiver method
  • DearchiverCreateInfo struct should contain a pointer to the device, so that it can be removed from PipelineStateUnpackInfo, RenderPassUnpackInfo, and ResourceSignatureUnpackInfo.
• Remove IDearchiver::CreateDeviceObjectArchive method and add IDearchiver::LoadStates method
  • IDeviceObjectArchive interface can likely be removed completely
  • The LoadStates method can either take a pointer to IArchive interface, or the pointer to the raw data and its size directly. In the latter case IArchive interface may not be necessary
• pArchive member should be removed from PipelineStateUnpackInfo, RenderPassUnpackInfo, and ResourceSignatureUnpackInfo structs.

In the new API, the packager will be allowed to pack pipeline states with missing dependencies when PSO_ARCHIVE_FLAG_ALLOW_OPEN_DEPENDENCIES flag is specified.
The dearchiver will become a stateful object and will allow loading multiple archives using the LoadStates method. When unpacking a PSO, the dearchiver will search dependencies in all archives it has loaded.

Implementation notes

• Make Dearchiver implementation a stateful object and move functionality of DeviceObjectArchiveBase to the dearchiver
  • In particular, all resource caches will move to the dearchiver implementation
• In the dearchiver implementation, maintain a list of archives that have been loaded
  • Load the list of resource names and make sure there is no duplication
  • Lazily unpack states as requested by the user: when a PSO misses some dependencies, walk back the list of loaded archives and try to find the missing states there. Fail with an error if the state is not found in any archive
  • Replace DeviceObjectArchiveBase with an internal object that will contain a pointer to the IArchive and some additional data (TBlockBaseOffsets, m_DebugInfo, etc.)

Add Unit tests that cover the new functionality:

• Dearchiver: loading PSO with dependencies in multiple archives (e.g. shaders - archive 0, signature - archive 1, PSO - archive 2)
• Render state packager: using PSO_ARCHIVE_FLAG_ALLOW_OPEN_DEPENDENCIES flag