/ShaderMake

Shader Compilation Tool

Primary LanguageC++MIT LicenseMIT

ShaderMake

Build Status

ShaderMake is a frond-end tool for batch multi-threaded shader compilation developed by NVIDIA DevTech. It is compatible with Microsoft FXC and DXC compilers by calling them via API functions or executing them through command line.

Features:

  • Generates DXBC, DXIL and SPIR-V code.
  • Outputs results in 3 formats: native binary, header file, and a binary blob containing all permutations for a given shader.
  • Minimizes the number of re-compilation tasks by tracking file modification times and include trees.

During project deployment, the CMake script automatically searches for fxc and dxc and sets these variables:

  • FXC_PATH - fxc from Windows SDK
  • DXC_PATH - dxc from Windows SDK
  • DXC_SPIRV_PATH - dxc with enabled SPIRV generation from Vulkan SDK

Command line options

Usage:

ShaderMake.exe -p {DXBC|DXIL|SPIRV} --binary [--header --blob] -c "path/to/config"
        -o "path/to/output" --compiler "path/to/compiler" [other options]
        -D DEF1 -D DEF2=1 ... -I "path1" -I "path2" ...

    -h, --help                show this help message and exit

Required options:

  • -p, --platform=<str> - DXBC, DXIL or SPIRV
  • -c, --config=<str> - Configuration file with the list of shaders to compile
  • -o, --out=<str> - Output directory
  • -b, --binary - Output binary files
  • -h, --header - Output header files
  • -B, --binaryBlob - Output binary blob files
  • -H, --headerBlob - Output header blob files
  • --compiler=<str> - Path to a FXC/DXC compiler

Compiler settings:

  • -m, --shaderModel=<str> - Shader model for DXIL/SPIRV (always SM 5.0 for DXBC)
  • -O, --optimization=<int> - Optimization level 0-3 (default = 3, disabled = 0)
  • --WX - Maps to '-WX' DXC/FXC option: warnings are errors
  • --allResourcesBound - Maps to -all_resources_bound DXC/FXC option: all resources bound
  • --PDB - Output PDB files in out/PDB/ folder
  • --stripReflection - Maps to -Qstrip_reflect DXC/FXC option: strip reflection information from a shader binary
  • --matrixRowMajor - Maps to -Zpr DXC/FXC option: pack matrices in row-major order
  • --hlsl2021 - Maps to -HV 2021 DXC option: enable HLSL 2021 standard

Defines & include directories:

  • -I, --include=<str> - Include directory(s)
  • -D, --define=<str> - Macro definition(s) in forms 'M=value' or 'M'

Other options:

  • -f, --force - Treat all source files as modified
  • --sourceDir=<str> - Source code directory
  • --relaxedInclude=<str> - Include file(s) not invoking re-compilation
  • --outputExt=<str> - Extension for output files, default is one of .dxbc, .dxil, .spirv
  • --serial - Disable multi-threading
  • --flatten - Flatten source directory structure in the output directory
  • --continue - Continue compilation if an error is occured
  • --useAPI - Use FXC (d3dcompiler) or DXC (dxcompiler) API explicitly (Windows only)
  • --colorize - Colorize console output
  • --verbose - Print commands before they are executed

SPIRV options:

  • --vulkanVersion=<str> - Vulkan environment version, maps to -fspv-target-env (default = 1.3)
  • --spirvExt=<str> - Maps to -fspv-extension option: add SPIR-V extension permitted to use
  • --sRegShift=<int> - SPIRV: register shift for sampler (s#) resources
  • --tRegShift=<int> - SPIRV: register shift for texture (t#) resources
  • --bRegShift=<int> - SPIRV: register shift for constant (b#) resources
  • --uRegShift=<int> - SPIRV: register shift for UAV (u#) resources

Config file structure

A config file consists of several lines, where each line has the following structure:

path/to/shader -T profile [-O3 -o "output/subdirectory" -E entry -D DEF1={0,1} -D DEF2={0,1,2} -D DEF3]

where:

  • path/to/shader - shader source file
  • -T - shader profile, can be:
    • vs - vertex
    • ps - pixel
    • gs - geometry
    • hs - hull
    • ds - domain
    • cs - compute
    • ms - mesh
    • as - amplification
  • -E - (optional) entry point (main by default)
  • -D - (optional) adds a macro definition to the list, optional range of possible values can be provided in {}
  • -O - (optional) optimization level (global setting used by default)
  • -o - (optional) output directory override

Additionally, the config file parser supports:

  • One line comments starting with //
  • #ifdef D, where D is a macro definition name (the statement resolves to true if D is defined in the command line)
  • #if 1 and #if 0
  • #else
  • #endif

Shader blob API

When the --blob command line argument is specified, ShaderMake will package multiple permutations for the same shader into a single "blob" file. These files use a custom format that is somewhat similar to regular TAR.

ShaderMake provides a small library with parsing functions to use these blob files in applications. This library can be statically linked with an application by including ShaderMake as a git submodule and linking the ShaderMakeBlob target to your application:

target_link_libraries(my_target PRIVATE ShaderMakeBlob)

Then include <ShaderMake/ShaderBlob.h> and use the ShaderMake::FindPermutationInBlob to locate a specific shader version in a blob. If that is unsuccessful, the ShaderMake::EnumeratePermutationsInBlob and/or ShaderMake::FormatShaderNotFoundMessage functions can help you provide a helpful error message to the user.