/CSL

C++ integrated shading language

Primary LanguageC++MIT LicenseMIT

Build Status GitHub license GitHub repo size

C++ integrated Shading Language

CSL is a C++ header-only library, self-transpiling into GLSL. It allows to write OpenGL shaders directly inside computer graphics applications code. The concept is that shader correctness is checked at compile-time by the C++ compiler, while a string corresponding to the GLSL code is produced at run-time by the graphic application. CSL can be considered as some kind of inlined GLSL compiler, meaning that if the shader should not compile, the application should not compile either. This project goals are to provide convenient and maintainable shader writing thanks to:

  • Having a syntax as close as possible to GLSL.
  • Having the shader code directly inside the C++ code.
  • Checking GLSL specification compliance at C++ compile-time as much as possible.
  • The possibility to use C++ as meta language for generic shader generation.

CSL requires a C++17 compliant compiler and does not require any external dependency. It built successfully using Visual Studio 2022 (Windows), Clang (Windows, Linux, Apple), and GCC (Windows, Linux, Apple).

This repo contains the CSL source files and a shader suite application, which shows and runs several CSL shaders, from didactic examples to more complex shaders.

CSL is a template-heavy library and compilation is currently noticeably slow. The run-time shader generation is however pretty fast. The whole shader suite, including all the examples present in this readme, takes approximately 5 ms to be generated.

Disclaimer : This project is a work in progress and the current coverage of the different GLSL specifications is only partial. Many functions or language features are missing. The syntax section gives a nice overview of what is currently available. The goal is to first make possible what is legal in GLSL. In a second time, the goal will be to make impossible what is not valid in GLSL.

Setup

As CSL is a header-only library, only the header files in the CSL/include/csl/ are needed, and simply include of the file <csl.hpp> is enough to use it. Here is a small program, showing a vertex shader example and its corresponding output. CSL syntax is described in more detail in the syntax section.

Code Output 
#include <CSL/include/csl/csl.hpp>
#include <iostream>

int main() {
    using namespace csl::glsl::vert_420;  
    Shader shader;

    Qualify<layout<location<0>>, in, vec3> position;
    Qualify<uniform, mat4> mvp;

    shader.main([&]{
          gl_Position = mvp * vec4(position, 1.0);
    });

    GLSLData data;
    shader.print_glsl(data);
    std::cout << data.stream.str() << std::endl;
}

   #version 420

   layout(location = 0) in vec3 position;
   uniform mat4 mvp;

   void main()
   {
      gl_Position = mvp * vec4(position, 1.0);
   }

For readability purposes, all outputs are shown as if the code used named variables. Please check the naming variables section for more details about the actual output.

Shader suite

alt text

The CSL shader suite is an application showcasing a collection of shaders written in CSL. For each shader, the application provides the output GLSL string, some metrics about runtime performances, and a way to explore the shader expression tree using ImGui. It also allows to see several shaders in action, thus serving as a practical test suite.

Building

Python must be installed as glad relies on it generate and download its files.

First clone the repo:

git clone https://github.com/thonatt/CSL.git

The shader suite can then be built using CMake:

cd CSL
mkdir build
cd build
cmake .. -G "Visual Studio 17 2022" ## Or any available generator.

Finally compile and run the application:

cmake --build . --config Release
.\bin\Release\CSL_ShaderSuite.exe ## Other generators might create it elsewhere.

Credits

Developped by Théo Thonat, initially from shader writing induced frustration, but in the end mostly to learn advanced template programming and explore C++ dark corners.

Special thanks to Simon Rodriguez for decisive and positive feedback about the initial project concept, the help with compilers cross-checking, and for many of the shaders examples.

CSL syntax

As GLSL and C++ share a common C base language, their syntax are quite similar. However, the goal of CSL is to write C++ code that can perform introspection, while keeping a syntax close to GLSL, meaning that some tradeoffs had to be made. This section covers the subtleties of the CSL syntax (hopefully nothing too repulsive !). It is also a summary of what is possible in CSL:

Shader setup

Shader type and GLSL version are setup using a specific namespace. For example, using namespace csl::glsl::vert_420 gives access to the built-in functions and built-in variables for a vertex shader with GLSL 4.20. Vertex, fragment, geometry, compute,and tesselation shaders are currently supported.

Starting a new shader requires to create a variable of type Shader. This type contains two important member functions. The first one is Shader::main which allows to setup the main function using a lambda function with no argument that returns nothing. The second one is Shader::print_glsl, which can be used to retrieve the std::string associated to the shader that can later be sent to the GPU. See the previous section for an example.

CSL assumes instructions are called sequentially and is not thread-safe.

Basic and Sampler types

CSL defines the various types used in GLSL. Most CSL types have the exact same type name as their GLSL counterpart. For example, void, vec3, ivec2, dmat4, mat4x3, sampler2DArray, uimageCube are all valid types that can be used as is. The only exceptions are double, float, int and bool as they would conflict with C++ keywords. Valid associated CSL typenames are Double, Float, Int, Bool - and Uint to keep the consistency.

Constructors and declarations are identical to GLSL. CSL and C++ basic types can also be mixed.

Show constructors example
Code Output 
Uint counter = 0;
vec2 uv = vec2(1.0, 2.0);
vec4 color = vec4(0.0, uv, Float(counter));
bvec3 m = not(bvec3(!Bool(true), false, false));

	uint counter = 0;
	vec2 uv = vec2(1.0, 2.0);
	vec4 color = vec4(0.0, uv, float(counter));
	bvec3 m = bvec3(true, false, false);

Naming variables

Because C++ objects do not have names at runtime, it is not possible to directly forward the CSL variable names to the GLSL output. As a consequence, CSL will perfom automatic variable naming, which has a significant impact on the output shader readability.

Automatic naming example
Code with automatic naming Actual output 
Qualify<in, vec3> normal;
Qualify<in, vec3> position;
Qualify<uniform, vec3> eye;

shader.main([&] {
  Float alpha = 1.2;
  vec3 V = normalize(eye - position);
  vec3 N = normalize(normal);

  Float result;
  result = alpha * dot(N, V);
});

in vec3 x4;
in vec3 x5;
uniform vec3 x6;

void main()
{
    float x8 = 1.2;
    vec3 x9 = normalize(x6 - x5);
    vec3 x10 = normalize(x4);
    float x11;
    x11 = x8*dot(x10, x9);
}

Therefore, it is possible to provide a name to any CSL variable. It can be done either when declaring a variable using the (const std::string&) constructor, or when initializing a variable using the <<(const std::string&) operator. Manual naming is rather cumbersome, but may be useful for certain cases such as:

  • Access to uniforms location with an explicit name.
  • Name consistency between vertex out and fragment in variables.
  • Output shader readability for debugging purposes.
Same example with manual naming
Code with manual naming Actual output 
  //naming during variable declaration
  Qualify<vec3, In> normal("normal");
  Qualify<vec3, In> position("position");
  Qualify<vec3, Uniform> eye("eye");

  shader.main([&] {
  	//naming during variable initialisation
  	Float alpha = Float(1.2) << "alpha";
  	vec3 V = eye - position << "V";
  	vec3 N = normalize(normal) << "N";

  	//naming during variable declaration
  	Float result("result");
  	result = alpha * dot(N, V);
  });

in vec3 normal;
in vec3 position;
uniform vec3 eye;

void main()
{
    float alpha = 1.2;
    vec3 V = eye - position;
    vec3 N = normalize(normal);
    float result;
    result = alpha*dot(N, V);
}

Operators and Swizzles

As C++ and GLSL share a common C base syntax, most of the operators keywords are identical and can be used as is. This includes for example:

  • +, -, *, / and their assignment operator counterparts,
  • ==, <, > , && and other binary relational or bitwise operators,
  • [] for component or row access.

One exception is the ternary operator ? :. Even if the synthax is similar between C++ and GLSL, it cannot be overloaded. Therefore it is replaced by a macro CSL_TERNARY with the 3 arguments.

Swizzles are GLSL-specific operators for verstatile vector components acces. In order to preserve all the swizzling possibilities while keeping the code simple, CSL uses global variables such as x, y, z, or w. The syntax for swizzle accessing is for example my_var(x, z, x) instead of my_var.xzx. To prevent namespace pollution, all those swizzle variables belong to a specific namespace corresponding to their swizzle set. Available namespaces are csl::swizzles::xyzw, csl::swizzles::rgba, csl::swizzles::stpq, and csl::swizzles::all which includes the previous three.

Swizzle and operators examples
Code Output 
using namespace csl::swizzles::rgba;
vec4 col("col");
vec4 out("out");

col = CSL_TERNARY(col(a) > 0, col, 1.0 - col);

// Can you guess what is actually assigned?
out(a) = col(b, a, r)(y, z)(s);

vec4 col;
vec4 out;
col = col.a > 0 ? col : 1.0 - col;
out.a = col.bar.yz.s;

Qualifiers and shader stage options

Qualifiers can be added to a type in CSL by using the template class Qualify. Qualifiers are classes, and may be templated if they require values. For example, uniform is a class, while layout is a template class, with layout qualifiers specified as template class parameters. binding is a template class, with binding values specified as unsigned int parameters. CSL qualifiers names are identical to GLSL, with spaces replaced by underscores.

Qualifiers examples
Code Output 
	Qualify<out, vec4> color;
	Qualify<layout<location<4>>, in, vec3> position;
	Qualify<layout<binding<0>>, uniform, sampler2DArray, Array<8>> samplers;

out vec4 color;
layout(location = 4) in vec3 position;
layout(binding = 0) uniform sampler2DArray samplers[8];

Shader stage options can be specified with the templated function shader_stage_option, using the template parameters as layout qualifiers.

Shader stage option examples
Code Output 
  // In a fragment shader:
  shader_stage_option<layout<early_fragment_tests>, in>();

  // In a geometry shader:
  shader_stage_option<layout<triangles>, in>();
  shader_stage_option<layout<line_strip, max_vertices<2>>, out>();

layout(early_fragment_tests) in;
layout(triangles) in;
layout(line_strip, max_vertices = 2) out;

Arrays and Functions

Arrays in CSL are declared as qualifiers using the Array class, with size as parameter. A size of zero is used for implicitely sized GLSL arrays. Indexing is done with the usual [] operator. Multiple sizes can be provided to define multidimensional arrays.

Array examples
Code Output 
// Array declaration.
Qualify<vec3, Array<5>> vec3x5;

// Array initialization.
Qualify<Float, Array<0>> floatX = Qualify<Float, Array<0>>(0.0, 1.0, 2.0);

// Multi dimensionnal arrays.
Qualify<mat3, Array<2, 2>> mat3x2x2 = Qualify<mat3, Array<2, 2>>(
Qualify<mat3, Array<2>>(mat3(0), mat3(1)),
Qualify<mat3, Array<2>>(mat3(2), mat3(3)));

// Usage.
vec3x5[0] = floatX[1] * mat3x2x2[0][0] * vvec3x5[1];

vec3 vec3x5[5];
float floatX[] = float[](0.0, 1.0, 2.0);
mat3 mat3x2x2[2][2] = mat3[2][2](mat3[2](mat3(0), mat3(1)), mat3[2](mat3(2), mat3(3)));
vec3x5[0] = floatX[1]*mat3x2x2[0][0]*vec3x5[1];

Functions in CSL are objects that can be created using the define_function function with a C++ lambda as parameter. The return type must be explicitely specified as template parameter. Returns are declared using the CSL_RETURN; or CSL_RETURN(expression); syntax. Parameters can be named using default argument values. The function can be called later in the code using the usual () operator with valid arguments. Function overloading is possible in CSL by providing multiple return types and multiple lambdas.

Function examples
Code Output 
// Empty function.
auto fun = define_function<void>([] {});

// Named function with named parameters.
auto add = define_function<vec3>("add",
  [](vec3 a = "a", vec3 b = "b") {
  CSL_RETURN(a + b);
});

// Function with some named parameters.
auto addI = define_function<Int>(
  [](Int a, Int b = "b", Int c = "") {
  CSL_RETURN(a + b + c);
});

// Function calling another function.
auto sub = define_function<vec3>([&](vec3 a, Qualify<inout, vec3> b = "b") {
  fun();
  CSL_RETURN(add(a, -b));
});

// Named function with several overloads.
auto square = define_function<vec3, ivec3, void>("square",
  [](vec3 a = "a") {
  CSL_RETURN(a * a);
},
  [](ivec3 b = "b") {
  CSL_RETURN(b * b);
},
  [] { CSL_RETURN; }
);

void x4()
{
}

vec3 add(vec3 a, vec3 b)
{
    return a + b;
}

int x8(int x9, int b, int x11)
{
    return x9 + b + x11;
}

vec3 x12(vec3 x13, inout vec3 b)
{
    x4();
    return add(x13, -b);
}

vec3 square(vec3 a)
{
    return a*a;
}

ivec3 square(ivec3 b)
{
    return b*b;
}

void square()
{
    return;
}

Building blocks

Selection, iteration and jump statements are available in CSL. As C++ and GLSL share the same keywords, CSL redefines them using macros with syntax CSL_KEYWORD, namely CSL_FOR, CSL_CONTINUE, CSL_BREAK, CSL_WHILE, CSL_IF, CSL_ELSE, CSL_ELSE_IF, CSL_SWITCH, CSL_CASE, and CSL_DEFAULT. Their behavior is mostly identical to C++ and GLSL. Here are some comments and a few limitations:

  • A CSL_SWITCH must contain a CSL_DEFAULT case, even if it happens to be empty.
  • CSL syntax for case value : is CSL_CASE(value) :.
  • for loops are declared using CSL_FOR( init-expression; condition-expression; loop-expression), where at most one init-expression and one condition-expression are supported. Any of those expressions can be empty.
  • Variables declared in CSL_FOR args expressions outlive the scope of the for body. It is possible to prevent that by putting explicitly the whole for loop in a scope.
  • Statements can be nested.
Building blocks examples
Code Output 
	// Empty for.
	CSL_FOR(;;) { CSL_BREAK; }

	// For loop with named iterator.
	CSL_FOR(Int i = Int(0) << "i"; i < 5; ++i, i+=2) {
		CSL_IF(i == 3) {
			++i;
			CSL_CONTINUE;
		} CSL_ELSE_IF(i < 3) {
			i += 3;
		} CSL_ELSE{
			CSL_FOR(; i > 1;)
				--i;
		}
	}

	// Not possible as i is still in the scope.
	// Int i; 

	{
		Bool b;
		CSL_FOR(Int j = Int(0) << "j"; b;) {
			CSL_WHILE(j != 3)
				++j;
			b = j < 5;
		}
	}
	// OK since previous for was put in a scope.
	Int j = Int(0) << "j";

	CSL_SWITCH(j) {
		CSL_CASE(0) : { CSL_BREAK; }
		CSL_CASE(2) : { j = 3; }
		CSL_DEFAULT: { j = 2; }
	}

for( ; ; ) {
    break;
}
for(int i = 0; i < 5; ++i, i += 2) {
    if (i == 3)
    {
        ++i;
        continue;
    }
    else if (i < 3)
    {
        i += 3;
    }
    else
    {
        for( ; i > 1; ) {
            --i;
        }
    }
}
bool x5;
for(int j = 0; x5; ) {
    while(j != 3) {
        ++j;
    }
    x5 = j < 5;
}
int j = 0;
switch(j) {
    case 0 : {
        break;
    }
    case 2 : {
        j = 3;
    }
    default : {
        j = 2;
    }
}

Structs and Interface blocks

CSL structs are declared using the syntax CSL_STRUCT(StructTypename, member-list ...);. As members in C++ have no way to know if they belong to a struct, CSL has to perform some form of reflection, based on C++ preprocessor magic. So to help the preprocessor looping over the members, member-list has to be declared using extra parenthesis such as: (Type1, member1), (Type2, member2), ...

Struct examples
Code Output 
// Struct declaration.
CSL_STRUCT(Block,
  (mat4, mvp),
  (vec4, center)
);

// Nested struct.
CSL_STRUCT(BigBlock,
  (Block, inner_block),
  (vec4, center)
);

// Variables declaration and naming.
BigBlock big_block("big_block");
Block block = Block(mat4(1), vec4(0)) << "block";

// Usage.
block.center = big_block.inner_block.mvp * big_block.center;

struct Block
{
    mat4 mvp;
    vec4 center;
};

struct BigBlock
{
    Block inner_block;
    vec4 center;
};

BigBlock big_block;
Block block = Block(mat4(1), vec4(0));
block.center = big_block.inner_block.mvp*big_block.center;

Nammed interface blocks are similar to structs with syntax CSL_INTERFACE_BLOCK(qualifiers-list ..., Typename, Name, member-list ... );. The qualifiers-list refers to the qualifiers associated to the block. In case multiple qualifiers as specified, extra parethesis must be put around the list. Name is name of the variable associated to the block.

Unnamed interface blacks are declared using the syntax CSL_UNNAMED_INTERFACE_BLOCK(qualifiers-list ..., Typename, member-list ...). In that case, block members belong directly to the current scope.

Interface block examples
Code Output 
// Unnamed interface block.
CSL_UNNAMED_INTERFACE_BLOCK(in, SimpleInterface,
  (Float, delta_time)
);

// Named array interface with multiple qualifiers.
CSL_INTERFACE_BLOCK((layout<binding<0>>, out, Array<3>), Output, out,
  (vec3, position),
  (vec3, velocity)
);

out[0].position += delta_time * out[0].velocity;

in SimpleInterface {
    float delta_time;
};

layout(binding = 0) out Output {
    vec3 position;
    vec3 velocity;
} out[3];

out[0].position += delta_time*out[0].velocity;

Since the qualifiers-list and member-list are parsed by the preprocessor, qualifiers and members typename must not contain any comma. To circumvent this issue, either extra parenthesis must be added to help the preprocessor, or typename alias should be used.

Type alias examples
Code Output 
using vec4x16 = Qualify<vec4, Array<16>>;
CSL_INTERFACE_BLOCK(
  (layout<binding<0>, std140>, uniform, Array<2>),  // Extra parenthesis. 
  MyInterface, vars,
  (vec4x16, vecs),                                  // Typename alias.
  ((Qualify<mat4, Array<4>>), mats)                 // Extra parenthesis.
);

layout(binding = 0, std140) uniform MyInterface {
    vec4 vecs[16];
    mat4 mats[4];
} vars[2];

Since CSL must rely on a macro for structs and interface blocks, members names and interface block variable names can be directly forwarded to CSL. Therefore there is no need for either automatic or manual naming in those cases.

Generic shader generation

Regular C++ workflow can be used to help generation of CSL shaders, such as changing values, manual unrolling or conditionnal expressions. The helper std::integral_constant can be used to pass constexpr parameters, which are useful for example to specify the size of an array.

The use of C++ as a meta-languages for CSL has limitations. For example, CSL scopes are still C++ scopes under the hood, so CSL declarations do not outlive C++ scopes.

Generic shader generation example
Code Output 
template<typename T>
auto shader_variation(T&& parameter, std::array<double, 2> direction, bool gamma_correction)
{
	using namespace csl::glsl::frag_420;
	using namespace csl::swizzles::rgba;
	Shader shader;

	Qualify<sampler2D, uniform> samplerA("samplerA"), samplerB("samplerB");
	Qualify<vec2, in> uvs("uvs");
	Qualify<vec4, out> color("color");

	shader.main([&] {
		vec2 sampling_dir = vec2(direction[0], direction[1]) << "sampling_dir";

		constexpr int N = T::value;
		Qualify<vec4, Array<2 * N + 1>> cols("cols");

		CSL_FOR(Int i = Int(-N) << "i"; i <= N; ++i) {
			cols[N + i] = vec4(0);
			for (auto& sampler : { samplerA, samplerB })
				cols[N + i] += texture(sampler, uvs + Float(i) * sampling_dir);
			color += cols[N + i] / Float(2 * N + 1);
		}

		if (gamma_correction)
			color(r, g, b) = pow(color(r, g, b), vec3(2.2));
		});

	return shader;
};

shader_variation(std::integral_constant<int, 9>{}, { 0, 1 }, true);
shader_variation(std::integral_constant<int, 5>{}, { 1, 0 }, false);

#version 420

uniform sampler2D samplerA;
uniform sampler2D samplerB;
in vec2 uvs;
out vec4 color;

void main()
{
    vec2 sampling_dir = vec2(0.0, 1.0);
    vec4 cols[19];
    for(int i = -9; i <= 9; ++i) {
        cols[9 + i] = vec4(0);
        cols[9 + i] += texture(samplerA, uvs + float(i)*sampling_dir);
        cols[9 + i] += texture(samplerB, uvs + float(i)*sampling_dir);
        color += cols[9 + i]/float(19);
    }
    color.rgb = pow(color.rgb, vec3(2.2));
}

#version 420

uniform sampler2D samplerA;
uniform sampler2D samplerB;
in vec2 uvs;
out vec4 color;

void main()
{
    vec2 sampling_dir = vec2(1.0, 0.0);
    vec4 cols[11];
    for(int i = -5; i <= 5; ++i) {
        cols[5 + i] = vec4(0);
        cols[5 + i] += texture(samplerA, uvs + float(i)*sampling_dir);
        cols[5 + i] += texture(samplerB, uvs + float(i)*sampling_dir);
        color += cols[5 + i]/float(11);
    }
}