tpp - Tiny PreProcessor

TPP is a tiny (single source file) C Preprocessor, meant as a low-level backend for compilers of C and C-like languages, whilst implementing pretty much all preprocessor extensions supported by other compilers, ontop of a large number of its own (tpp-specific) extensions.

Usage

TPP is designed to be used in 1 of 2 ways:

As a back-end for writing C and C-like compilers with all of the work needed for dealing with macros, directives, warnings, and preprocessor-related pragmas already taken care of.
For this purpose, TPP is meant to be included statically as part of another project's source tree.
- For samples on how this can be done, look in /samples
As a standalone Preprocessor (similar to GNU's cpp utility).
For this purpose, you may build src/frontend.c to produce the commandline utility

Features

TPP operates without any intermediate buffers, in that it is capable of preprocessing/tokenizing text in real-time from stream-like input sources:
```
cat file.c | tpp
```
The above command can be used to preprocess text, without any need for any intermediate buffers (beyond what is needed for a single token) before TPP will start producing output.

This means that TPP will be able to parse input, and print preprocessed output in real-time, without needing to wait for you to close a PIPE, or reach the end of its input file, first.

In a sense, this can be compared to python's interactive commandline, which allows it to execute code as you write it. So while an interactive C commandline ~~is probably impossible~~ is possible, an interactive C preprocessor ~~most definitely isn't~~ obviously is, too. - Because that's what TPP is.
Invocation of the TPP frontend is also designed to be usable by other utilities via the --pp switch, which, rather than producing preprocessed output, will emit individual tokens seperated by '\0'-characters, allowing you to leave all of the tokenization to tpp, and do the processing yourself in a seperate process.

Preprocessor Extensions

All extensions can be hard enabled/disabled when building tpp, or soft-enabled/disabled via #pragma extension("-fname") / #pragma extension("-fno-name") directives.

Supported extensions can be tested for with __has_known_extension("name"), and supported+enabled extensions with __has_extension("name"). In turn, __has_known_extension and __has_extension can be tested for with #ifdef.

Trigraphs (??*)
Digraphs (<% is {, etc...)
STDC __VA_ARGS__ (#define printf(...) fprintf(stderr, __VA_ARGS__))
GCC-style varargs macros (#define printf(format...) fprintf(stderr, format))
GCC-style va-comma (#define printf(format, ...) fprintf(stderr, format, ##__VA_ARGS__))
GCC-style if-else in expressions (#if foo ?: bar; same as #if foo ? foo : bar)
c++20 __VA_OPT__ (#define printf(format, ...) fprintf(stderr, format __VA_OPT__(,) __VA_ARGS__))
'\e' escape codes in strings and characters (for ASCII ESC, iow: '\x1b')
Binary literals (0b00101010)
MSVC's __pragma(foo) built-in function-like directive/macro
STDC's _Pragma("foo") built-in function-like directive/macro
GCC-style #warning My Message Here directives
GCC-style #include_next <header.h> directives
OBJc-style #import <header.h> directives
#ident and #sccs directives (true origin unknown; accepted by some ancient compilers for placing comments into .o-files)
STDC magic macros __FILE__, __LINE__, __TIME__ and __DATE__
Magic macro __BASE_FILE__
Magic macros __INCLUDE_LEVEL__ and __INCLUDE_DEPTH__
Magic macro __COUNTER__
Magic macro __TIMESTAMP__
Magic macro __COLUMN__ (tpp-specific; like __LINE__, but the current column-position)
STDC-compliant preprocessor #if-expressions
All STDC-required preprocessor directives (#define, #undef, #if, #ifdef, #ifndef, #elif, #else, #endif, #include, #pragma, #error)
Clang-style feature test macros:
- __has_include()
- __has_include_next()
- __is_identifier(), __is_deprecated(), __is_poisoned()
- __has_attribute(), __has_builtin(), ...
- __is_builtin_identifier() (tpp-specific)
- __has_extension(), __has_known_extension() (tpp-specific)
- __has_warning(), __has_known_warning() (tpp-specific)
Treat #!-directives as comments (i.e. #!/bin/tpp is treated as a comment)
Treat $ as a character, which may thus appear in an identifier
Preprocessor assertions (no; not those from <assert.h>..., but this):
- #assert cpu(mycpu)
- #unassert cpu(yourcpu)
- #if #cpu(mycpu)
Multi-char character literals (e.g. #if 'abc' == 6513249)
__VA_COMMA__ (tpp-specific extension)
- #define printf(format, ...) fprintf(stderr, format __VA_COMMA__ __VA_ARGS__)
- Expands to , when __VA_ARGS__ are non-empty and /**/ when empty.
- Essentially the same as __VA_OPT__(,)

__VA_NARGS__ (tpp-specific extension)

Expands to the decimal representation of the # of arguments in __VA_ARGS__

Useful for overloading macros by argument count

#define min_1(a)       a
#define min_2(a, b)    ((a) < (b) ? (a) : (b))
#define min_3(a, b, c) min_2(min_2(a, b), c)
#define min(...)       min_##__VA_NARGS__(__VA_ARGS__)

Optional support for traditional macro expansion rules (disabled by default, can be enabled temporarily via #pragma extension)

#define CAT(a,b) a ## b
#define STR(x)   #x
#pragma extension(push, "-ftraditional-macro")
#define T_CAT(a, b) a/**/b
#define T_STR(x)    "x"
#pragma extension(pop)
CAT(10, 20)   // 1020
STR(10)       // "10"
T_CAT(10, 20) // 1020
T_STR(10)     // "10"

Test for with #if __has_known_extension("traditional-macro")

Alternative parenthesis for macro definitions (tpp-specific extension):
- Allow use of:
  - #define foo(a,b) ...
  - #define foo[a,b] ...
  - #define foo{a,b} ...
  - #define foo<a,b> ...
- In other words: define macros that use tokens other than (...)-pairs for arguments
- Test for with #if __has_extension("alternative-macro-parenthesis")
Self-recursive macros (tpp-specific extension):
- STDC specifies that macros that try to expand to themselves should not be allowed to do so when this happens
- TPP has an optional extension to relax this rule for function-like macros defined while this extension is active
- Such function-like macros are allowed to expand to themselves, so-long as arguments passed during expansion differ from all other invocations of the same macro along the macro-expansion call-stack
- Test for with #if __has_known_extension("macro-recursion")
- Enable/disable with #pragma extension("-fmacro-recursion") and #pragma extension("-fno-macro-recursion")
Whitespace-sensitivity (tpp-specific extension):
- STDC specifies that whitespace should be stripped around macro bodies. - However, some programming languages are whitespace sensitive, meaning that the removal of whitespace in such cincumstances may alter program behavior.
- For this purpose, TPP has an option to keep whitespace if necessary.
- Test for with #if __has_extension("macro-argument-whitespace")
Allow select operations on string literals in preprocessor expressions (tpp-specific extension):
- #if "FOO" != "BAR"
- #if "FOO"[0] == 'F'
- #if "FOO"[1:] == "OO"
- #if #"FOO" == 3
- Test for with #if __has_extension("strings-in-expressions")
Allow for GCC-style __builtin_* function calls in preprocessor expressions (tpp-specific extension):
- #if __builtin_ffs(0x100) == 9
- Test for with #if __has_extension("builtins-in-expressions")
Allow for #@ as a replacement for # to convert macro arguments into character literals, rather than strings:
- ```
#define STR(x) #x
#define CHR(x) #@x
STR(f) // "f"
CHR(f) // 'f'
```
- Test for with #if __has_extension("charize-macro-argument")
Prevent in-place expansion of macro arguments during macro-function-expansion (tpp-specific extension) (s.a. /test/pound_xclaim.h)
- Test for with #if __has_extension("dont-expand-macro-argument")
Logical Xor operators (A ^^ B; same as !!(A) ^ !!(B)) (tpp-specific extension)
Precise date/time macros (tpp-specific extension):
- __DATE_DAY__, __DATE_WDAY__, __DATE_YDAY__, __DATE_MONTH__, __DATE_YEAR__
- __TIME_SEC__, __TIME_MIN__, __TIME_HOUR__
- Test for with #ifdef <macro-name>
Evaluate preprocessor expressions, and expand to their decimal/string representation (tpp-specific extension):
- For example: __TPP_EVAL(10 + 20) would expand to exactly 1 token 30
- Highly useful because you can do stuff like:
```
#define CAT2(a, b) a##b
#define CAT(a, b) CAT2(a, b)
#define ISLESS_0 more than
#define ISLESS_1 less than
#define SELECT(n) CAT(ISLESS_, __TPP_EVAL(n < 10))
SELECT(7)  // Expands to `less than'
SELECT(10) // Expands to `more than'
```
- Can also be combined with "strings-in-expressions" to operate on strings:
```
__TPP_EVAL("foobar"[3:]) // Expands to "bar"
```
- Test for with #ifdef __TPP_EVAL
Generate unique IDs for a given keyword/name (that remain unchanged when re-compiling the same file without altering its contents) (tpp-specific extension):
- For example, __TPP_UNIQUE(foo) expands to a unique decimal integer that will remain the same for foo within the entire input (though not necessarily within other files that may be preprocessed seperately)
- Test for with #ifdef __TPP_UNIQUE
Load the contents of a #include-file into a string (tpp-specific extension):
- For example, __TPP_LOAD_FILE("file.c") will expand to the the contents of file.c, contained in, and escaped as a "string"
- Test for with #ifdef __TPP_LOAD_FILE
An arbitrary number of __COUNTER__-like counters (tpp-specific extension):
- For example, each time __TPP_COUNTER(foo) is encountered, it will expand to 1 greater than its previous expansion (starting at 0)
- Similar behavior is also possible with __COUNTER__, however using __TPP_COUNTER, you can have an arbitrary number of counters operating independent of each other.
- Test for with #ifdef __TPP_COUNTER
Random number generation (tpp-specific extension):
- Generate a random number in [0,hi) with __TPP_RANDOM(hi)
- Generate a random number in [lo,hi) with __TPP_RANDOM(lo, hi)
- Test for with #ifdef __TPP_RANDOM
Re-interprete the contents of a string token as input text (tpp-specific extension):
- ```
#define foo 42
__TPP_STR_DECOMPILE("foo bar") // Expands to `42 bar'
```
- Test for with #ifdef __TPP_STR_DECOMPILE
Pack together a string token from the ASCII codes of its individual characters (tpp-specific extension):
- For example, __TPP_STR_PACK(0x48, 0x65, 0x6c, 0x6c, 0x6f) expands to "Hello"
- Test for with #ifdef __TPP_STR_PACK
More extensions exist, but I feel those are not important enough to document here
- Most notably, supported #pragma directives, such as #pragma once, #pragma push_macro(), etc...

GrieferAtWork/tpp

tpp - Tiny PreProcessor

Usage

Features

Preprocessor Extensions