Feedback to language designers writing source-to-source compilers a.k.a. transpilers

[stefanos82]

Hey Chris,

I hope you are doing well.

I love your content and I learn so much from you mate, I cannot thank you enough!

As the title of this issue says, I would like your feedback about those who transpile to C language, if possible.

Based on your own personal experience using C for years, what would be your advice to such language designers, what to avoid adding in their generated C code or how deep should they go with portability without getting bitten by C's peculiarities, such as well-hidden undefined behavior and such?

I'm looking forward for your answer.

Keep up the good work.

Regards,

Stefanos

[skeeto]

I'm glad you find my writing useful, Stefanos! I have little experience with transpiling to C, and I don't how much insight I can provide from my general C experience, but I'll try. Though, on reflection, in terms of what to avoid and what to do, I don't think it's so much different than implementing a language interpreter/VM in C. In general I expect generated code to have even less excuse depending on null-terminated strings (strcpy, etc.). Machine-generated code easily handles even the most tedious length tracking, (not that it's difficult in the first place). It won't need to use sizeof on its own string literals, i.e. in a macro, since it already knows the lengths. Transpilers can make generous use of memset and memcpy, just as C compilers do in their own code generation. There is also no excuse for generated code exhibiting unwanted arithmetic results, particularly overflow. It can generate every necessary check, no matter how tedious, and cannot simply forget. That typically means checks before all integer multiplications, and divisor checks before all integer divisions (of variable divisors). Caveat: Generated code may therefore be noticeably slower than what someone would write by hand. The details depend on the semantics of the input language. Does it also have undefined behavior itself? If not, then it cannot allow it in the transpiled C program. The most common such case I see in language implementations in C is stack overflow. Using C's own call stack for the language implementation is an obvious choice, but it means you get C's behavior on stack overflow, including the inability to handle it like an error. (This is something CPython specifically avoids, for example.) Similarly, does the input language have sufficient facilities to express the program's constraints (input ranges, etc.) such that it can avoid generating run-time checks? This is a tricky part of language design that involves lots of trade-offs. C dominates benchmarks in language shootouts because it largely leaves it up to the programmer to decide what checks are necessary, versus having, say, complex type semantics to express constraints in the program, so that a compiler can prove run-time checks are unnecessary. It's also why C compiles so quickly, typically most time being spent re-parsing headers in each translation unit. The call stack issue is tricky. In an interpreter/VM it's easily solved using a custom VM call stack, but compiling to C means making C calls and using the C stack. The best I can imagine is linking a runtime with a signal / exception handler to recover from stack overflows, including unwinding. If the language has defined signed arithmetic overflow, for addition, subtraction, and multiplication, the generated C code can do the operation in an unsigned type, then cast the result to signed, getting the proper result without undefined behavior. I do this in VMs. Division is trickier because unsigned isn't an option, and you need to check for and handle division by -1, which can overflow (e.g. INT_MIN/-1). Promotion to int: operands narrower than int are promoted to int before the operation. In particular, this includes narrower unsigned types (a trap, and one of C's major design flaws). If the language has a uint16 type, it will not translate to a direct uint16_t*uint16_t in C, which will (on 32-bit machines) promote to int*int and can overflow. A transpiler must understand these semantics. Always compile left-shifts using an unsigned left operand, even if the operand is signed in the input language. Obviously that's the same for logical right-shift. Arithmetic right-shift is in practice just a signed right-shift. That's not guaranteed by the C specification, where it's implementation defined, but that's how it works in every implementation I know. As mentioned, the right-hand operator may need to be range-checked at run-time before the shift if it's not constant. It's likely that the input language logical shift operator gives zero for huge shifts, which is undefined behavior in C so it must be handled specially. That's everything that comes to mind right now.

[stefanos82]

For once again, you blew my mind 😆

Seriously mate, I wish I had a mentor like you; I would have mastered my craft years ago!

Well...if you think about it, if you continue writing these masterpieces of yours, you will become my mentor in an implicit way 🤔

I hope my old brain can understand the tricky parts, because no matter how much I love C++ (which has lost its track by the way), I always find myself back in C getting my hands dirty and feel the excitement I once had as a university student back in 2002.

If you ever decide to write a book about C, let me know.

[stefanos82]

Oh I forgot to ask! Jason Turner has this amazing concept https://github.com/lefticus/cpp_weekly/issues for his YouTube series.

Do you think you could benefit anything from this concept for your blogging?

Would it direct towards what your own audience wants to learn or is it something that could put extra burden on your shoulders?

I don't want you to feel pressed or something like that; I'm just suggesting, because I have learned so many C++ things from Jason I honestly did not know about and quite frankly I feel relieved that an expert, like Jason, feels quite frustrated with many things in C++.

[skeeto]

I know what you mean. C never stops being exciting for me either. I've been brushing up on C++ the last few months for work because I'm involved in a project fully invested in "modern" C++. I need to know it better in order to keep up, and I'd forgotten more C++ than I realized. So I recently re-read Scott Meyers books, and others. I've been in continual shock at how complex it is, far more than I had realized or remembered. I will never be able to fit all these rules in my head. Some topics I cannot even follow the syntax (e.g. Concepts), and so I've completely given up on understanding them. Despite all this time investment, I'm not sure if I'll ever be confident writing or reviewing non-trivial modern C++. I can now fully appreciate the saying, "C++ appeals to the ego, not the intellect." My usual tools don't work as well either: GDB, ASan, and UBSan are lacking in C++. GDB struggles with templates and displaying complex C++ types, and both Clang and GCC have gaps instrumenting modern C++ for sanitizers. I've observed a number of false negatives, e.g. use-after-frees going uncaught because they went uninstrumented through a std::span. In contrast, GDB mostly works well with C, and I cannot remember the last time I saw a meaningful sanitizer false negative in a C program.

Do you think you could benefit anything from this concept for your blogging?

Probably, yes. I've been active in r/C_Programming for years, using it as a source of ideas eventually inspiring projects and blog articles. For example, my recent SDL2 article the result of observing the same set of mistakes repeated across dozens of projects shared on reddit. While this subreddit has a high signal-to-noise ratio relative to the rest of reddit (a low bar), I still wish it was better: More activity, fewer low-effort questions, more advanced discussions, less topic repetition. After dry spells, in my head I consider options for starting a new, alternative C discussion forum. Jason Turner's idea looks promising, so thanks for sharing!

[lemire]

I personally prefer to write C code, when given a chance. The problem is that C tends to migrate to C++ over time, due to peer pressure. You definitively can be productive with modern C++ (ideally C++17 or better).

[stefanos82]

I feel so relieved that two experts in their field (hey Daniel @lemire, good to see you here!) share the same concerns more or less as mine.

It's such a shame that C++ has become a massive...how to describe it in the most accurate way possible, like HomeDepot or any other massive DIY warehouse?

All I need to use is a screwdriver to do my job, that's all!

Update: So, when should we wait for the forum? I'm looking forward to learn so many things from great minds scattered out there in the unknown corners of internet.