[feature] better support for storage and type qualifiers

[scolsen]

C supports a couple of different type and storage qualifiers which have significant impact on its semantics, such as const, volatile, static, extern.

Currently, there are only two ways to emit these sorts of qualifiers from Carp:

1. Define a C template that has the qualifiers as part of the template's declaration.
2. Use the annotations meta field to annotate bindings with qualifiers. This is currently only supported for defn forms.

Both methods are fine, but lead to some clunkiness, for instance, it's currently impossible to declare a Carp function that takes a const qualified argument, without using some type trickery; one has to register the "constant variant" of a type e.g. (register-type ConstInt "int const")) or resort to defining the function as a template. This is unfortunate, since it means there's a whole class of functions that, though expressible in plain Carp, need to be written in C or using some kludge just because of a type qualifier at interface boundaries.

Furthermore, the placement of some qualifiers, such as const matters:

int const *: (non-const) pointer to constant int
int* const: constant pointer to a (non-const) int
int const * const: constant pointer to a constant int.

The current situation also makes it quite difficult to qualify Carp's generic types. For example, I can't think of how one could const qualify the C type corresponding to a Carp generic using the current approaches without either knowing exactly what name the compiler will generate (brittle) and using this information to write a correct deftemplate.

In order to better support this, we can do a few things:

1. Extend annotations meta support. As mentioned, the value of this field is currently only emitted for function declarations. It should be emitted for variables (def forms) as well. Additionally, the annotations mechanism currently emits the annotation on the left of the declaration. This breaks the "read right to left" rule for complex declarations and could make it difficult to anticipate what will happen. At the same time, it is the necessary placement for storage qualifiers like static.
2. We should make it possible to qualify types, e.g. in the context of a function signatures. Writing:

(sig foo (Fn [(Const (Ptr Int))] Unit))

Should produce the declaration:

void foo(int* const);

And

(sig foo (Fn [(Ptr (Const Int))] Unit))

Should produce:

void foo(int const*);

3. We should support storage qualifiers only on declarations, since these are only permitted for declarations. So, writing something like (static foo), assuming foo is already defined and static is a special meta macro that sets a storage-class field, should produce:

static int foo = 0;

How should we implement this?

I think some combination of all three points considered above would be best, that is, I'd propose we:

1. Extend annotations meta to apply to all declarations, keep it maximally flexible to allow users to add whatever qualifiers they want, with the understanding that its behavior is to place the qualifiers immediately to the left of the declaration. This is great in that is allows users to emit qualifiers that may not yet be supported by more direct means in Carp, may be bound to weird macros, etc.
2. Add compiler backed type qualifier types in Carp that emit c type qualifiers. This might seem like overkill—couldn't we use meta for this too? In theory we could, but I think it's more consistent to use a type. Type qualifiers modify types, and indeed, good C compilers will treat <T> const and <T> as distinct, and complain if you attempt to use one in the position of another. Since we have a stronger type system in carp, I think it makes sense to define this at type-level. Furthermore, we need to recognize that, unlike storage class qualifiers, type qualifiers may appear on declarations and in function parameter positions—they are effectively proper types. Using higher-order types for this leads to the very natural syntax:

(sig foo (Fn [(Const (Ptr Int))] Unit))

Contrarily, if we attempted to use meta information for such qualifications, what would define the meta on? The type itself? That wouldn't work, for then all (Ptr Int) would become int* const. On the function parameter names and declarations? That would also be a bit bizarre as we don't currently support meta on parameters and the type checker would need to start consulting the meta of every binding to make sure the type is actually the given type and not a qualified type.

One can also then coerce to const/non-const types where needed, just as is sometimes necessary in C when calling across functions that have different const expectations (typically the programmer needs to confirm the non-const function doesn't modify the data): (the (Const (Ptr Int)) (address foo)).

To this end, we should add the following compiler-backed special type-qualifier types:

(Const a): emits <a> const
(Volatile a): emits <a> volatile
(Restrict (Ptr a)): emits <a>* restrict

Then, the carp type checker will also align with a good C compiler and complain about using a non-const type in a const position. Note that restrict is only valid for pointer types. We could also make restrict valid for the other reference types Ref, Box by defining it as a sumtype.

3. Finally, we can add a couple of convenience storage-class qualifier macros that use annotations under the hood once annotations are exteneded to support def declarations. e.g. (extern foo) emits extern <T> foo.

As an alternative to (2.) we could extend register-type to support template style generics, and then define a qualifying type like Const as: (register-type (Const a) "$a const"). This may be better insofar as it decouples this feature from the compiler. It would also permit users to declare a "left side" variant if needed (register-type (ConstLeft a) "const $a"). This would also take an approach that already works for specific types (register-type ConstInt "int const") and extended it to make things easier/more flexible.

[scolsen]

One downside to supporting type qualifiers through register-type is that they won't have an obvious/valid constructor.

If we defined (register-type (Const a) "$a const") the only way to construct this type would be to either:

• Define a template that performs the cast, use the template: (deftemplate const-init (Fn [a] (Const a)) "<decl>($a a)" "{return ($a const)a}") which is slightly problematic insofar as it returns a copy.
• Perform the cast in carp itself wherever you need the type (the (Const Int) (Unsafe.coerce foo)), which is the same as the template, it just requires the user to write more on the Carp side.

In either case, it's impossible to construct a value of the type ex nihilo directly in a declaration. Neither of these options is especially ergonomic and might be a nudge toward implementing more elegant support for this in the compiler.

Also; I haven't thought about what implications the type qualifiers have for the borrow checker on the Carp side, if any, but if if they do that would be another reason to implement support for them directly into the compiler.