Threading improvments

[Frityet]

Currently, the only thread safety that is present is that the count, totalsize and reference_count fields on the metadata are atomic. This is not actually that thread safe, and there is much more we can do. However, thread safety should be toggleable with a macro, as It does present a performance hit

[Frityet]

Some ideas:

atomics do have measurable overhead in single-threaded programs ... but avoiding unnecessary refcount changes (i.e. supporting both moving and borrowing) may be enough to make the overhead negligible. Still, consider how you could support disabling the thread support at compile-time.

use of seq_cst (the default) is never what you want for any real-world problem I've encountered, and it is very expensive so you should actively avoid it. Hereafter I will only speak about use of atomics with reference counting.

incref should use memory_order_relaxed
decref should use memory_order_release

the last decref must use memory_order_acquire. There are numerous ways this can be combined with the previous (merged acq_rel, explicit fence, later acq, optimistic fast-path falling back to another way, ...); the optimal choice is not obvious, and is likely to depend on the expected refcount statistics, as well as the architecture.

there is a likely race condition if trying to incref an object's field, if the field points to an object with only a single reference, and another thread is writing to the field and decref'ing that object. For maximum safety, the approach is to defer the actual free, so that the memory remains valid, and if would you try to incref that, give up and try again (beware: implementing this correctly is nontrivial). ... but since this is C maybe we could just say "don't do that"?

From this reddit comment