weak reference support
brody4hire opened this issue · 3 comments
This could really help me get my experimental data management project working with no_std: https://github.com/brodycj/sclang
I don't need all of the standard functions that are supported by std::rc::Weak, I just need to be able to convert between strong & weak RC references as needed.
It may be ideal to get this working with Arc but my project is not yet ready for multi-threaded support anyways (and may never be).
My experimental project is working on a way to handle automatic memory management with circular data, without the random GC overhead. I make no guarantees at this point, not about correctness & not about resource efficiency, just a first experimental step. Rust's strong & weak RC reference mechanism is extremely helpful, hoping to find a way to get past this issue.
From a quick look through rc.rs both here & in main Rust project I suspect this should not be super-difficult, and I may try hacking this myself. I just wanted to open this issue to get the discussion started. Thanks!!
UPDATE: I have since discovered that Tokio's portable-atomic-util DOES provide Weak reference support with its Arc support for no-std, so I will probably just use that project for my needs. Thanks!!
Hi @brodycj,
This is outside the scope of our project. You’re welcome to fork it and work on it yourself.
What you're describing seems to relate to tracing reference counting (RC). Although there have been attempts within the Rust community, implementing this can be quite challenging. Unfortunately, there’s no simple solution; auto memory management comes with its trade-offs.
Based on my research, a smart garbage collection (GC) algorithm is generally more effective than RC for these issues and can sometimes outperform manual memory management. However, since Rust doesn’t support the necessary backend for such allocators, implementing this isn't feasible at the moment at least with a good enough/easy interface.
Hey thanks for the response!
Any further you may have concerning efforts you may have seen in terms of a tracing RC solution or on your research on smart GC vs other mechanisms would be extremely helpful.
I would love to find a way to get my idea for automatic memory management working without the semi-random overhead of mark-and-sweep GC. I think the overhead of semi-random mark-and-sweep GC would increase with the number of reachable objects kept in the heap, while my idea is able to get rid of this semi-random mark-and-sweep GC. Yes there would be a different memory management cost with my idea, and I will be looking for anyone who could help analyze and optimize this.
I would also like to start raising some funds to help support this kind of work. It may not be very much in the near term but I am hoping to find a way to expand this kind of investigation and improve the situation over time, especially for low-power embedded systems.
I would also like to clarify something that I think was too confusing: my experimental project is focused on demonstrating this memory management idea with a toy scripting language, with intention to focus more on targeting this idea for languages such as JavaScript, Python, and even potentially Java, Go, maybe even Swift.
Thanks again and my apologies for burdening you with something may not be of any interest to you. Your response was already very informative.
@brodycj You're welcome mate. Improving GC and RC can be approached in various ways, such as enhancing allocators, tracing, marking, etc. This is a vast research area with many complexities.
The semi-random overhead issue persists and gets worse(more repetitive) even with tracing RC because it's difficult to predict how many cyclic references need resolving/pointlessly checking. This is why I suggested GC might be a better option. I've experimented with memory management in a toy language of my own and still struggle with these concepts. It's a tough choice.
Opting for GC means you must drop support destructor functions, as they need to run in a possibly separate thread and out of order, making them unsafe, that's the reason Golang focuses on defer not destructors. On the other hand, implementing a tracing RC system is complex, especially for resolving cyclic dependencies with dynamic objects, such as Go's interface{} or Rust's dyn trait. There's no perfect solution; it's about minimizing the overhead and finding the best fit for your specific use case.
The decision between GC and RC depends on various factors, including the language's requirements, performance considerations, and safety concerns. Both approaches have trade-offs, and extensive research is ongoing to improve each method. For instance, recent advancements in concurrent GC techniques and hybrid approaches are promising but still come with their challenges. Ultimately, the goal is to balance performance, safety, and ease of implementation.
I need to add: don't make this your main focus. It's one of those problems that can consume years of your life without results. I've already spent too many hours on it myself. Treat it as a fun experiment to tackle in your free time.