proposal: arena: new package providing memory arenas
danscales opened this issue · 388 comments
Note, 2023-01-17. This proposal is on hold indefinitely due to serious API concerns. The GOEXPERIMENT=arena code may be changed incompatibly or removed at any time, and we do not recommend its use in production.
Proposal: arena: new package providing memory arenas
Author(s): Dan Scales (with input from many others)
Last updated: 2022-2-22
Discussion at https://golang.org/issue/51317
Abstract
We propose implementing memory arenas for Go. An arena is a way to allocate a set of memory objects all from a contiguous region of memory, with the advantage that the allocation of the objects from the arena is typically more efficient than general memory allocation, and more importantly, the objects in the arena can all be freed at once with minimal memory management or garbage collection overhead. Arenas are not typically implemented for garbage-collected languages, because their operation for explicitly freeing the memory of the arena is not safe and so does not fit with the garbage collection semantics. However, our proposed implementation uses dynamic checks to ensure that an arena free operation is safe. The implementation guarantees that, if an arena free operation is unsafe, the program will be terminated before any incorrect behavior happens. We have implemented arenas at Google, and have shown savings of up to 15% in CPU and memory usage for a number of large applications, mainly due to reduction in garbage collection CPU time and heap memory usage.
Background
Go is a garbage-collected language. Application code does not ever explicitly free allocated objects. The Go runtime automatically runs a garbage-collection algorithm that frees allocated objects some time after they become unreachable by the application code. The automatic memory management simplifies the writing of Go applications and ensures memory safety.
However, large Go applications spend a significant amount of CPU time doing garbage collection. In addition, the average heap size is often significantly larger than necessary, in order to reduce the frequency at which the garbage collector needs to run.
Non-garbage-collected languages also have significant memory allocation and de-allocation overhead. In order to deal with complex applications where objects have widely varying lifetimes, non-garbage-collected languages must have a general-purpose heap allocator. Because of the differing sizes and lifetimes of the objects being allocated, such an allocator must have fairly complex code for finding memory for a new object and dealing with memory fragmentation.
One approach to reducing the allocation overhead for non-garbage-collected languages is region-based memory management, also known as arenas. The idea is that applications sometimes follow a pattern where a code segment allocates a large number of objects, manipulates those objects for a while, but then is completely done with those objects, and so frees all (or almost all) of the objects at roughly the same time. The code segment may be allocating all the objects to compute a result or provide a service, but has no need for any of the objects (except possibly a few result objects) when the computation is done.
In such cases, region-based memory allocation using an arena is useful. The idea is to allocate a large region of memory called an arena at the beginning of the code segment. The arena is typically a contiguous region, but may be extensible in large chunk sizes. Then all the objects can be allocated very efficiently from the arena. Typically, the objects are just allocated consecutively in the arena. Then at the end of the code segment, all of the allocated objects can be freed with very low overhead by just freeing the arena. Any result object that is intended to be longer-lived and last past the end of the code segment should not be allocated from the arena or should be fully copied before the arena is freed.
Arenas have been found to be useful for a number of common programming patterns, and when applicable, can reduce memory management overhead in non-garbage collected languages. For instance, for a server serving memory-heavy requests, each request is likely independent, so most or all of the objects allocated while serving a particular request can be freed when the request has been fulfilled. Therefore, all the objects allocated during the request can be allocated in an arena, and then freed all at once at the completion of the request.
In a related vein, arenas have been useful for protocol buffer processing, especially when unmarshalling the wire format into the in-memory protocol message object. Unmarshalling a message's wire format to memory can create many large objects, strings, arrays, etc., because of the complexity of messages and the frequent nesting of sub-messages inside other messages. A program may often unmarshal one or more messages, make use of the in-memory objects for a period of time, and then be done with those objects. In this case, all of the objects created while unmarshalling the message(s) can be allocated from an arena and freed all at once. The C++ protocol buffer documentation provides an example of using arenas. Arenas may similarly be useful for other kinds of protocol processing, such as decoding JSON.
We would like to get some of the benefits of arenas in the Go language. In the next section, we propose a design of arenas that fits with the Go language and allows for significant performance benefits, while still ensuring memory safety.
Note that there are many applications where arenas will not be useful, including applications that don't do allocation of large amounts of data, and applications whose allocated objects have widely varying lifetimes that don't fit the arena allocation pattern. Arenas are intended as a targeted optimization for situations where object lifetimes are very clear.
Proposal
We propose the addition of a new arena
package to the Go standard library. The arena package will allow the allocation of any number of arenas. Objects of arbitrary type can be allocated from the memory of the arena, and an arena automatically grows in size as needed. When all objects in an arena are no longer in use, the arena can be explicitly freed to reclaim its memory efficiently without general garbage collection. We require that the implementation provide safety checks, such that, if an arena free operation is unsafe, the program will be terminated before any incorrect behavior happens.
For maximum flexibility, we would like the API to be able to allocate objects and slices of any type, including types that can be generated at run-time via reflection.
We propose the following API:
package arena
type Arena struct {
// contains filtered or unexported fields
}
// New allocates a new arena.
func New() *Arena
// Free frees the arena (and all objects allocated from the arena) so that
// memory backing the arena can be reused fairly quickly without garbage
// collection overhead. Applications must not call any method on this
// arena after it has been freed.
func (a *Arena) Free()
// New allocates an object from arena a. If the concrete type of objPtr is
// a pointer to a pointer to type T (**T), New allocates an object of type
// T and stores a pointer to the object in *objPtr. The object must not
// be accessed after arena a is freed.
func (a *Arena) New(objPtr interface{})
// NewSlice allocates a slice from arena a. If the concrete type of slicePtr
// is *[]T, NewSlice creates a slice of element type T with the specified
// capacity whose backing store is from the arena a and stores it in
// *slicePtr. The length of the slice is set to the capacity. The slice must
// not be accessed after arena a is freed.
func (a *Arena) NewSlice(slicePtr interface{}, cap int)
The application can create an arbitrary number of arenas using arena.New
, each with a different lifetime. An object with a specified type can be allocated in a particular arena using a.New
, where a
is an arena. Similarly, a slice with a specified element type and capacity can be allocated from an arena using a.NewSlice
. Because the object and slice pointers are passed via an empty interface, any type can be allocated. This includes types that are generated at run-time via the reflect
library, since a reflect.Value
can be converted easily to an empty interface.
The application explicitly frees an arena and all the objects allocated from the arena using a.Free
. After this call, the application should not access the arena again or dereference a pointer to any object allocated from this arena. The implementation is required to cause a run-time erro and terminate the Go program if the application accesses any object whose memory has already been freed. The associated error message should indicate that the termination is due to access to an object in a freed arena. In addition, the implementation must cause a panic or terminate the Go program if a.New
or a.NewSlice
is called after a.Free
is called. a.New
and a.NewSlice
should also cause a panic if they are called with an argument which is not the correct form (**T
for a.New
and *[]T
for a.NewSlice
).
Here is some sample code as an example of arena usage:
import (
“arena”
…
)
type T struct {
val int
}
func main() {
a := arena.New()
var ptrT *T
a.New(&ptrT)
ptrT.val = 1
var sliceT []T
a.NewSlice(&sliceT, 100)
sliceT[99] .val = 4
a.Free()
}
There may be an implementation-defined limit, such that if the object or slice requested by calls to a.New
or a.NewSlice
is too large, the object cannot be allocated from the arena. In this case, the object or slice is allocated from the heap. If there is such an implementation-defined limit, we may want to have a way to expose the limit. We’ve listed it as one of the possible metrics mentioned in the “Open Issues” section. An alternate API would be to not allocate the object or slice if it is too large and instead leave the pointer arguments unchanged. This alternate API seems like it would be more likely to lead to programming mistakes, where the pointer arguments are not properly checked before being accessed or copied elsewhere.
For optimization purposes, the implementation is allowed to delay actually freeing an arena or its contents. If this optimization is used, the application is allowed to proceed normally if an object is accessed after the arena containing it is freed, as long as the memory of the object is still available and correct (i.e. there is no chance for incorrect behavior). In this case, the improper usage of arena.Free
will not be detected, but the application will run correctly, and the improper usage may be detected during a different run.
The above four functions are the basic API, and may be sufficient for most cases. There are two other API calls related to strings that are fairly useful. Strings in Go are special, because they are similar to slices, but are read-only and must be initialized with their content as they are created. Therefore, the NewSlice
call cannot be used for creating strings. NewString
below allocates a string in the arena, initializes it with the contents of a byte slice, and returns the string header.
// NewString allocates a new string in arena a which is a copy of b, and
// returns the new string.
func (a *Arena) NewString(b []byte) string
In addition, a common mistake with using arenas in Go is to use a string that was allocated from an arena in some global data structure, such as a cache, which that can lead to a run-time exception when the string is accessed after its arena is freed. This mistake is understandable, because strings are immutable and so often considered separate from memory allocation. To deal with the situation of a string whose allocation method is unknown, HeapString
makes a copy of a string using heap memory only if the passed-in string (more correctly, its backing array of bytes) is allocated from an arena. If the string is already allocated from the heap, then it is returned unchanged. Therefore, the returned string is always usable for data structures that might outlast the current arenas.
// HeapString returns a copy of the input string, and the returned copy
// is allocated from the heap, not from any arena. If s is already allocated
// from the heap, then the implementation may return exactly s. This function
// is useful in some situations where the application code is unsure if s
// is allocated from an arena.
func HeapString(s string) string
Of course, this issue of mistakenly using an object from an arena in a global data structure may happen for other types besides strings, but strings are a very common case for being shared across data structures.
We describe an efficient implementation of this API (with safety checks) in the "Implementation" section. Note that the above arena API may be implemented without actually implementing arenas, but instead just using the standard Go memory allocation primitives. We may implement the API this way for compatibility on some architectures for which a true arena implementation (including safety checks) cannot be implemented efficiently.
Rationale
There are a number of possible alternatives to the above API. We discuss a few alternatives, partly as a way to justify our above choice of API.
Removing Arena Free
One simple adjustment to the above API would be to eliminate the arena Free
operation. In this case, an arena would be freed automatically only by the garbage collector, once there were no longer any pointers to the arena itself or to any objects contained inside the arena. The big problem with not having a Free
operation is that arenas derive most of their performance benefit from more prompt reuse of memory. Though the allocation of objects in the arena would be slightly faster, memory usage would likely greatly increase, because these large arena objects could not be collected until the next garbage collection after they were no longer in use. This would be especially problematic, since the arenas are large chunks of memory that are often only partially full, hence increasing fragmentation. We did prototype this approach where arenas are not explicitly freed, and were not able to get a noticeable performance benefit for real applications. An explicit Free
operation allows the memory of an arena to be reused almost immediately. In addition, if an application is able to use arenas for almost all of its allocations, then garbage collection may be mostly unneeded and therefore may be delayed for quite a long time.
APIs that directly return the allocated objects/slices
An alternate API with similar functionality, but different feel, would replace (*Arena).New
and (*Arena).NewSlice
with the following:
// New allocates an object of the given type from the arena and returns a
// pointer to that object.
func (a *Arena) New(typ reflect.Type) interface{}
// NewSlice allocates a slice of the given element type and capacity from the
// arena and returns the slice as an interface. The length of the slice is
// set to the capacity.
func (a *Arena) NewSlice(typ reflect.Type, cap int) interface{}
An example of usage would be:
a := arena.New()
floatPtr := a.New(reflect.TypeOf(float64(0))).(*float64)
byteSlice := a.NewSlice(reflect.TypeOf(byte(0)), 100).([]byte)
This API potentially seems simpler, since it returns the allocated object or slice directly, rather than requiring that a pointer be passed in to indicate where the result should be stored. This allows convenient use of Go’s idiomatic short variable declaration, but does require type assertions to convert the return value to the correct type. This alternate API specifies the types to be allocated using reflect.Type
, rather than by passing in an interface value that contains a pointer to the required allocation type. For applications and libraries that already work on many different types and use reflection, specifying the type using reflect.Type
may be convenient. However, for many applications, it may seem more convenient to just pass in a pointer to the type that is required.
There is an efficiency distinction in the NewSlice
call with the two choices. In the NewSlice
API described in the "Proposal" section, the slice header object is already allocated in the caller, and only the backing element array of the slice needs to be allocated. This may be all that is needed in many cases, and hence more efficient. In the new API in this section, the Slice
call must allocate the slice object as well in order to return it in the interface, which causes extra heap or arena allocation when they are often not needed.
Another alternative for a.New
is to pass in a pointer to type T and return a pointer to
type T (both as empty interfaces):
// New, given that the concrete type of objPtr is a pointer to type T,
// allocates an object of type T from the arena a, and returns a pointer to the
// object.
func (a *Arena) New(objPtr interface{}) interface{}
An example use of this API call would be: intPtr := a.New((*int)(nil)).(*int)
. Although this also allows the use of short variable declarations and doesn’t require the use of reflection, the rest of the usage is fairly clunky.
Simple API using type parameterization (generics)
We could have an optional addition to the API that uses type parameterization to express the type to be allocated in a concise and direct way. For example, we could have generic NewOf
and NewSliceOf
functions:
// NewOf returns a pointer to an object of type T that is allocated from
// arena a.
func arena.NewOf[T any](a *Arena) *T
// NewSliceOf returns a slice with element type T and capacity cap
// allocated from arena a
func arena.NewSliceOf[T any](a *Arena, cap int) []T
Then we could allocate objects from the arena via code such as:
intPtr := arena.NewOf[int](a)
We don’t think these generic variants of the API can completely replace the suggested methods above, for two reasons. First, the NewOf
function can only allocate objects whose type is specified at compile-time. So, it cannot satisfy our goal to support allocation of objects whose type is computed at run-time (typically via the reflect
library). Second, generics in Go are just arriving in Go 1.18, so we don’t want to force users to make use of generics before they are ready.
Compatibility
Since this API is new, there is no issue with Go compatibility.
Implementation
In order to fit with the Go language, we require that the semantics of arenas in Go be fully safe. However, our proposed API has an explicit arena free operation, which could be used incorrectly. The application may free an arena A while pointers to objects allocated from A are still available, and then sometime later attempt to access an object allocated from A.
Therefore, we require that any implementation of arenas must prevent improper accesses without causing any incorrect behavior or data corruption. Our current implementation of the API gives a memory fault (and terminates the Go program) if an object is ever accessed that has already been freed because of an arena free operation.
Our current implementation performs well and provides memory allocation and GC overhead savings on the Linux amd64 64-bit architecture for a number of large applications. It is not clear if a similar approach can work for 32-bit architectures, where the address space is much more limited.
The basic ideas for the implementation are as follows:
- Each arena
A
uses a distinct range in the 64-bit virtual address space A.Free
unmaps the virtual address range for arenaA
- The physical pages for the arena can then be reused by the operating system for other arenas.
- If a pointer to an object in arena
A
still exists and is dereferenced, it will get a memory access fault, which will cause the Go program to terminate. Because the implementation knows the address ranges of arenas, it can give an arena-specific error message during the termination.
So, we are ensuring safety by always using a new range of addresses for each arena, in order that we can always detect an improper access to an object that was allocated in a now-freed arena.
The actual implementation is slightly different from the ideas above, because arenas grow dynamically if needed. In our implementation, each arena starts as a large-size "chunk", and grows incrementally as needed by the addition of another chunk of the same size. The size of all chunks is chosen specifically to be 64 MB (megabytes) for the current Go runtime on 64-bit architectures, in order to make it possible to recycle heap meta-data efficiently with no memory leaks and to avoid fragmentation.
The address range of these chunks do not need to be contiguous. Therefore, when we said above that each arena A uses a distinct range of addresses, we really meant that each chunk uses a distinct range of addresses.
Each chunk and all the objects that it contains fully participate in GC mark/sweep until the chunk is freed. In particular, as long as a chunk is part of an arena that has not been freed, it is reachable, and the garbage collector will follow all pointers for each object contained in the chunk. Pointers that refer to other objects contained in the chunk will be handled very efficiently, while pointers to objects outside the chunk will be followed and marked normally.
The implementation calls SetFinalizer(A, f)
on each arena A as it is allocated, where f
calls A.Free
. This ensures that an arena and the objects allocated from it will eventually be freed if there are no remaining references to the arena. The intent though is that every arena should be explicitly freed before its pointer is dropped.
Because unmapping memory is relatively expensive, the implementation may continue to use a chunk for consecutively allocated/freed arenas until it is nearly full. When an arena is freed, all of its chunks that are filled up are immediately freed and unmapped. However, the remaining part of the current unfilled chunk may be used for the next arena that is allocated. This batching improves performance significantly.
Because of the large 64-bit address space, our prototype implementation has not required reusing the virtual addresses for any arena chunks, even for quite large and long-running applications. However, the virtual addresses of most chunks can eventually be reused, since there will almost always be no more reachable pointers to anywhere in the chunk. Since the garbage collector sees all reachable pointers, it can determine when an address range can be reused.
The implementation described above demonstrates that it is possible to implement the Arena API for 64-bit architectures with full safety, while still providing performance benefits. Many other implementations are possible, and some may be tuned for other types of usage. In particular, because of the 64 MB chunk size, the above implementation may not be useful for applications that need to create a large number of arenas that are live at the same time (possibly because of many concurrent threads). It is probably most appropriate that there should only be a few to 10's of arenas in use at any one time. Also, it is not intended that arenas be shared across goroutines. Each arena has a lock to protect against simultaneous allocations by multiple goroutines, but it would be very inefficient to actually use the same arena for multiple goroutines. Of course, that would rarely make sense anyway, since the lifetimes of objects allocated in different goroutines are likely to be quite different.
Open issues
Another possibility in the design space is to implement the API described in the "Proposal" section, but without the safety checks, or with an option to disable the safety checks. The idea here is that the performance savings from the use of arenas can be increased by doing an implementation that doesn't have safety guarantees. As compared to the implementation described above, we can avoid the mapping and unmapping overhead, and reuse the memory of an arena much more quickly (and without OS involvement) when it is freed. We have done a prototype of such an implementation, which we call "unsafe arenas". We have seen an additional 5-10% improvement in performance in some cases when using unsafe arenas rather than our safe arena implementation. However, we feel very strongly that arenas in Go need to be safe. We do not want the use of arenas to lead to memory bugs that may be very hard to detect and debug, and may silently lead to data corruption. We think that it is better to continue to optimize the implementation of safe arenas, rather than trying to support unsafe arenas.
It would be useful to have some run-time metrics associated with arenas. The desired metrics will depend somewhat on the final API, so we have not yet tried to decide the exact metrics that will cover the application needs. However, here are some metrics which might be useful:
- the number of arena created and freed
- the number of current arenas and the maximum number of arenas that have been active at one time
- the total number of bytes allocated via arenas, and the average number of bytes allocated per arena
- the (constant) limit on the largest-size object or slice that can be allocated from an arena
Another open issue is whether arenas can be used for allocating the elements of a map. This is
possible, but it is not clear what a good API would be. Also, there might be unusual cases if the arena used for the main map object is different from the arena used to allocate new elements of the map. With generics arriving in Go 1.18, generic maps (or hash tables) can now be implemented in user libraries. So, there could be a user-defined generic map implementation that allows optionally specifying an arena for use in allocating new elements. This might be the best solution, since that would allow for greater flexibility than adjusting the semantics of the built-in map
type.
Protobuf unmarshalling overheads
As noted above, arenas are often quite useful for reducing the allocation and GC overhead associated with the objects that are created as a protobuf message is being unmarshaled. We have prototyped changes to the protobuf package which allow for providing an arena as the allocation option for objects created during unmarshalling. This arrangement makes it quite easy to use arenas to reduce the allocation and GC overhead in applications that make heavy use of protobufs (especially unmarshalling of large protobufs). If the arena proposal is accepted and implemented in Go, then it would make sense to extend the protobuf package to provide such an arena allocation option.
What is the reason to add this to the standard library as opposed to building a third party package?
Your specification says:
The application explicitly frees an arena and all the objects allocated from the arena using a.Free. After this call, the application should not access the arena again or dereference a pointer to any object allocated from this arena. The implementation is required to cause a run-time erro and terminate the Go program if the application accesses any object whose memory has already been freed.
If I read it correctly, this means that it is permitted to keep pointers into a released arena (i.e. stray pointers) as long as you do not explicitly dereference them. However, this means that the compiler must now be careful not to dereference any pointer it knows not to be nil
unless user code explicitly does so, lest it be a pointer into a released arena
. This seems like it would significantly reduce the potential for optimisation as otherwise, the compiler seems to be allowed to perform such accesses, knowing that each reachable object can also be dereferenced safely.
If the rules were tightened to say that you have to erase all pointers into an arena
before calling Free
, not only would these optimisations be enabled, but there would also be a way to reclaim the address space occupied by the arena: the garbage collector could be programmed to check if any pointers into the arena address space remain and if there aren't any, it could allow the address space to be reclaimed. If it finds a stray pointer, it could likewise abort the program in much the same manner as when you dereference a stray pointer.
Another benefit is that it's less likely to have tricky edge cases where stray pointers could remain in some data structures (e.g. as the result of some string manipulation or append
operations which may only some times return a pointer to one of their arguments), causing them to be dereferenced later only under specific, hard to reconstruct circumstances. By prohibiting the presence of any stray pointers after a call to Free
, this kind of error would be much easier to find.
With this issue addressed, I'm very interested in this proposal. It will be very useful for complex temporary data structures that need to be built step by step and deallocated all at once.
@hherman1 as far as i understood this proposal, there are a lot of troubles for example in appending to slices, e.g. the code could be more readable if you just use append
, without calling arena-specific methods.
Even though, the idea to manualy handle memory freeing sounds great for me, cause there are a lot of specific cases, when you don't want to hope on the garbage collector
I'd be curious to understand the problems that an arena solves that can't be solved by either using sync.Pool, allocating a slice of a given type (for example a binary tree allocating a slice of nodes), or a combination of both techniques. It seems to me that Go provides idiomatic ways of addressing at least the specific protobuf use-case mentioned here.
In addition to the above, doesn't the Go allocator already have sizes classes? If the proposal does proceed, I think we need to see a prototype outperform the runtimes allocator and is enough gain to justify the ecosystem complexity.
We don’t think these generic variants of the API can completely replace ....
First, the NewOf function can only allocate objects whose type is specified at compile-time. So, it cannot satisfy our goal to support allocation of objects whose type is computed at run-time (typically via the reflect library).
What's the main usecase for wanting to allocate types created via reflect in arenas? Marshalling & unmarshall?
Are those usecases compelling enough? Honest question.
Second, generics in Go are just arriving in Go 1.18, so we don’t want to force users to make use of generics before they are ready.
Is there a hurry in adding arenas? It can always wait one or two release cycles before been added so that people are ready with generics.
In other words, if hypothetically speaking, the arena-generics design was the better API; we shouldn't shelve it just because generics aren't ready yet.
This proposal sounds really good to me from a user's perspective, but one detail causes concern: wouldn't something like all code eventually end up needing arena support? For example, imagine I have a web service where I want to allocate an arena for each web request. This sounds like a great use case for this, but I'd need encoding/json
to have an Arena mode, and maybe text/template
/html/template
to have Arena modes so their working sets can use the Arena. Probably same for various clients (SQL, http, etc.) Is that what you see as the path forward or am I missing something?
What is the reason to add this to the standard library as opposed to building a third party package?
It's hard to do this safely in a third party package. Consider a struct allocated in the arena that contains pointers to memory allocated outside the arena. The GC must be aware of those pointers, or it may incorrectly free ordinary-memory objects that are still referenced by arena objects. A third party package would have to somehow make the garbage collector aware of those pointers, including as the pointer values change, which is either hard or inefficient.
However, this means that the compiler must now be careful not to dereference any pointer it knows not to be nil unless user code explicitly does so
That is already true. Go code can use pointers that point to memory that was allocated by C, or that was allocated by syscall.Mmap
. The compiler already can't casually dereference a pointer.
the garbage collector could be programmed to check if any pointers into the arena address space remain and if there aren't any, it could allow the address space to be reclaimed
I believe that we can already do that. If the garbage collector sees a pointer to a freed arena, it can crash the program. If we have two complete GC cycles after an arena is freed, we can know for sure that there are no remaining pointers into that address space, and we can reclaim the addresses. However, I don't know if the current patches implement that.
I'd be curious to understand the problems that an arena solves that can't be solved by either using sync.Pool, allocating a slice of a given type (for example a binary tree allocating a slice of nodes), or a combination of both techniques. It seems to me that Go provides idiomatic ways of addressing at least the specific protobuf use-case mentioned here.
As you note, sync.Pool
only permits allocating a specific type. It's reasonable to use if all allocations are the same type. An arena is when most allocations are different types. That is the case for protobuf allocations: each protobuf is implemented as a different Go struct. It's also the case for many uses of, for example, JSON.
In addition to the above, doesn't the Go allocator already have sizes classes? If the proposal does proceed, I think we need to see a prototype outperform the runtimes allocator and is enough gain to justify the ecosystem complexity.
I'm not sure how size classes are related.
As @danscales mentions, there is already an implementation, which is in use internally at Google. It does overall outperform the runtime allocator for cases like RPC servers that transmit data as protobufs.
First, I feel this should live in the runtime
package, or at least in runtime/arena
, as it seems fairly runtime specific. Which is my main concern with this, that other implementations might not support it and then packages using it would not be usable on those implementations.
Also, a question for clarification:
It is probably most appropriate that there should only be a few to 10's of arenas in use at any one time.
One of the main usecases mentioned is protobuf decoding. ISTM that, if every call to proto.Unmarshal
creates an arena, used until that message is done with, we would very quickly outpace 10's of arenas by orders of magnitude on a loaded gRPC server.
as far as i understood this proposal, there are a lot of troubles for example in appending to slices, e.g. the code could be more readable if you just use append, without calling arena-specific methods.
AIUI the proposal would not interoperate with append
, i.e. you'd indeed have to use arena-specific methods.
What's the main usecase for wanting to allocate types created via reflect in arenas? Marshalling & unmarshall?
Are those usecases compelling enough? Honest question.
Yes, marshaling and unmarshaling. These cases are compelling for, for example, network RPC servers that must serialize and unserialize data for every request.
One implementation caveat: a 64-bit address space isn't quite the inexhaustible vastness it first seems because many environments impose tighter restrictions. (I recently used a 4GB mmap as an optimization but found OpenBSD's default ulimit is 1.5GB and some users seem to impose their own limits.)
Which reminds me:
It is not clear if a similar approach can work for 32-bit architectures, where the address space is much more limited.
Does this mean this package would use build-tags to limit itself to 64-bit architectures? And would programs using this then not run on 32-bit platforms, unless they specifically code around that? Also, this seems all the more reason to put this into runtime
, to make clear that it's platform dependent.
Hi @Merovius, my interpretation was that the API could be implemented everywhere, but it would not increase efficiency everywhere:
Note that the above arena API may be implemented without actually implementing arenas, but instead just using the standard Go memory allocation primitives. We may implement the API this way for compatibility on some architectures for which a true arena implementation (including safety checks) cannot be implemented efficiently.
@thepudds ah thank you, I missed that, alleviates my concerns around portability. Though I'd find it a bit confusing to provide the API without actual arenas. But, fair enough.
@Merovius raises an interesting question, though, around what the overhead of the non-arena implementation of the arena api would be…
As you note, sync.Pool only permits allocating a specific type. It's reasonable to use if all allocations are the same type. An arena is when most allocations are different types. That is the case for protobuf allocations: each protobuf is implemented as a different Go struct. It's also the case for many uses of, for example, JSON.
Assuming the problem with protobufs is the actual objects and not some internal buffer, here is how I would naively use sync.Pool
.
Today you might see code like this:
book := &pb.AddressBook{}
Rather I would suggest the generated AddressBook
code includes a constructor function:
book := NewAddressBook()
The protoc generated code in its package would look something like this:
var addressBookPool sync.Pool = sync.Pool{
New: func() interface{} {
return new(AddressBook)
},
}
func NewAddressBook() *AddressBook {
return addressBookPool.Get().(*AddressBook)
}
func (ab *AddressBook) Close() error {
*ab = AddressBook{}
addressBookPool.Put(ab)
return nil
}
I assume if arena existed similar changes to have a constructor that uses it would need to happen as well? If not that burden would fall to the object creator which is in fact how I use sync.Pool with protobufs today (when needed)
Now maybe more is needed say:
func (ab *AddressBook) Close() error {
ab.closeOnce.Do(func() {
*ab = AddressBook{}
addressBookPool.Put(ab)
})
return nil
}
Or maybe we need to also generate and use pools/constructors of *structs
referenced in AddressBook
(or do what some pkgs like gogoproto optionally do and not use pointers for objects nested in proto structs). Lots of nuance that is besides the point of this discussion-
But I would like to understand why the arena approach is better. I'd also be curious to bench an arena prototype against sync.Pool
and an approach like the above.
If the rules were tightened to say that you have to erase all pointers into an
arena
before callingFree
, not only would these optimisations be enabled, but there would also be a way to reclaim the address space occupied by the arena: the garbage collector could be programmed to check if any pointers into the arena address space remain and if there aren't any, it could allow the address space to be reclaimed. If it finds a stray pointer, it could likewise abort the program in much the same manner as when you dereference a stray pointer.
In addition to what Ian already said about the compiler not dereferencing stray pointers:
One reason that we would like to able to deal with pointers to objects in the arena staying around after it freed (as long as they are not used) is because it may be quite tricky and disruptive to have to remove them from the stack. A pointer to an object in the arena could have been passed around as an arg and is still on the stack as an argument or local variable, but will not be used again. It would be good to ensure that the programmer does not have to nil out these pointers on the stack just to satisfy arenas, when they will naturally go away later.
We don’t think these generic variants of the API can completely replace ....
Second, generics in Go are just arriving in Go 1.18, so we don’t want to force users to make use of generics before they are ready.
Is there a hurry in adding arenas? It can always wait one or two release cycles before been added so that people are ready with generics. In other words, if hypothetically speaking, the arena-generics design was the better API; we shouldn't shelve it just because generics aren't ready yet.
Yes, I agree. There's no rush to figure out and add arenas, so it may be worth waiting to use the generics API if there's consensus on it being a better API. The first point does remain - which is that we would prefer to have some API that allows for allocating a type that is created/calculated at runtime. However, the generics API could become the main API, and the one that allows for dynamic types could be the secondary API.
This proposal sounds really good to me from a user's perspective, but one detail causes concern: wouldn't something like all code eventually end up needing arena support? For example, imagine I have a web service where I want to allocate an arena for each web request. This sounds like a great use case for this, but I'd need
encoding/json
to have an Arena mode, and maybetext/template
/html/template
to have Arena modes so their working sets can use the Arena. Probably same for various clients (SQL, http, etc.) Is that what you see as the path forward or am I missing something?
Yes, that's a good point. If we have a package that is allocating lots of memory (often by doing decoding of wire format) and we want to use arenas with it, then we will have to plumb in a way to send down an optional allocator function that it should use if enabled (in our case, the allocator function would use an arena). That's what we have prototyped for the protobuf unmarshaling library calls (see last section). I don't see any easy way to make packages use arenas, but open to suggestions.
Hi @danscales
If we have a package that is allocating lots of memory (often by doing decoding of wire format) and we want to use arenas with it, then we will have to plumb in a way to send down an optional allocator function that it should use if enabled (in our case, the allocator function would use an arena).
Is there a rough estimate of how many new APIs this might translate to in the standard library, for example?
Also, a question for clarification:
It is probably most appropriate that there should only be a few to 10's of arenas in use at any one time.
One of the main usecases mentioned is protobuf decoding. ISTM that, if every call to
proto.Unmarshal
creates an arena, used until that message is done with, we would very quickly outpace 10's of arenas by orders of magnitude on a loaded gRPC server.
Yes, that is true. It is possible there is another implementation that could deal with that many arenas simultaneously. But I will note that if you have 100s of requests in flight at one time, then they may not be allocating that much new data while serving each individual request, so arenas may not really be required in that case. Arenas are really more useful/appropriate for unmarshalling large data objects for which you may do a lot of processing. sync.Pool may be more appropriate if the allocated data per request is small (since there may also be only a limited number of known object types).
Yes, I agree. There's no rush to figure out and add arenas, so it may be worth waiting to use the generics API if there's consensus on it being a better API. The first point does remain - which is that we would prefer to have some API that allows for allocating a type that is created/calculated at runtime. However, the generics API could become the main API, and the one that allows for dynamic types could be the secondary API.
Is there any idea of (A) what is the net cost of doing this with reflection and (B) will it be possible to not do it with reflection when using the generics API?
Yes, that is true. It is possible there is another implementation that could deal with that many arenas simultaneously. But I will note that if you have 100s of requests in flight at one time, then they may not be allocating that much new data while serving each individual request, so arenas may not really be required in that case. Arenas are really more useful/appropriate for unmarshalling large data objects for which you may do a lot of processing. sync.Pool may be more appropriate if the allocated data per request is small (since there may also be only a limited number of known object types).
What message sizes/throughputs was the prototype tested at? Many small requests/responses is certainly the norm in my background, but I know google was looking hard (I believe?) a Vitess blog post that mainly dealt with very large data replication messages. Certainly if proto performance gets worse than status quo at high throughput levels, many people wouldn't be thrilled with that.
This proposal sounds really good to me from a user's perspective, but one detail causes concern: wouldn't something like all code eventually end up needing arena support? For example, imagine I have a web service where I want to allocate an arena for each web request. This sounds like a great use case for this, but I'd need
encoding/json
to have an Arena mode, and maybetext/template
/html/template
to have Arena modes so their working sets can use the Arena. Probably same for various clients (SQL, http, etc.) Is that what you see as the path forward or am I missing something?Yes, that's a good point. If we have a package that is allocating lots of memory (often by doing decoding of wire format) and we want to use arenas with it, then we will have to plumb in a way to send down an optional allocator function that it should use if enabled (in our case, the allocator function would use an arena). That's what we have prototyped for the protobuf unmarshaling library calls (see last section). I don't see any easy way to make packages use arenas, but open to suggestions.
Add memory allocator to context? It seems really silly but doing a second round of "context-aware" updates to add context in more places seems better to me than doing a second round of "context-aware" updates that adds an entirely new type of object that ends up needing to be sent everywhere.
Hi all,
One quick thought: could you utilize the write barrier that is currently used by the compiler/GC to detect writes to arena allocated memory outside of the arena. Then on .Free, copy the values that are referenced out to the main heap?
This would let you effectively use an arena as a large scratch area, with the GC assisting in figuring out which values are not just scratch.
Is the code for the experimental protobuf implementation published? The scope of code that needs changing in order to take advantage of per-request arenas seems huge, but I'm eager to see a real world example in hopes I'm wrong!
@tarndt A protobuf message can include many different messages (as they are called) arranged in a complex memory layout. This is all relatively seamless for the program but it means that unmarshaling a protobuf isn't just a matter of allocating a single object type, it can involve dozens of different types. So you would need dozens of different sync.Pool
structures. And in order to use the pools effectively you would need a complicated piece of code to release all the data back to the appropriate pools, which is much more complicated, and slower, than releasing an arena.
Frankly an arena sounds a lot simpler. And let's not forget that an arena lets you notify the runtime exactly when the memory is no longer needed. A pool will stick around in a much less predictable fashion.
@evanphx I think that would require enabling the write barrier at all times, which would be quite a bit slower than the current approach.
I'm in favor of this. Getting pooling and other allocation-related performance bits correct has been a challenge for us with Thrift as well, so the advantages can go beyond protobuf. I think arenas would simplify things considerably, both in terms of implementation and in ease of ensuring correctness.
@danscales: since you already have a proposed implementation for the feature, I think it would make discussing the proposal as a whole much easier if you shared its CL. Cheers!
I like the general idea. I think in some places like e.g. creating an AST when parsing files the performance benefits can be literally tenfold.
However, the generics API could become the main API, and the one that allows for dynamic types could be the secondary API.
As I was reading the original proposal, this was definitely the main idea that came to mind. Using generics feels more natural and easier for the majority of cases, so it should be the main API, I think. I also believe that adding a generic std API in Go 1.19 would be completely fine, for example - especially given how the use of generics here would be fairly simple.
I don't see any easy way to make packages use arenas, but open to suggestions.
I don't have a good solution for you here, but I think it's worth noting that we've been in a very similar situation before: contexts. Some APIs fixed this by adding a second FooContext
function, while others added a context field to a struct.
There's always the option of an existing API taking an arena via context.WithValue
, though I seem to recall that each call to WithValue
incurs an allocation of its own. Passing an arena as part of a context could still be a worthwhile idea to experiment on, though, as it could make adoption a lot easier for a significant portion of existing stable libraries.
Personally, I don't very much like the idea of passing around allocators, context or not. I don't feel that traversing the context-chain on every allocation is a very good idea. I also don't want to have to add context.Context
arguments to any call to data-structures which might need to allocate - heap.Insert(ctx, v)
feels wrong. context.Context
has a definite connotation with cancellation and network-traversal to me.
IMO this should ~always be treated as an implementation detail by packages who know that they would benefit from it. For example, it might be possible for proto.Unmarshal
to transparently use an arena if it is willing to rely on a Finalizer to call Free
. The API of the proto
package would then not change at all. The top-level message to pass to proto.Unmarshal
would still have to be allocated on the heap, but at least all the nested sub-messages could be allocated in the arena when the proto
package recurses. You'd get most of the benefit without having to plumb allocators.
And even if we're not willing to do that, the way to go would IMO be for the proto package to expose a UnmarshalArena[M Message](a *arena.Arena, b []byte) *M
function, which is then called by the gRPC server (for example), which directly allocates the arena in grpc.NewServer
- without giving the ability to plumb it further than that.
i.e. IMO the level of plumbing should be kept to an absolute minimum. In particular, I don't see a reason why protobuf generated code would need to mention arenas at all, FWIW.
If we do plumb, I'd argue that yes, we should add Allocator
fields/arguments to any data structure/type/function which would benefit (combined with a type allocator.Heap struct{}
which uses new
and make
). Making it explicit would serve as an incentive to keep the plumbing depth shallow anyways.
Maybe the best way to put it is that I see arenas as very much alike to sync.Pool
. We don't pass those around across API boundaries either.
I think that that would cause problems in cases where you need to, for instance, unmarshal a large corpus of different small json files. Some arena is better than none, probably, but it stings to be just incapable of using this stdlib feature with other parts of the stdlib in a use case it was clearly intended for.
@danscales can you share more details on the applications you tested the arena protoype implementation with? What kind of applications are they, how much data do they process, what are their performance requirements?
One of the things I find a bit unclear about this proposal is exactly who it's supposed to benefit. You note that:
It is probably most appropriate that there should only be a few to 10's of arenas in use at any one time.
and
[I]f you have 100s of requests in flight at one time, then they may not be allocating that much new data while serving each individual request, so arenas may not really be required in that case. Arenas are really more useful/appropriate for unmarshalling large data objects for which you may do a lot of processing.
This seems to restrict the subset of apps that would benefit from this incarnation of arenas somewhat significantly. The added focus on protobufs makes me think that the authors have a very specific set of applications in mind.
Is such an intrusive change worth the 15% speed gain for a subset of apps of unknown size?
@CannibalVox For json specifically, sorry, just realized this is pretty much nonsense. As the objects might outlast the Decoder, the lifetime of the arena can't be tied to it - and in general, whenever an arena outlasts a function call, the caller probably needs to be aware of it. My bad.json.NewDecoder
could allocate an arena, which would allow you to pipe the concatenated JSON data in. More generally, there are often ways to write APIs which allow you to take advantage of arenas without actually mentioning arenas in your API.
a common mistake with using arenas in Go is to use a string that was allocated from an arena in some global data structure, such as a cache, which that can lead to a run-time exception when the string is accessed after its arena is freed. This mistake is understandable, because strings are immutable and so often considered separate from memory allocation.
Could this be encoded as a separate type instead? With generics, at least the language has the expressivity of Arena[string]
, which would also work for non-strings. But if this proposal requires tying in with the compiler/runtime, perhaps the string case should be a completely separate type that the compiler treats like a string
in most contexts?
On @Merovius's point
and in general, whenever an arena outlasts a function call, the caller probably needs to be aware of it.
Wouldn't arenas open a can of worms where any API method could potentially be returning an arena pointer that you need to be aware of?
One line I like to use to explain how Go differs from C and C++ is, "in Go, every pointer is either valid or nil." That line has caveats for unsafe, cgo, syscalls, and any other ways to work outside the type system, of course.
The proposed mechanism here provides a "safe" way to obtain pointers to dead objects, which may or may not terminate the program upon dereference. If it is accepted, then my line instead becomes, "in Go, every pointer is either valid or nil, except those that came from an arena, which you have no way to detect in general." This is a different sense of "memory safety" than what we have now.
I also agree that in Go code we must be explicit when some code is not memory-safe (or at least dangerous), and arenas seem like such a thing to me.
Otherwise it opens up a lot of possibilities for subtle errors that were not possible previously: e.g. if we have a transparent arena allocation on every web request for protobuf, for example, it would also mean that all fields in structs that are unmarshalled using protobuf are also not memory-safe, including string
s. This would mean that if by any chance someone wants to have e.g. a global cache of some structs keyed by an object id, they would probably think about cloning the struct itself that was allocated on the arena, but I doubt that it is obvious that e.g. a key inside that struct, such as an ID string
field would also not be safe to be used in a map.
So, with transparent arena-allocated structs you could very easily write (incorrect) code like the following (mutexes removed for brevity):
var globalCache map[string]*someObject
...
func handleWebRequest(obj *someObject) {
globalCache[obj.ID] = proto.Clone(obj) // obj.ID is also arena-allocated, so it's not safe to use it as a map key
}
Of course, even with explicit arenas as arguments everywhere it would be easy to make such a mistake, but similar to how Go handles errors, it at least makes you think about the implications of using an arena more and that's a good thing.
@zephyrtronium ISTM that means you have to add arenas to the list "unsafe, cgo, syscalls and any other way to work outside the type system". Also, race conditions.
But honestly, I feel like "every pointer is either nil or invalid" is already overly simplistic. For example, consider this:
type X struct { m map[int]string }
func (x *X) F(a int, b string) { x.m[a] = b }
new(X)
is a non-nil pointer. But is it "valid"? Calling its method will panic. I'd argue that makes it invalid. The Go typesystem just doesn't encode that invariant.
But yes, arenas definitely are a thing that should be considered with care. And we should discourage APIs which are easy to misuse.
Hm. I think this discussion begs the question:
The implementation calls SetFinalizer(A, f) on each arena A as it is allocated, where f calls A.Free. This ensures that an arena and the objects allocated from it will eventually be freed if there are no remaining references to the arena. The intent though is that every arena should be explicitly freed before its pointer is dropped.
Why is that the intent? ISTM an arena does not represent a resource beyond memory, so ISTM that they are exactly what finalizers are useful for. A lot of the discussion about safety could be solved by relying on finalizers and the GC to call Free, instead of expecting the programmer to do it and expose a source of mistakes. So why don't we?
new(X) is a non-nil pointer. But is it "valid"? Calling its method will panic. I'd argue that makes it invalid. The Go typesystem just doesn't encode that invariant.
@Merovius I'd argue that that's a slightly different form of validity. The pointer returned by new(X)
is a valid pointer. It's just that when you call the method on it, the object is not properly constructed/initialised.
It might be reasonable to make the difference in "memory safety" more concrete, now that we have generics. We could have arenas work with an arena.Pointer[T]
type instead of plain pointers, with only a Deref() T
method (and maybe a HeapCopy() *T
). This makes it much clearer where arena pointers are used, and it makes documentation on the dangers more accessible. Similar types would make sense for slices and strings.
To deal with the situation of a string whose allocation method is unknown,
HeapString
makes a copy of a string using heap memory only if the passed-in string (more correctly, its backing array of bytes) is allocated from an arena.
Existing Go programs — and, especially, existing Go libraries — have been written with Go's “immutable string” semantics in mind. No existing packages written against mainline Go call HeapString
(because it doesn't exist yet), so with that approach, in order for a program that uses arenas to be memory-safe it must call HeapString
on every string passed (even indirectly) to any non-arena-aware package.
I believe that that approach makes arena-allocated string
values much too error-prone. They would appear to be safe to pass as type string
, but the type string
would now refer to two types that are fundamentally different:
-
Ordinary, immutable, heap-allocated
string
values, which are garbage-collected and safe to use ~everywhere.- Storing a short fragment of a
string
in a global data structure today may over-retain a larger backing allocation, but that allocation will show up on memory profiles and — crucially! — will not cause the program to crash or produce corrupted output or undefined behavior.
- Storing a short fragment of a
-
Arena-allocated
string
values, which are functionally more like slices: they are no longer safe to read after a call returns, and not guaranteed to have stable contents over time.
To me, those factors imply that the “arena-allocated contiguous slice of bytes” type should be []byte
or some newly-defined “unsafe string” type, not string
proper. The argument I could see in favor of string
is that some APIs operate on type string
instead of []byte
— but wasn't part of the point of adding generics so that we could abstract over the differences between immutable strings and ephemeral slices for APIs that don't care about those differences?
To draw from existing experience in other languages: because Java objects tend to be mutable, Java programs are often littered with defensive copying, which not only makes them more difficult to reason about, but also adds a lot of allocation and copying overhead that would not be needed in a program that distinguishes between mutable and immutable values (as Go programs do today).
@zephyrtronium ISTM that means you have to add arenas to the list "unsafe, cgo, syscalls and any other way to work outside the type system". Also, race conditions.
I think you missed my point. Arenas as proposed look like plain, normal types that give plain, normal pointers. They are not. They work outside the existing Go rules, requiring direct support from the runtime per #51317 (comment). Unsafe, cgo, and syscalls explicitly indicate that they are not Go. Race conditions are not part of the Go language. Does it make sense to just say that this new package is not part of Go?
But honestly, I feel like "every pointer is either nil or invalid" is already overly simplistic.
Sure, I could be more specific and say that "there are no dangling pointers in Go," which is still a condition violated by this proposal. I consider new(X)
in your example to be a valid pointer to an object containing a nil pointer, as @muscar said, but that would be an argument about what "valid" means, and I think off-topic here.
Is the proposed Arena type meant to be safe for concurrent use?
Hi @zephyrtronium, the proposal includes:
Also, it is not intended that arenas be shared across goroutines. Each arena has a lock to protect against simultaneous allocations by multiple goroutines, but it would be very inefficient to actually use the same arena for multiple goroutines. Of course, that would rarely make sense anyway, since the lifetimes of objects allocated in different goroutines are likely to be quite different.
@zephyrtronium I think you missed my point as well :) The pointers returned by arenas as proposed might seem special. But they are not that special, really. There are plenty of examples in the Go language where functions return pointers with caveats on when and how they can be used, lest your program panics, be it that they can't be modified, the pointed to values can't be copied or that they can't be retained after a callback returns. All of these conditions are error-prone, but we live with them just fine.
They are also set apart from the other examples you mention (unsafe, cgo and syscall) in that they are still safe. Your program might panic, if you violate their invariants, but it won't have undefined effects.
It might be reasonable to make the difference in "memory safety" more concrete, now that we have generics. We could have arenas work with an
arena.Pointer[T]
type instead of plain pointers, with only aDeref() T
method (and maybe aHeapCopy()
*T). This makes it much clearer where arena pointers are used, and it makes documentation on the dangers more accessible.
IMO that is a bad idea. It would make it all but impossible to use them without polluting your API with arena usage. For example, the flagship use-case of protobuf encoding would now require all nested messages to be this new, special pointer type. As I said above, I don't think we should, generally, let arena usage cross API boundaries.
There are plenty of examples in the Go language where functions return pointers with caveats on when and how they can be used
For a specific example, consider io.Reader
and io.Writer
. The Read
and Write
methods are explicitly not allowed to retain the argument slice after the call has returned, because the caller may mutate it concurrently and any access would be racy.
All of these conditions are error-prone, but we live with them just fine.
That's not a reason to add even more edge cases to the language :).
Your program might panic, if you violate their invariants, but it won't have undefined effects.
That's technically true. But it makes reasoning about pointer lifetimes trickier. One of the appeals of GC'd languages if that you don't have to worry about lifetimes most of the time. This sounds like a decision that shouldn't be made lightly.
It would make it all but impossible to use them without polluting your API with arena usage.
That is not necessarily bad. Unsafe pointers are special, and they are marked as such. Arena pointers are special, and they should be marked as such if arenas make it into the language. But again, the proposal as it stands sounds like a heavy price to pay for the gains outlined.
One of the appeals of GC'd languages if that you don't have to worry about lifetimes most of the time.
I don't think that's actually true, especially given mutability and concurrency. Go programmers do have to worry about lifetimes today, because they have to worry about aliasing bugs, data races, and memory leaks (especially via goroutine leaks and global variables).
The major question for me is: how much additional worry about lifetimes would this proposal introduce? For types other than string
, I think the answer is almost none: a Go library that today is free of data races, aliasing bugs, and memory leaks, would still be free of those classes of bugs if passed arena-allocated arguments.
Should runtime (or compiler / go vet when possible to deduce statically) then just outright panic when using arena-allocated string
in map
s that are heap-allocated? If so it would solve the main issue I personally have with arena allocation being transparent for users.
There is still a question about e.g. how append
would work (even if arena usage is transparent) given that it reallocates underlying data every time the slice grows. There is a lot of code that doesn't expect append
without initial capacity to be very expensive. Not sure how big of a deal it is, but it might be in some cases.
I don't think that's actually true, especially given mutability and concurrency. Go programmers do have to worry about lifetimes today, because they have to worry about aliasing bugs, data races, and memory leaks (especially via goroutine leaks and global variables).
Fair point, mixing concurrency and mutability is one of the areas where you do have to worry about lifetimes. And memory leaks do happen in GC'd languages. But these are somewhat "special" scenarios: programmers have been admonished over and over not to mix mutable state and concurrency. Mutable global variables are a smell which most people know about. But making arena pointers just regular values adds a whole new area of uncertainty. Any pointer dereference could now theoretically panic.
[A] Go library that today is free of data races, aliasing bugs, and memory leaks, would still be free of those classes of bugs if passed arena-allocated arguments.
Yes, it will still be free of those classes of bugs, but prone to a whole new class: use after free on arena pointer that it doesn't know about.
Yes, it will still be free of those classes of bugs, but prone to a whole new class: use after free on arena pointer that it doesn't know about.
How so? As far as I can see, essentially every use-after-free bug that would occur with arenas has a corresponding (existing) aliasing bug or data race without arenas.
In both cases, the bug is caused by either unexpectedly accessing data retained after a call, or accessing data unexpectedly mutated during a call.
So, we already have this, sort of! It's called mmap
and using the mmapped space as backing store (usually via unsafe). And we do indeed have to do defensive copying of things accessed through such a thing, and it's a pain. That said, it can still be better than not having that.
I am having vague thoughts that this design might want to be changed in some way if we had immutability as a trait we could express in the type system.
@bcmills say you implement a global cache where you store pointers to arenas (nevermind if that's a good idea or not). Is it not possible to accidentaly pass an entry from the cache to a library function after the backing arena has been freed? And would that not result in a panic where it wouldn't have before? Or am I missing the point you are trying to make?
@muscar, a global cache of pointers to arenas has the same failure modes as a global cache of, say, heap-allocated []byte
.
If you do a map lookup like:
func lookup(b []byte) {
v, ok := m[string(b)]
…
}
then your program can panic or even crash if b
is mutated during the map access.
Similarly, if you have a function like this:
type S struct{ b []byte }
func deref(s *S, f func()) int {
if len(s.b) > 0 {
f()
return s.b[0]
}
return 0
}
then your program can panic if f
mutates the struct pointed to by s
.
A race on freeing an arena is conceptually not that different from any other data race.
A use-after-free bug with an arena is conceptually not that different from any other aliasing bug. (It has an especially deep connection to the aliasing bugs that arise with sync.Pool
today!)
Thanks for the examples @bcmills. I see what you mean, and I agree with the point you made. My example was indeed an instance of an aliasing bug.
One of the things that makes me uneasy about this proposal is the potential for this kind of problems to become more pervasive. The examples you provided are well chosen, and they illustrate the current problems well.
Both of them involve []byte
which is different from a pointer, at least from the type system's perspective. A Go programmer who sees []byte
will hopefully be more vigilant. Yes, it's a somewhat weak argument. But at least the language gives you a sort of heads up that things could get tricky.
But making arena pointers indistinguishable from GC'd pointers could get hairy because now you have to be extra careful when you see pointers because you don't know where they come from. That's why marking them, e.g. like C#'s Memory<T>
type, has some benefits. In addition to it being an explicit marker that the programmer is dealing with non GC'd memory, it also collaborates with the GC, e.g. by Pin()
-ing objects.
But making arena pointers indistinguishable from GC'd pointers could get hairy because now you have to be extra careful when you see pointers because you don't know where they come from. That's why marking them, e.g. like C#'s Memory type, has some benefits. In addition to it being an explicit marker that the programmer is dealing with non GC'd memory, it also collaborates with the GC, e.g. by Pin()-ing objects.
I believe that these concerns are unnecessary. One of the key benefits of the arena pattern is how easy it is to avoid the sorts of problems you're describing. Imagine you are the engineer writing the code that creates & frees the arena- what sorts of mistakes would you have to make for another engineer's actions to cause freed memory to be dereferenced? Given that you are in a position to know what data is being allocated from the arena, and also to control the flow of that data, how would such a thing even be possible if you act correctly?
The problem, then, is that the engineer who creates the arena must be in the driver's seat, and that does require arenas to be accepted from callers in most situations. As Merovius observed above- the json/encoding library can't use arenas "under the hood" because the caller has to be the one who creates & frees the arena so that there's no "magic" silently producing transient pointers. (That said, @Merovius I think you're right that you can just make the arena an optional property of the decoder and not even have to pass the arena in for any particular method.)
Imagine you are the engineer writing the code that creates & frees the arena- what sorts of mistakes would you have to make for another engineer's actions to cause freed memory to be dereferenced?
This exercise in itself is interesting. The burden of this exercise will now lie on the shoulders of implementers who use arenas to some extent: "How can the users of my library misues it? What do I have to warn them about? What do I have to document?" And this last point, relying on docs instead of automated checks is key. Computers are good at following rules. Humans not so much. How did documenting ownership and lifetime details work for C or C++?
Given that you are in a position to know what data is being allocated from the arena, and also to control the flow of that data
Am I? If so then that somewhat circumscribes and restricts what can be done with arenas. If you want to use them as an internal implementation detail of a serialisation library or something similar then fine. You have control over a lot of it, and you don't have to even expose that you're using arenas to your library's clients. But that begs the question if such a change is worth the runtime and language support.
how would such a thing even be possible if you act correctly?
That's a pretty big assumption to make, and I'm happy for whatever help compilers can give me to help me act correctly.
The problem, then, is that the engineer who creates the arena must be in the driver's seat, and that does require arenas to be accepted from callers in most situations. As Merovius observed above- the json/encoding library can't use arenas "under the hood" because the caller has to be the one who creates & frees the arena so that there's no "magic" silently producing transient pointers.
Cool. So we come back to the point of arenas being used in a very specific case, as a performance optimisation. If the measurements support it then by all means go for an arena. But again, the changes it would require from the runtime, and the language would open the door to some nastier issues in my view.
This exercise in itself is interesting. The burden of this exercise will now lie on the shoulders of implementers who use arenas to some extent: "How can the users of my library misues it? What do I have to warn them about? What do I have to document?" And this last point, relying on docs instead of automated checks is key. Computers are good at following rules. Humans not so much. How did documenting ownership and lifetime details work for C or C++?
Arenas were developed as a pattern in C & C++ to deal with the exact problem you're describing because they don't have those issues. The correct usage of a memory arenas is ultimately that it's allocated at the start of a method, it's always freed at the end of that same method, and all allocations made between those two points happen in the arena. Arenas are not intended for long-lived objects, or really any objects that escape that method at all.
If so then that somewhat circumscribes and restricts what can be done with arenas.
Yes, absolutely! Arenas are an allocation pattern intended for a strict subset of use cases that, in C, dramatically improves allocation performance & reduces bookkeeping in those use cases. It does increase bookkeeping for us, because we normally use garbage collection, but it also improves performance by even more than usual. This is not intended, I don't think, for every use case. There's a specific hole in go's memory performance story, and it's not exactly arena-shaped, but arenas do fill most of the space, I think.
I don't see/ctrl+f any proposed behavior for the zero Arena value. It seems to me that having a nil Arena always allocate from the heap would make opt-in usage very easy.
This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group
@CannibalVox I know what arenas are. And how they are used in C or C++. What I was getting at (in a very roundabout way) is that they are a very specific pattern only useful in some scenarios. We seem to agree on that point :).
The point I was trying to make is that going straight to adding language level support for a very specific pattern seems a bit rushed. Once you add something to a language it's hard to go back. And you have to write docs for it, make sure newcomers to the language don't misuse it, etc.
That's why a good strategy is to try to implement it as a library. You can always add features to the language later, when you understand the use cases better, if it's warranted.
Also, extending the language in the proposed way would only benefit this one specific case. If you really need language support it makes sense to add smaller, more generic features that other users could use, independently of the one specific use case that prompted them. In this case you could add raw memory handles that the language and GC know about. Allow the memory region to be pinned, and let the GC know that. This would serve as a base for the arena library as well as other scenarios.
What is the cost benefit analysis on making this change to the language? You get a 15%-20% speedup on some very specific scenarios on x86-64 linux at the cost of added complexity for every user of the language irrespective of them using the feature or not.
Hi @danscales
If we have a package that is allocating lots of memory (often by doing decoding of wire format) and we want to use arenas with it, then we will have to plumb in a way to send down an optional allocator function that it should use if enabled (in our case, the allocator function would use an arena).
Is there a rough estimate of how many new APIs this might translate to in the standard library, for example?
No, there is no estimate, since this is just a proposal and its possible use would be completely optional for any packages, of course. I expect there would be lots of consideration and experimentation before deciding that it may be worth adding optional support for arenas in a particular package. (Maybe a little similar to being very cautious and slowly gaining experience before adding adding any generic APIs to existing packages, now that generics is just arriving)
I’m generally in fair of this but have two concerns.
“If this optimization is used, the application is allowed to proceed normally if an object is accessed after the arena containing it is freed, as long as the memory of the object is still available and correct“
I feel like this will only make things worse by hiding genuine errors. Why do we need this? I feel like much better is to immediately panic after free is called regardless of whether arena is really freed or not.
Second is the fact that this library would return regular Go objects. Go is very much about reducing cognitive load and as others already pointed out arena allocations will force developers to always keep in mind that some memory might be arena allocated and they can’t simply store it for later use. Given Go lacks immutability developers already has to think about that all the time and hope documentation covers that. We don’t need to add more cognitive load.
Given that arena allocation is very niche usecase why don’t we also make it very explicit. Let API return special object that returns empty interface and has to be type asserted every time you need to access the underlying object. At least that way we are very explicit about arena allocation and developers will think twice when trying to store underlying object somewhere else.
The point I was trying to make is that going straight to adding language level support for a very specific pattern seems a bit rushed.
That is not the plan. The plan is to add a new package. That's a library-change, not a language change.
What is the cost benefit analysis on making this change to the language? You get a 15%-20% speedup on some very specific scenarios on x86-64 linux
I don't think the scenarios are that specific. Approximately every request-based server could benefit from it without too much intrusiveness. i.e. ~every HTTP or (g)RPC server. That's a large slice of the Go ecosystem.
They are, however, specific enough to IMO justify that the problems aren't that likely. It is relatively easy to document "the Handler is not allowed to retain the request message" and keeping to that - and detecting if you don't via tests.
Yes, that is true. It is possible there is another implementation that could deal with that many arenas simultaneously. But I will note that if you have 100s of requests in flight at one time, then they may not be allocating that much new data while serving each individual request, so arenas may not really be required in that case. Arenas are really more useful/appropriate for unmarshalling large data objects for which you may do a lot of processing. sync.Pool may be more appropriate if the allocated data per request is small (since there may also be only a limited number of known object types).
What message sizes/throughputs was the prototype tested at? Many small requests/responses is certainly the norm in my background, but I know google was looking hard (I believe?) a Vitess blog post that mainly dealt with very large data replication messages. Certainly if proto performance gets worse than status quo at high throughput levels, many people wouldn't be thrilled with that.
As mentioned, the performance gains are expected for dealing with large messages that are being unmarshaled or building large data structures during processing that can all be freed at once. Even if a package (such as protobuf) adds support for an arena allocator, its use is entirely optional. For a program that is dealing with all small requests and message sizes, the program would just not specify an arena allocator when unmarshaling, so the normal heap allocation be used, so there would be no reduction in performance. Arenas would only be used after experimentation to determine if they are noticeably improving performance.
as others already pointed out arena allocations will force developers to always keep in mind that some memory might be arena allocated and they can’t simply store it for later use.
As others have pointed out as well, you already have to keep that in mind. It is very common to have requirements like this. It is simply not true that you can assume that any pointer is safe to store in a global data-structure or otherwise to retain (which is the main concern).
This discussion does make me consider one serious concern: strings.
In the above proposal, downstream code that is not aware of the arena (or may have been written before the existence of arenas!) seems to need to be aware that arenas may exist in order to avoid doing completely normal things with arena-allocated strings.
All of this goes back to my understanding of how arenas should work, which is that the user who allocates the arena is responsible for controlling what downstreams they send them to. The string thing seems like such a huge surprise though that I don't think you'll be able to adequately prepare anybody for that behavior even if the arena author is aware of their repsonsibilities.
It may be that there does need to be some language-level support. An ArenaPointer[T]
type that the compiler considers interchangeable with T- that is, you can call all the same methods and access all the same fields as T, but there is a type-safe check that everyone understands what is going on, and an ArenaString
that is unambiguously different. Methods to convert a pointer to a heap copy and an unsafe
method that will give you the underlying T/string. That seems like a howitzer compared to the proposal, but the string thing really does spook me. That seems like it is going to cause unexpected problems almost constantly.
[I]f you have 100s of requests in flight at one time, then they may not be allocating that much new data while serving each individual request, so arenas may not really be required in that case. Arenas are really more useful/appropriate for unmarshalling large data objects for which you may do a lot of processing.
This seems to restrict the subset of apps that would benefit from this incarnation of arenas somewhat significantly. The added focus on protobufs makes me think that the authors have a very specific set of applications in mind.
Is such an intrusive change worth the 15% speed gain for a subset of apps of unknown size?
It's definitely reasonable to question how useful arenas will be vs how "intrusive". Their use is entirely optional, and the only user-visible change would likely be the appearance of a new arena package with its API (no language change). So, for most users they shouldn't be "intrusive". But it is good to discuss if arenas puts a burden on package/library writers, and/or if there are rules about usage of arenas that can minimize that burden.
What is the cost benefit analysis on making this change to the language? You get a 15%-20% speedup on some very specific scenarios on x86-64 linux at the cost of added complexity for every user of the language irrespective of them using the feature or not.
My argument is that a (properly implemented) arena only increases complexity for those who choose to use it. There's a lot of stdlib features I've never personally interacted with that exist for situational use. I think that there's a strong argument that arenas CANNOT be satisfactorily implemented outside golang/go and it's very much a "put it here or not at all" thing.
@Merovius it’s true that you can’t assume any pointer is safe to store but it is in most cases unless documented otherwise which is quiet rare in my practice. Arena essentially puts a huge burden on library writers. Either they have to painstakingly document every function or make sure any public API returns non arena allocated copies. Without arena allocation that’s relatively easy - you just forget the object and let it go. With arena you have to carefully examine whole execution path to find where the object come from and whether it’s arena allocated. Yes, you can imagine using special naming scheme for arena allocated variables or cases where even letting go of some object is dangerous because some internal structure might still retain it. But it still feels like arena allocation makes it even more complex and dangerous for no good reason. And at least with wrappers there’s some additional barrier. Either that or get rid of Free and let GC worry about when arena is safe to deallocated. But then the question is, will it be performant enough to warrant the complexity.
and/or if there are rules about usage of arenas that can minimize that burden.
I'm wondering if there's an opportunity to provide a (separately packaged?) complementary API that nudges towards the less pathological patterns of concurrency using arenas. Is this something where 80% or 90% of the use cases are going to be solving the same problems?
Either they have to painstakingly document every function or make sure any public API returns non arena allocated copies.
I would describe this as the correct use of arenas to begin with. Arenas should not last beyond the stack they're allocated in, and stuff allocated via the arena should not escape the call graph beneath that function.
Maybe that's the answer? Require arena allocation on the stack, if an arena pointer escapes a method where the arena that allocated it is on the stack, move it to the heap, ban assigning arena pointers to anything on the heap (or auto-heapify it if you try to assign it to something on the heap)
Would this provide as significant an improvement with a generational GC? I'm sure it would still be faster, but would it still be enough to be worth it?
@CannibalVox yes but the point I’m getting at, who’s gonna enforce it? Or is it another case of “assume developers understand all the caveats”. There’s already too many cases of that in Go and this adds another. That’s why I feel like there needs to be either some barrier in convenience or make it smarter so that incorrect usage is simply impossible. Automatically making objects escape sounds like a great idea.
@CannibalVox yes but the point I’m getting at, who’s gonna enforce it? Or is it another case of “assume developers understand all the caveats”. There’s already too many cases of that in Go and this adds another. That’s why I feel like there needs to be either some barrier in convenience or make it smarter so that incorrect usage is simply impossible. Automatically making objects escape sounds like a great idea.
I would like the auto-escape, I feel like the compiler already understands the situations where an arena object need to evaluate whether it needs to move to the heap, so hopefully that's possible.
However, I don't think "don't return objects that you, personally, have just freed on the previous line of code" is that tough of a sell. I'm definitely more concerned with situations where an engineer did not expect arena pointers to be persisted, like basically anything involving strings.
@creker I get the impression you are overestimating how common usages of arenas are going to be. There are a handful of use-cases for them and you are only affected by them, as a library author, if you decide to do so. And even then, their use will be limited to a couple of places.
I think it's instructive to look at a concrete use-case: An RPC server. In that case, the documentation will be "a handler might not retain a pointer to the request message". And the pointer to keep track of (for the framework author) will be that single request message. That's not a huge burden.
Apart from that, yes, as a library user you need to read documentation that says "don't retain such and such pointers", but again, it's fairly common that you have to do that anyways.
FWIW I've been trying to come up with places where I might use arenas. I can't think of any. And that's not for lack of worry about doing lots of small allocations - it's that arenas are just not the right tool for the jobs I have that problem with.
@Merovius that’s only if we assume that developers themselves understand that they should almost never use arenas. But if we look at sync.Pool which is everywhere I don’t think that would hold true. Especially if people are coming from other languages where arenas are very popular like C++. Eventually arenas would get into many packages regardless of actual need for it and Go ecosystem would suffer if arenas are as fragile as proposed. The notion that allocations are a bane of existence of every Go developer is very much popular.
seems like a hassle with little benefit. where is the simplicity? the line between spec and implementation is blurring. Where is the CL?
@creker Your concern was "it places a huge burden on library authors". ISTM that you are now saying that burden is mostly self-inflicted by people using it where they shouldn't. So, maybe that burden is a bug, not a feature of the design.
FWIW that there are a couple APIs that interact with the GC, which are similarly dangerous. runtime.SetFinalizer
, for example. Or the accepted proposal for pointer-pinning. And sync.Map
also tends to be abused by people (but TBF is rarely ever harmful). I don't think we should necessarily let us deter from adding useful APIs, to protect people from themselves.
@danscales In case you missed it, I would like to push my earlier question: Why does the design rely on the intent for users to call Free
? I think if we can rely on finalizers, a lot of the pushback in this issue would be resolved, as most usages can be made fully transparent. That also would make them more generally useful, as the need to call Free
makes them prohibitive for many applications.
@danscales In case you missed it, I would like to push my earlier question: Why does the design rely on the intent for users to call
Free
? I think if we can rely on finalizers, a lot of the pushback in this issue would be resolved, as most usages can be made fully transparent. That also would make them more generally useful, as the need to callFree
makes them prohibitive for many applications.
The design does include a finalizer for arenas, but I don't see how that changes anything: there are plenty of situations where the arena can be GC'd but its allocations are glued to something.
@Merovius See the section in the proposal "Removing arena free" in the "Rationale" section. We find that arenas do not give any improvement in performance on (and can make things worse) if arenas don't have an explicit arena.Free
and instead are only freed when the garbage collectors notices that there are no more pointers to the arena or the objects in the arena. One of the big wins of arenas (where they can be applied) is that the memory can be almost immediately reused at the point of the arena free, rather than waiting for a complete GC cycle. And, if most memory is allocated and freed via arenas, you may actually decrease how often the garbage collector needs to be called.
The conversation seems to be going back and forth between:
- "Arenas aren't generally useful and puts too much burden on the programmer to ensure no heap objects point into an arena allocation."
- "Arenas are really useful for specific use cases, and in those use cases there's no need to have heap objects point into an arena allocation, therefore there's no problem."
Maybe there's a compromise here where the API is designed in such a way as to indicate that it's a temporary workspace. Some languages have a with
or with-x-y-z
pattern that takes a block, or lambda and cleans up the managed thing afterwards (transaction, file, etc):
var (
result Something
err error
)
arena.With(func(a arena.Page) {
var ptrT *T
a.New(&ptrT)
... go about your business
})
Yes, it allocates a closure. But, we're already used to the semantics of calling functions and having the stack space be cleaned up. That's essentially what we're trying to achieve with arenas. Perhaps unfortunately, the fact that Go does escape analysis for stack allocations won't work the same for arena allocations, and there will certainly be bugs associated with it. But maybe that's lintable?
The conversation seems to be going back and forth between:
- "Arenas aren't generally useful and puts too much burden on the programmer to ensure no heap objects point into an arena allocation."
- "Arenas are really useful for specific use cases, and in those use cases there's no need to have heap objects point into an arena allocation, therefore there's no problem."
Maybe there's a compromise here where the API is designed in such a way as to indicate that it's a temporary workspace. Some languages have a
with
orwith-x-y-z
pattern that takes a block, or lambda and cleans up the managed thing afterwards (transaction, file, etc):var ( result Something err error ) arena.With(func(a arena.Page) { var ptrT *T a.New(&ptrT) ... go about your business })
Yes, it allocates a closure. But, we're already used to the semantics of calling functions and having the stack space be cleaned up. That's essentially what we're trying to achieve with arenas. Perhaps unfortunately, the fact that Go does escape analysis for stack allocations won't work the same for arena allocations, and there will certainly be bugs associated with it. But maybe that's lintable?
Two things:
- Do we actually, really, truly, foresee engineers not being able to wrap their minds around what an arena is and what it does? I understand that you personally are not taking a position on this question in your comment here, but a lot of the discussion around the difficulty of using arenas seems to focus on the idea that somebody might just not really understand that when they call arena.Free(), the memory they have allocated by the arena... will be freed. That seems completely incredible to me.
- This idea does not actually prevent anyone from escaping arena-allocated data, nor does it make it especially harder to do so by accident. I assume that in any case where there was a problem, there's still a problem.
Just to clarify, @bcmills said
Arena-allocated string values, which are functionally more like slices: they are no longer safe to read after a call returns, and not guaranteed to have stable contents over time.
They are guaranteed to have stable contents over time. They either have their originally allocated contents, or they crash the program when accessed. In no situation would they successfully look like a different string than their original contents.
Similarly, arenas are better than the sync.Pool
allocation scheme that some have proposed as an alternative. If you use a sync.Pool
to do allocation, you run the risk of use-after-free bugs ("free" = return to pool) that are undetectable. With arenas, a use-after-free either still gives you the correct contents, or crashes the program. In no case will you mistakenly read (or be otherwise aliased to) the contents of some unrelated allocation as you would with a sync.Pool
implementation.
Arguably,
Free
should zero the underlying memory to ensure a crash as early as possible…
The proposal uses unmapping as the mechanism to ensure crashes. Because unmapping is somewhat expensive, the proposal batches them up to lower overhead, but that of course leaves a window where an object is freed but still accessible. Perhaps there should be a debug mode where unmapping is immediate. (Maybe when running with -race
? That's kind of the grab-bag that we throw expensive debugging things into, but it might be appropriate here.)
I don't think just zeroing is a good idea. It may cause quick crashes on pointer data, but may just cause incorrect computation (especially on nonpointer data, but potentially on pointer data as well).
... a lot of the discussion around the difficulty of using arenas seems to focus on the idea that somebody might just not really understand that when they call arena.Free(), the memory they have allocated by the arena... will be freed. That seems completely incredible to me.
Speaking as someone who spends a fair amount of time teaching new Go programmers, I don't think that's particularly incredible at all. Ideally, nothing in Go makes you think about memory being freed. That's a C concept. In Go, Java, Python, &c. it only applies in edge cases, and then only as an implementation detail. Hence, my reservations about this proposal come from the pedagogical perspective.
[Arena-allocated strings] are guaranteed to have stable contents over time. They either have their originally allocated contents, or they crash the program when accessed. In no situation would they successfully look like a different string than their original contents.
That's better than it could be, but I still wouldn't describe a thing that decays into a poison-pill that crashes your program as “stable contents”. 😅
Also, it is not intended that arenas be shared across goroutines. Each arena has a lock to protect against simultaneous allocations by multiple goroutines, but it would be very inefficient to actually use the same arena for multiple goroutines. Of course, that would rarely make sense anyway, since the lifetimes of objects allocated in different goroutines are likely to be quite different.
We can just add heap-less functions that allocate all objects on stacks and panic on heap-escaping since arenas are bounded to goroutines.
@danscales Darn, don't know how I overlooked that. That makes me feel a lot worse about this proposal, as it really requires exposing arenas in the API to be useful. I feel like performance optimizations like this should be implementation details of packages. But okay, that's that, then.
Also, it is not intended that arenas be shared across goroutines. Each arena has a lock to protect against simultaneous allocations by multiple goroutines, but it would be very inefficient to actually use the same arena for multiple goroutines. Of course, that would rarely make sense anyway, since the lifetimes of objects allocated in different goroutines are likely to be quite different.
We can just add heap-less functions that allocate all objects on stacks and panic on heap-escaping since arenas are bounded to goroutines.
Arena allocated objects need to be able to be returned back to the arena root without issue which is escape from the method that allocated it (and anyway, prevents it from living on the stack)