After Jackson serializes a large number of objects, the metadata space of the JVM cannot be freed
dylan-tao opened this issue · 11 comments
Version： 2.13.5
Description： After Jackson serializes a large number of objects, when the Classloader unloads the loaded class, Jackson refers to the serialized object and cannot free the JVM's metadata space
Expectation： When the classloader unloads a loaded class, it can explicitly manually release or unload the serialized class, allowing the JVM to automatically reclaim and release it when the meta space is full
ObjectMapper will cache serializers and deserializers, which will have to depend on classes being handled (via Reflection-accessed Methods, Fields etc).
There is no way around that; (de)serializer caching/reuse is an absolute must for performance reasons.
But as long as you drop reference to ObjectMapper (and possible ObjectReaders, ObjectWriters), those references would go away.
Perhaps you can use Soft- (or was it Weak-) Reference for mapper if this is important thing to do.
@cowtowncoder @dylan-tao TypeFactory has a singleton instance that may hold onto Class references.
Its cache can be cleared explicitly using:
com.fasterxml.jackson.databind.type.TypeFactory.defaultInstance().clearCache();
Good point, @pjfanning. It's an unfortunate thing, global stateful (wrt caching) singleton :-(
@cowtowncoder @dylan-tao TypeFactory has a singleton instance that may hold onto Class references.
Its cache can be cleared explicitly using:
com.fasterxml.jackson.databind.type.TypeFactory.defaultInstance().clearCache();
Thanks, I'll test it and wait for my good news.
Good point, @pjfanning. It's an unfortunate thing, global stateful (wrt caching) singleton :-(
@cowtowncoder @dylan-tao TypeFactory has a singleton instance that may hold onto Class references.
Its cache can be cleared explicitly using:
com.fasterxml.jackson.databind.type.TypeFactory.defaultInstance().clearCache();
Unfortunately, it still cannot be released. I found that the jackson cache in the following figure has not been released. Can you provide the cache release method in the information?
Good point, @pjfanning. It's an unfortunate thing, global stateful (wrt caching) singleton :-(
@cowtowncoder @dylan-tao TypeFactory has a singleton instance that may hold onto Class references.
Its cache can be cleared explicitly using:com.fasterxml.jackson.databind.type.TypeFactory.defaultInstance().clearCache();Unfortunately, it still cannot be released. I found that the jackson cache in the following figure has not been released. Can you provide the cache release method in the information?
If you want to view it in real time, I provide the log of the stack, and import and analyze the following files through Eclipse Memory Analyzer to see the relevant jackson reference information in the figure above.
Heap File：https://drive.google.com/file/d/1ncTJuMLZRncyGXSqhlbU-h6wM8Sy1-sB/view?usp=sharing
Eclipse Memory Analyzer：https://eclipse.dev/mat/downloads.php
Would you not consider using microservices? If having the overhead of a couple of classes in memory is a really big worry, I think you really need to be reconsidering your architecture.
The ReadOnlyClassToSerializerMap instance lifecycle should closely match the SerializerCache lifecycle and the SerializerCache should be GCable after you dereference the ObjectMapper instance that it relates to.
Yes, the remaining retention is wrt Serializer caching & makes sense since there are typically many more serialized value types (as it's the exact runtime type vs statically declared for deserialization).
And to drop those caches it is necessary to drop ObjectMapper (and ObjectWriters created from one, if any, but they are usually one-offs being light-weight).
@pjfanning @cowtowncoder yeah solved, solution read and write locks control the lifecycle of an ObjectMapper instance, class descriptions and instance references will be recycled, thanks！
Created FasterXML/jackson-databind#4659 to address this for Jackson 3.0 -- TypeFactory life-cycle should be bound to individual ObjectMapper to avoid having to trigger clearCacheson global defaultTypeFactory` instance.