Optimize the memory usage with `__slots__`
ArtyomVancyan opened this issue · 2 comments
There is still a place to optimize memory usage. Currently, on processing a 63MB video, it is about 0.5GB of memory used. In the scope of the issue, profile the application with large videos to find the core bottlenecks.
mprof run thumbnails -I 6 input.mp4
mprof plot -o memory_usage.png
input.mp4 was 3840x2160 12.5Mbps, about 6:56 minutes, 670mb filesize.
output png is 46mb.
single frame png was between 6.5 and 9mb.
When I created this issue, I thought the __slots__ would make a difference in memory usage. Also, the __slots__ attribute is efficient while having a very huge number of instances. So this does not make sense, and I am skipping this issue by closing it.
While working on the memory optimization and finding the bottlenecks, the profiling graphs showed that the main memory is used by ThreadPoolExecutor when allocating an unknown count of threads at once.
Keeping the context, I am moving the issue to: