man-group/ArcticDB

Library cache can exhaust file descriptors with LMDB

poodlewars opened this issue · 1 comments

Describe the bug

No response

Steps/Code to Reproduce

  • Create an Arctic instance with LMDB and hold it in memory
  • Create many libraries and call get_library on them. Let the library object fall out of scope.
  • On Windows, eventually get,
OSError: [Errno 24] Too many open files

A similar thing would probably happen on Linux.

More details:

https://chat-man.slack.com/archives/C02NEG3V38X/p1728482885941529

We should probably just remove the library cache?

Expected Results

  • All libraries accessed, no errors

OS, Python Version and ArcticDB Version

Backend storage used

LMDB

Additional Context

No response

Dupe of #1904