JuliaData/MemPool.jl

Management of files

DrChainsaw opened this issue · 1 comments

Maybe first question to ask is if this package is intended for people like me who only wish to use the disk as an extension of the RAM?

Anyways, it seems like the files created needs to be manually deleted (e.g. by calling pooldelete) when no longer needed, or?

If I want files to be deleted when the DRef is no longer used, is it correct to wrap the DRef in a Ref and register a finalizer for that Ref (while ofc making sure all other parts of the program only interact with the referenced data through some struct wrapping the Ref)?

It seems to work in casual testing but I'm afraid there might be hidden pitfalls with it when let loose in the wild.

Automatic swap-to-disk (and deletion upon DRef being finalized) was implemented in #60. Please see the environment variables in

MemPool.jl/src/MemPool.jl

Lines 118 to 129 in 49bb60f

if parse(Bool, get(ENV, "JULIA_MEMPOOL_EXPERIMENTAL_FANCY_ALLOCATOR", "0"))
membound = parse(Int, get(ENV, "JULIA_MEMPOOL_EXPERIMENTAL_MEMORY_BOUND", repr(8*(1024^3))))
diskpath = get(ENV, "JULIA_MEMPOOL_EXPERIMENTAL_DISK_CACHE", joinpath(default_dir(), randstring(6)))
diskdevice = SerializationFileDevice(FilesystemResource(), diskpath)
diskbound = parse(Int, get(ENV, "JULIA_MEMPOOL_EXPERIMENTAL_DISK_BOUND", repr(32*(1024^3))))
kind = get(ENV, "JULIA_MEMPOOL_EXPERIMENTAL_ALLOCATOR_KIND", "MRU")
if !(kind in ("LRU", "MRU"))
@warn "Unknown allocator kind: $kind\nDefaulting to MRU"
kind = "MRU"
end
GLOBAL_DEVICE[] = SimpleRecencyAllocator(membound, diskdevice, diskbound, Symbol(kind))
end
to get an idea of how to use it (documentation is forthcoming).