pypersist is a persistent memoisation framework for Python 3. Persistent memoisation is the practice of storing the output of a function permanently to a disk or a server so that the result can be looked up automatically in the future, avoiding any known results being recomputed unnecessarily.
pypersist is available from PyPI, and the latest release can be installed using, for example:
pip3 install --user pypersist
Alternatively, the latest development version can be installed using Github:
git clone https://github.com/mtorpey/pypersist.git
pip3 install --user ./pypersist
To use, import the persist class from the pypersist package:
from pypersist import persistand use it as a decorator when writing a function:
@persist
def double(x):
return x * 2
print(double(3))
print(double(6.5))This will store the outputs of the double function in a directory called
persist/double/, in a machine-readable format.
One can specify various arguments to persist. For example:
@persist(key=lambda x,y: (x,y),
hash=lambda k: '%s_to_the_power_of_%s' % k,
pickle=str,
unpickle=int)
def power(x, y):
return x ** y
print(power(2,4))
print(power(10,5))will store the outputs of power in human-readable files with descriptive
filenames.
Many more options are available. See the persist class documentation for a
full description, or launch the included notebook on
Binder
for more examples.
See this HackMD and the Issue tracker for current plans.
Please cite this package as:
[Tor20] M. Torpey, pypersist, Python memoisation framework, Version X.Y (20XX), https://github.com/mtorpey/pypersist.
pypersist was created as part of the OpenDreamKit project: https://opendreamkit.org/
This part of the project is summarised in this report.
|
This infrastructure is part of a project that has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 676541. |