Framework for fault-tolerant cooperative distributed processing.
HighFive is a simple distributed processing framework designed so you and your friends can donate CPU time to each other to run large, interesting computations. It's easy to spin up a HighFive master and connect as many remote workers as you want to it!
Don't want to lock down all your computers for hours while you wait for results? Since HighFive was designed for cooperative work, the workers can disconnect at any time without harming the main process! Then, they can reconnect to the same process later and get back to work!
HighFive is written in Python 3, and uses
asyncio for managing
connections. Because it uses
async/await syntax, it is
currently only supported by Python 3.5 and up.
HighFive is available on PyPI!
Simply run pip install highfive to install the latest version.
You can also install HighFive manually:
- clone the repository
cdinto the root directory- (optional) create a virtual environment
- run
pip install .
You can see a simple example of HighFive in the example/ directory of this
project, and run it for yourself! After installation, simply run the master
with python3 example/sum_master.py, then join the worker pool with python3 example/sum_worker.py.
More thorough documentation is coming soon!