Implement distributed version of streaming parallel EM-tree
Opened this issue · 0 comments
cmdevries commented
Compress transmission of integer accumulators between machines vectors using https://github.com/lemire/FastPFOR.
Hadoop + HDFS (just get hadoop to hand over the bytes, or use HDFS directly).
ZeroMQ + GlusterFS.
Apache Spark might work well with python bindings for library, https://github.com/apache/spark.
HDFS + Erlang scheduler (gascheduler) + C++ code as a simple TCP server.