DBCDK/morph

Deployment is slow

Mic92 opened this issue · 2 comments

Mic92 commented

Evaluation is done locally and without threads: https://github.com/nix-community/nix-eval-jobs could potentially fix this as it does evaluation in threads.
Deployment is also done in serial rather than parallel: I guess using go routines to do deployment in parallel?

I can out-perform morph with a 30 line python function single-handed: https://github.com/Mic92/doctor-cluster-config/blob/master/fabfile.py

  • It does not require downloading packages to my local machine
  • Each machine can evaluate and contribute its cores
  • It only has the disadvantage that it requires nix code to be in a git repository -> rsync might be a better choice

What if we use the nodes fixed point? It seems we would have quadratic computation then.

Mic92 commented

https://github.com/zhaofengli/colmena#colmena supports parallel evaluation using nix-eval-jobs.