fails with >23100 tips
Opened this issue · 2 comments
The embedding codes fails when using a tree with > ~23100 tips. I believe this is due to the C code in ape, but still, this is then a limitation of HyPhyTree.
Thank you for your comment.
I tried it with 23,000 tips of simulated data and was able to do the calculation on my computer. However, it used about 50GB of memory. The current data structure and calculation method require large computational resources.
Even if the distance matrix can be calculated, the subsequent embedding into the hyperbolic space requires about O(N^2.3) of computation (https://arxiv.org/abs/1903.08977), so the computation time is extensive, and I think further innovations are needed.
I'm sorry, I didn't consider the application to such a huge phylogenetic tree, and so far, I haven't come up with a radical way to deal with such data.
Thanks for the feedback! For now, I've just trimmed the tree to the most prevalent taxa