[refactor] GosearchCrawler should save subpackages into module field and not new Product.
Closed this issue · 1 comments
timgluz commented
Lessons learned from CpanCrawler
:
If product has many submodules, then it seriously slows down saving data and the same data is duplicated.
TODO:
Save each subpackage as module in the Product model and also in version model as users may remove submodules between each release.
timgluz commented
Actually, it doesnt work for Golang, because each submodule may include own set of dependencies;
For example, the etcd
includes 2 subpackages client
and auth
. Gosearch api exposes very different set of dependencies for each of them:
- http://go-search.org/view?id=github.com%2fcoreos%2fetcd%2fclient
- http://go-search.org/view?id=github.com%2fcoreos%2fetcd/auth
So, i will close this tasks;