[enhancement] Tool to import torrent data from external sources
jbruchon opened this issue · 6 comments
It would be helpful to have a tool that could add/update torrent data to the database from external sources such as a list of magnet links, infohashes, .torrent files, or a copy of someone else's magnetico DB. This would allow external tools that can assemble such resources (e.g. a web crawler, a DB dump from a torrent hosting site, or a pile of .torrent files a user has collected) to be used to collect torrent data.
I agree. I think having a clearly defined export format would be great too. I'll work on it. =)
The only issue perhaps is that magnetico is still evolving fast so it'd be hard to guarantee a stable export format. If that's well-understood, I think there is still some huge benefits in having the features you have mentioned.
Also #187.
I'm primarily (almost exclusively) interested in importing data, not necessarily exporting it (a CSV dumped from the DB would probably be fine, really). I have access to multiple internet connections and I can siphon data from those separately, but I can't copy the DB from one and inject its data into the other which is the main thing I'd like to do. I can probably do it with SQLite tools but it'd be better to have something that can merge the data in a more automated way.
I think that torrent import is pretty easy to implement with current persistence layer. Not sure about export but probably it's easy too even with current methods (like search with pagination).
At the risk of repeating myself to the point of annoyance, each magneticod instance having the option of distributing its DB across each other instance via a secondary DHT seems like a one-stop-shop solution to pooling the efforts of multiple instances...
I think that it would be good to use the JSON Lines format which is already supported by magneticod with one exception: we should also have discoveredOn field there.