Rutracker parser
This scripts creates local copy of rutracker and allow to view/search throught this copy.
Parser
parser.py - creates local copy of all torrents at rutracker.org.
Its saves line for each torrent in table.txt with (separator - /t):
- id
- title
- size (in bytes)
- seeds
- peers
- hash
- downloads
- date
- category
Description for each torrent saves to ./descr/012/0123456 where id = 0123456
For using script needs files login.txt with username/passwords and proxy.txt with proxies.
Example of login.txt:
username1 password1
username2 password2
Example of proxy.txt:
127.0.0.1 8080
127.0.0.1 8081
Example of use
python3 ./loader.py --ids 0000001 5160000 --threads 200 --qsize 25 --resume
Args
--ids 0000001 0001000 - download specified range of ids
--ids_file file_with_ids.txt - download ids from specified file
--ids_ignore old_finish.txt - exclude ids not existed in specified file (as example, skip doesn't existed ids from previous crawling)
--random - download in random order
--threads 100 - count of threads for downloading
--proxy_file proxy.txt - specified file with socks5 proxies (default - proxy.txt)
--login_file login.txt - specified file with logins and passwords (default - login.txt)
--resume - resuming previous crawling (skipping downloaded ids from finished.txt)
--print - with 'resume' closes program after showing finished/left counters
--folder descriptions - specifying dir for descriptions of (default - descr)
--qsize 20 - max queue for downloading (default - 30)
Converting
pack.sh - pack descriptions for viewer
sort.py - sort and pack descriptions for viewer
Viewer
viewer.py - allow search throught local copy.
For work needs:
- table_sorted.tar.bz2 with table_sorted.txt
- descr with dirs 000, 001, 002, ... which contains:
- 00000.tar.bz2, 00001.tar.bz2, ..., 00099.tar.bz2 for 000
- 00100.tar.bz2, 00101.tar.bz2, ..., 00199.tar.bz2 for 001
- ...
Screenshot
In search:
- word for include word
- -word for exclude word
- limit:5 for set limit of search results (by default - 20)
Double click on hash to copy magnet-link into clipboard.
Search is running by seeds count. (if want change - resort table_sorted.txt in table_sorted.tar.bz2 as you want).