FlandreDaisuki/dmhy-subscribe

Use WebTorrent instead of deluge?

maple3142 opened this issue · 21 comments

FlandreDaisuki commented

This is good for making less complicated dependencies to installation but how can I monitor the downloading queue. I've tried webtorrent $magnet --vlc but it's too slow and webtorrent $magnet but it can't close terminal session.

The same issue happens if use aria2 (node wrapping). That's why I use deluged and deluge-console. If any node module can do both functions, I would happy to change deluge by that.

I also have a plan to write a server-client that server is daemon (any magnet client) and client is cli (or webUI or both).

FlandreDaisuki commented

or open another option API this like:
$ dmhy --webtorrent -d[--dest] $HOME?

maple3142 commented

I think use a command template is better. It is easy to integrate with other downloader.
Example(may be need to use exec rather that spawn):

deluge-console add $link$ #default
webtorrent $link$ -o anime/$title$
# And others...

And use detached option can let main process close without closing child process.

const {spawn}=require('child_process')
const p=spawn('webtorrent.cmd',process.argv.slice(2),{
	detached: true,
	stdio: 'ignore'
})
p.unref()
process.exit()
  • 👍1
FlandreDaisuki commented

I prefer $$title keep the same to magnet setting. It can be another option also but miner priority.

I'll write the option API --webtorrent to call child process. The default destination can use downloads-folder but optional with -d/--dest.
The most annoying thing in this change is README. 😂😂

  • 👍1
FlandreDaisuki commented

cc @maple3142

I tested webtorrent and some wired bugs happened.
_001

If a magnet stuck, that detached process will wait forever and no one can handle until user kill it. In the worst case, the stuck magnets will spawn A LOT of process (especially use cron or debugging) and no one manage them. But deluged can handle those not finished magnets even restart deluged.

I'll try to support aria2 as a similar tool instead of webtorrent later.

maple3142 commented

@FlandreDaisuki
I think create a wrapper script which pipe stdin,stdout between `webtorrent' command, and add some protection.
Then spawn it from main process?(detached)

FlandreDaisuki commented

Update: It works after add shell:true in spawnOpt

I'm not familiar on child_process, and here is a workaround and some bug
code

After I call $ dmhy --client aria2c -d /tmp dl LHK-05, then ps aux | grep aria2c instantly, but I can't find running process. I also print the command which pass to spawn and work if execute in terminal directly.

maple3142 commented

It is because spawn(command: string,args: string[],options: object).
In this case, I think exec(command: string,options: object) is better.

Reference: Child Process

  • 👍1
  • ❤️1
FlandreDaisuki commented

commit/1220ce44 is a alpha version of support aria2c. (not from npm)
Let's try for a while. 😂

FlandreDaisuki commented

目前有幾種 option 但我不確定要全兜在一起可不可行：

  1. deluged(server) + deluge-console(RPC) + deluge(UI) 這最沒問題
  2. aria2c(server) + aria2crpc(RPC) + webui-aria2(UI) 架構會跟上面一樣，依賴也差不多麻煩
  3. shell call 系列
    1. webtorrent: 卡住就無解了，無法設定做種時間 (要手動 kill process)
    2. aria2c: 有 timeout，有 seed-time 來讓 process 自動結束但卡住還是無解
maple3142 commented

aria2c可以算shell call
aria2crpc也不難，不過需要用額外的option
webui之類的不用處理吧，它們就我的理解不都是使用jsonrpc去取得aria2c服務的資訊嗎？
webtorrent直接在downloader/webtorrent.js裡用require('webtorrent')去用就好了吧

FlandreDaisuki commented

最困難的就是這個 webtorrent 不會自己結束的問題 🙈

要是走射後不理的 detached 路線，卡住要手動 kill (之前說過如果搭配 cron 排程有可能開好幾十甚至上百個 process)
要是走跟著 parent process 路線(你 fork 改的那種)，手動下指令要是卡住還可以 ^c 來終止，要是自動排程同理會生出一大堆無法結束的 process，此外有多個 threads(帖) 的情況也會塞車

所以我還是偏好完全放棄 shell call 系列，去支援上面兩個
因為如果用 aria2c 的 shell call, 就必須下 --seed-time 參數，這也決定了 process 存活時間 (也跟排程頻率有關)，直接下 --seed-time=0 技術上沒問題但破壞整個 BT 生態系 🤔

maple3142 commented

不會自己結束的是 webtorrent-cli
webtorrent 在 client.on('close',cb) 時結束process就好

而aria2c的 --seed-time 我覺得直接做成選項就好吧，讓使用者自己覺得

FlandreDaisuki commented

webtorrent 在 client.on('close',cb) 時結束 process 就好

我是在Docs有看到 client.destroy(callback)
但你要在什麼條件下執行他？假設 client.torrents[] 中就是有一個只有 10% 而且 no peers
如果是平行下載你要在什麼條件下結束他？
如果是順序下載那個 10% 就是卡在第一個下載不下來後面全卡住？
這樣是不是連 main process 也要跟著設 timeout... 🤔

aria2c 的 RPC 版本我可能會想辦法設計的好看一點，包含 timeout 那些設定

maple3142 commented

架構不是對於每個下載就execFile一次嗎，因此會有多個downloader/webtorrent.js的process
而在裡面每個都有一個client，每個client只會下載一個torrent

而下載queue的控制應該放在Database裡面
不過這又會面臨到main process不能exit
aria2c也有這問題

FlandreDaisuki commented

execFile 我沒用過我不確定這樣的想法(main#execFile → downloader/*.js)會不會有什麼雷就是了ww
不過 execFile 有 timeout 可以設應該是還行，看來全域 timeout 不可避了 🙈

有空再來改好了... (不知道為什麼沒有人寫 JS 的 torrent 帶 UI + API 的管理器... 💀)

maple3142 commented

不需要在db.download放timeout吧，都由downloader自己控制，db.download只負責呼叫downloader而已

https://github.com/webtorrent/webtorrent-desktop

FlandreDaisuki commented

https://github.com/webtorrent/webtorrent-desktop

他不是管理器，他原本的目的也和一般的 torrent client 不一樣
所以用 webtorrent 來載 BT 感覺真的很奇怪...

maple3142 commented

webtorrent和aria2c(非RPC模式)都是用來做單次下載用的，不像有個後台服務

FlandreDaisuki commented

我覺得最終解還是另外寫一個 dmhy-subscribe-server 把整個 aria2 包進去
然後這個 cli 就是送 RPC request 給 server
server 部份還可以架在 pm2 上

我當初不是想做單次下載使用的，是給排程檢查用的，檢查完就關掉，所以才這樣設計