Can rqbit replace transmission-daemon service?

[yuriry]

I am testing my bittorrent tracker implementation, also written in Rust (not released yet), and the scenario I'm using my tracker is the following:

• My tracker is running on localhost:8000
• transmission-daemon is running on a server on a local network
• I upload the myfile.mp4 to the folder that the transmission-daemon is using to store files
• I create a private torrent using transmission-create -t http://localhost:8000/announce -o myfile.mp4.torrent myfile.mp4
• I add myfile.mp4.torrent to transmission-daemon through its web UI
• I open Transmission app on my localhost and add myfile.mp4.torrent to it
• Transmission app starts downloading myfile.mp4 from the server where transmission-daemon is running to localhost. The connection between Transmission app and transmission-daemon is made through my tracker running on the localhost

When I replace transmission-daemon with rqbit, the download does not happen.

Before debugging rqbit, I wanted to ask if this scenario is theoretically possible with rqbit?

Thank you in advance!

[ikatson]

Hey, if you mean that you add a torrent with a localhost tracker it should work, at least in principle

[yuriry]

I have a test in my tracker for peer_id length from this spec: http://bittorrent.org/beps/bep_0003.html
It is failing on peer_id length

[ikatson]

Try running "make devserver", it writes /tmp/rqbit-log and look at the tracker_comms messages

[ikatson]

Ah well then looks like you figured it out from the server side.

Not sure what's wrong with peer ID, maybe it's buggy, but didn't have issues like this before

[yuriry]

I disabled peer_id length validation in my tracker, now I see in the logs that the tracker returns rqbit as a peer to Transmission app, but Transmission app does not display it as a peer in the Properties window. May be it also looks at peer_id. Will start devserver to understand what is wrong with peer_id length

[yuriry]

The difference is Transmission app sends something like this as peer_id -TR3000-brwn1ez7kwnr and rqbit sends -rQ0001-%9F%05E%86%9B%87%8A%23Q%DC%E6%A7. I probably need to look at some decoding on the tracker side...

[ikatson]

Rqbit generates random bytes of length 20, and because it's not ASCII it just gets urlencoded.
Looking quickly at bep 0003 it doesn't seem to be required for it to be ASCII.

You just need to urldecode it on the tracker, usually your http framework does it for you, as all query parameters are urlencoded

[ikatson]

Alternatively, if other clients do lowercase ASCII I would be happy to merge a PR that makes peer IDs lowercase ASCII

But in your tracker implementation it would still be beneficial to handle query parameters as urlencoded always

[yuriry]

Other clients (Transmission) doing lowercase ASCII does not mean my tracker processes peer_id correctly :(

When I urlencode peer_id from rqbit, I'm getting 20-byte vector, but trying to convert that vector to utf8 string fails because Uuid::new_v4() generates random bytes.

bep 0003 says peer_id should be string of length 20, but does not say it should be ASCII or even UTF, they only say that the value almost certainly have to be escaped. They also say all strings in a .torrent file that contains text must be UTF-8 encoded, but nothing about peer_id :(

I'll try modifying generate_peer_id() in rqbit tomorrow to see if my tracker with url-encoded peer_id can make Transmission client work.

[ikatson]

Ok, I just looked it up from real peer ids.

• transmission and qbittorrent create valid utf-8 peer_ids
• utorrent and BitTorrent don't do utf-8 - it's binary

So it looks like utf-8 happens only sometimes, so I won't be ok to merge the change to generate_peer_id in absence of other reasons.

The crux of the problem is your tracker improperly handling URL query parameters. You have this map: HashMap<String, Option<String>> in your function arguments. This map's values should all be urldecoded, and optionally utf-8 decoded (if you need strings, not bytes) before use.

It just so happens to be that you are lucky that Transmission encodes it into ASCII that doesn't require percent-encoding (urlencoding), but if you try uTorrent, it'll also fail.

Do you have a hand-rolled HTTP framework or are you manually parsing the tracker's HTTP GET URL into this map?

[ikatson]

Btw, to address your initial question about replacing transmission-daemon:

it looks like you are creating a new torrent that did not exist before and start seeding it with the client. Although it could work, there are a few limitations of using rqbit for this:

1. it doesn't support torrent file creation. There's a function for this, but it's not exposed. So you'll need to keep doing "transmission-create" or smth like that.
2. It doesn't support uTP protocol. While in practice it's not a big deal, this would be if you're the first peer and can't expose rqbit ports to the internet. Usually rqbit tries to expose ports to the internet via upnp, but there's no guarantee it would work. If this fails, peers won't be able to connect to you at all. Usually it's ok if you're not a blocker if there are some other seeds, but if you're the only one it'll be a problem.

[yuriry]

1. it doesn't support torrent file creation. There's a function for this, but it's not exposed. So you'll need to keep doing "transmission-create" or smth like that.

This is OK, I was planing creating new torrents with transmission-create or similar tool for now.

2. It doesn't support uTP protocol. While in practice it's not a big deal, this would be if you're the first peer and can't expose rqbit ports to the internet. Usually rqbit tries to expose ports to the internet via upnp, but there's no guarantee it would work. If this fails, peers won't be able to connect to you at all. Usually it's ok if you're not a blocker if there are some other seeds, but if you're the only one it'll be a problem.

No support for uTP protocol is OK for now, I configured port forwarding to make sure my first peer and the tracker are exposed to the internet. The tracker has also an address map that maps private address/port to internet address/port when it responds to announce/scrape requests from outside clients.

It looks like the main problem is to fix peer_id handling for binary IDs in the tracker.

Regarding HTTP framework: I currently use httparse and urlencoding.

Thank you for all your comments and help!

[ikatson]

Looks like httparse returns just path which includes query string after "?".

If you take that, split on &, then on = to make pairs, then call urlencoding::decode_binary on the values it should to the job