Broadcasting transactions with Electrum and EPS over Tor
3ntranced opened this issue · 27 comments
Hi, can this please be explained to me. I can just broadcast transaction with electrum when connected to eps? The transaction then goes into my bitcoind node? And then, how do I ensure it mixes with other transactions that are broadcast by my node?
Or basically how do I broadcast a transaction without revealing my ip?
EPS just uses sendrawtransaction which just does a regular broadcast from your node. It has OK privacy properties, but your IP could still be found by those transaction tracking companies that try connect to every single node to track where transactions came from.
Adding the option for broadcasting transactions via Tor in the works. It's been in the plan since the beginning.
Hopefully my PR would be added to Bitcoin Core soon bitcoin/bitcoin#13152 so that Electrum Personal Server could broadcast transactions without relying on centralized DNS seeds. More reviewers would be helpful.
Right now, for private transaction broadcasting I would recommend using Tor Browser and a blockchain explorer website that allows broadcasting transactions. For example here or here. When sending a payment in Electrum, don't click Broadcast but instead click Copy which copies the transaction hex string to your clipboard from which you can paste into the web form for broadcasting over Tor.
Understood. Thank you
Sorry I keep closing. You prob want to keep this open since you are working on it.
Hi
There is something I do not understand.
EPS just uses sendrawtransaction which just does a regular broadcast from my node. But since my node it is only connected over Tor (and electrum only to my localhost node). Is not already broadcasting throught Tor?
Thank you
In the case of the bitcoin node connecting to the internet only over Tor, then using sendrawtransaction will never reveal your IP addresses because the transaction will only be broadcast over Tor connections. So yes that will work.
That has the downside that when synchronizing you need to download the entire blockchain over tor, which would be very slow. Downloading the blocks over Tor doesn't improve privacy because every full node will download all the blocks anyway. However it would help in a situation where you don't want your ISP or VPN provider to even know that you're using bitcoin (but they'd still know you were using tor).
The attractive thing about broadcasting over Tor but downloading over clearnet is that you can use the fast clearnet connections without costing too much privacy, and still have privacy when you broadcast.
Thanks for your explanation chris-belcher all very clear. That will be a great improvement.
My GUI fixes this by broadcasting with an instance of electrum with TOR proxy. Want a PR, guys? (not easily compatible with windows, however. I'm working on porting my script to windows.)
Hi
There is something I do not understand.
EPS just uses sendrawtransaction which just does a regular broadcast from my node. But since my node it is only connected over Tor (and electrum only to my localhost node). Is not already broadcasting throught Tor?Thank you
yes. timing correlation is still an issue, but if you only connect your full node to hidden service peers, i hear it's a bit better. Once dandelion is ready, broadcasting transactions over TOR will be obsolete.
My GUI fixes this by broadcasting with an instance of electrum with TOR proxy. Want a PR, guys? (not easily compatible with windows, however. I'm working on porting my script to windows.)
Hi, I iam sorry to say that I gnore the meaning of PR. I am using ubuntu to run the fullnode and I would love to hear more about that please. Thank you
My GUI fixes this by broadcasting with an instance of electrum with TOR proxy. Want a PR, guys? (not easily compatible with windows, however. I'm working on porting my script to windows.)
Hi, I iam sorry to say that I gnore the meaning of PR. I am using ubuntu to run the fullnode and I would love to hear more about that please. Thank you
I made a GUI specifically for Ubuntu, but it will probably work on other linux that has grep and wget.
I'd need an expert to look at it before I'd recommend anyone broadcast transactions with it, and I'd appreciate if someone could review it.
Edited: I don't know Markdown formatting, so it's lines 603-639 of https://github.com/robust-cuttlefish/electrum-personal-server-GUI/blob/master/electrum-personal-server-gui.py
https://github.com/robust-cuttlefish/electrum-personal-server-GUI
The patch was based on the electrum documentation.
Once dandelion is ready, broadcasting transactions over TOR will be obsolete.
This is not actually true according to the authors of the dandelion paper (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-June/016071.html)
Since these guarantees are probabilistic, a node cannot be sure whether all
of its peers are monitoring it. Dandelion does not protect against these
adversaries, and individuals who are worried about targeted deanonymization
should still use Tor.
There's good discussion on this thread too https://www.reddit.com/r/Bitcoin/comments/6gsy4b/new_bip_called_dandelion_is_a_way_to_protect/
This is not actually true according to the authors of the dandelion paper
My mistake. would it be better to then torify a full node, and use both dandelion and tor? or is electrum over tor better?
Broadcasting over tor with dandelion deployed would be the best possible way in terms of broadcasting privacy. Torifying a full node is needed only if you want to hide the fact that you're even using bitcoin.
Now that bitcoin/bitcoin#13152 is merged, can we move forward with broadcasting over Tor?
Sure why not. Is there something stopping us?
Well, getnodeaddresses will be released in 0.18.0, so this needs to be backwards compatible with older versions.
How should it work exactly? If tor is running, pick a random onion address and broadcast? Or a set of random nodes?
Just wondering if you're working on this, since it was your issue and PR to have getnodeaddresses. If not I'd be interested in doing it.
I probably won't be working on it for the next few weeks/months, so if you're interested that would be great(!)
One way I imagine it working is that if Electrum sends the method blockchain.transaction.broadcast then EPS starts a new thread and that thread tries to connect to 4 other bitcoin nodes over tor which it finds using getnodeaddresses. Once there are 4 open connections which accept txes then the new EPS thread pushes the unconfirmed transaction to all of them and then closes each connection and ends the thread.
There don't have to be exactly 4 connections, I just came up with that number now. The purpose of the multiple connections is to reduce the chances of one node blackholing the transaction. The code has to be able to connect to tor hidden services too.
The config.ini_sample file would have two new configs:
tor_host = 127.0.0.1
tor_port = 9050
These fields are used to tell EPS where to connect to tor. Also the existing broadcast_method config will have a new option tor.
The code would have to implement the socks5 proxy protocol and the bitcoin p2p protocol. The old version JoinMarket has some python2 code implementing the p2p network, so that may be useful, although this code connects only to one node at a time and so has the black hole problem. https://github.com/JoinMarket-Org/joinmarket/blob/master/joinmarket/peertopeer.py (Note that this code has a bug where it will crash/go wrong if the newly connected peer doesn't send verack within 20 seconds before the first heartbeat.)
In later work EPS can be updated to allow connections from more than one wallet at a time, by recoding it to use asyncio or select(). When that happens then creating a whole other thread for tor broadcasting won't be necessary, so therefore it would be nice if any tor broadcasting code is easy to modify to use with asyncio or select() for when that gets added.
I merged PR #124 (thanks again for that @andrewtoth) and played around with it myself, and edited the code a little bit. We can carry on discussion here since the PR is closed.
Occasionally I get a few errors like this which come from the getnodeaddresses RPC call:
ERROR:2019-06-22 00:34:08,423: BitcoinCore v0.18.0 is required to broadcast through Tor
ERROR:2019-06-22 00:34:08,524: BitcoinCore v0.18.0 is required to broadcast through Tor
ERROR:2019-06-22 00:34:08,726: BitcoinCore v0.18.0 is required to broadcast through Tor
ERROR:2019-06-22 00:34:09,029: BitcoinCore v0.18.0 is required to broadcast through Tor
That happens because the RPC call returns an error. There are 8 active threads yet only 4 error messages appear, so I wonder if the cause is some concurrency issue of 8 threads trying to use the RPC instance all at the same time.
I wonder if theres a way to have the debug messages print which thread they're being called from, that might help with reading the debug log around tor broadcasting. I'll work on this more later.
Yes, it's a concurrency issue. That is why I have a sleep between starting the threads. Perhaps increasing the rpcworkqueue in bitcoin.conf could fix this.
This latest commit dd96954 moves all the getnodeaddresses RPC calls to the main thread, so avoiding this concurrency issue.
I've tested it a bit and I'm fairly happy with how it works. The INFO level logging looks like this:
INFO:2019-06-23 23:58:29,655: Listening for Electrum Wallet on ('127.0.0.1', 50002)
INFO:2019-06-23 23:58:30,947: Electrum connected from ('127.0.0.1', 37694)
INFO:2019-06-23 23:58:32,416: Broadcasting tx 7f2efb7b622dfc1a418ff1503925ab0be18d93d47005ac5cf1e974d67bab5bb2 with broadcast method: tor
INFO:2019-06-23 23:58:33,668: Uploaded transaction via tor to node ('74.138.26.36', 8333)
INFO:2019-06-23 23:59:04,140: Uploaded transaction via tor to node ('84.159.149.119', 8333)
INFO:2019-06-23 23:59:04,388: Uploaded transaction via tor to node ('2406:da18:f7c:4351:591b:4881:3986:3703', 8333)
INFO:2019-06-23 23:59:08,175: Uploaded transaction via tor to node ('104.198.94.212', 8333)
INFO:2019-06-24 00:00:38,826: Uploaded transaction via tor to node ('52.13.58.244', 8333)
INFO:2019-06-24 00:02:11,900: Broadcasting tx b0b76305823f9b0b2d098adae2fb238ffa4ad9b12a2d407c360f965b2031a3a0 with broadcast method: tor
INFO:2019-06-24 00:02:12,384: Uploaded transaction via tor to node ('82.221.111.136', 8333)
INFO:2019-06-24 00:03:13,934: Uploaded transaction via tor to node ('84.248.162.190', 8333)
INFO:2019-06-24 00:03:44,740: Uploaded transaction via tor to node ('23.98.43.170', 8333)
INFO:2019-06-24 00:03:49,769: Uploaded transaction via tor to node ('159.89.182.64', 8333)
INFO:2019-06-24 00:05:04,153: Uploaded transaction via tor to node ('23.125.224.84', 8333)
INFO:2019-06-24 00:06:40,223: Broadcasting tx 5a8f58bf0cfb305c613deba586a471c3cab74dc9ac8e938c5203e9ca44d657e9 with broadcast method: tor
INFO:2019-06-24 00:06:41,002: Uploaded transaction via tor to node ('18.204.135.109', 8333)
INFO:2019-06-24 00:06:41,634: Uploaded transaction via tor to node ('176.230.178.48', 8333)
INFO:2019-06-24 00:07:41,859: Uploaded transaction via tor to node ('74.105.145.200', 8333)
INFO:2019-06-24 00:07:42,514: Uploaded transaction via tor to node ('37.221.209.222', 8333)
INFO:2019-06-24 00:08:43,500: Uploaded transaction via tor to node ('158.174.131.171', 8333)
INFO:2019-06-24 00:10:48,609: Electrum wallet disconnected
Transactions to test with can be obtained by using getrawmempool and getrawtransaction on a bitcoin node. It's a bit odd that almost every tx in my node's mempool was accepted by about 4-5 other nodes out there on the network. I'd expect them to already be in most other node's mempools
I wonder if its useful to include the thread ID in the debug log messages too.
Very nice! I see from those logs you managed to broadcast to an ipv6 node... I recall from testing that it wouldn't work for ipv6, so not sure how that worked for you. If there's anything else you need for this let me know I'm happy to help.
No idea about the ipv6. I didn't change anything around that.
I've made a new issue (#130) asking people to help test. I've publicized it on all the social media so hopefully we'll get some users trying it out in real life conditions.
Edit: https://twitter.com/chris_belcher_/status/1143180966451695617
https://www.reddit.com/r/Bitcoin/comments/c4pzwu/if_you_use_electrum_personal_server_and_send/
https://www.reddit.com/r/Electrum/comments/c4q3ss/if_you_use_electrum_personal_server_and_send/
A feature I missed is that EPS could detect whether Tor is already running on the local host and then switch to that automatically. Users should still configure walletbroadcast=0 in their bitcoin.conf but this would still be helpful. Electrum itself apparently does it: https://github.com/spesmilo/electrum/blob/9f28f8bcc64d3b921f8bb94d38b584b862520c8c/electrum/gui/qt/network_dialog.py#L513
For credit and history sake, the above tor-detecting feature was suggested to me by ghost43 in the #electrum IRC channel
This feature is now in the new release 0.2.0, closing