Online Transacting via TOR Hidden Services

[DavidBurkett]

Rendered link to RFC document

The proposal is to support building transactions via TOR hidden services to provide bitcoin-esque addresses, prevent MITM attacks, and make transacting easier by eliminating the need to setup port forwarding.

This helps pave the way for offline transacting via an SBBS/Grinbox-like relay system (future RFC).

[antiochp]

Without going into too much of the details here - why TOR hidden services and not I2P hidden services?
Particularly given the current WIP here mimblewimble/grin#2932 and mimblewimble/grin#2712

It would not make much sense to use I2P for the node p2p layer and TOR for wallet communication.

This is not a comment on the merits of I2P vs TOR, more a comment that we should pick one and agree to use it consistently.

Maybe the initial effort here should be to better compare the two via proof of concept wallet implementations?

[DavidBurkett]

@antiochp I mentioned the reasons in #tx-building keybase channel, but in a nutshell, wallets aren't necessarily expected to be online long-term whereas nodes most likely are. It's harmful to the i2p network to connect for brief periods of time. Not so for TOR. Also, TOR makes it really easy to generate new addresses and there's a few other important advantages. i2p is still more suitable for the node, though.

[antiochp]

Pros/cons of isp vs tor notwithstanding - I do feel strongly that we need to use one tool and not two tools if it can be avoided.
We want to minimize the surface area of our dependencies and there is huge overlap between the two options here (which strongly suggests we should not be using both).

What is the downside of using tor hidden services (over i2p hidden services) for the long lived node? (Given that hidden services would not use exit nodes).

[DavidBurkett]

What is the downside of using tor hidden services (over i2p hidden services) for the long lived node? (Given that hidden services would not use exit nodes).

There's still a limited number of internal relays, but yes, avoiding exit nodes avoids most of the harm typical of tor usage. However, unless the whole network is switching to tor, we would have to connect to other peers through exit nodes, or setup a few dedicated peers to serve both networks. Those few peers become an obvious chokepoint though, and can quickly isolate the tor-only nodes if they go offline or choose to be malicious.

[lehnberg]

The hope is that the addressing mechanism described here will become the new default method for sending and receiving, deprecating several less secure and less private methods in the process.


As is mentioned in the RFC, tor introduces a non-trivial external dependency. To make this the default method, it would need to work for most types of devices and operating systems, and most users, merchants, and other ecosystem participants would need to be willing to adopt it. I see some of the below is mentioned in unresolved questions, but it would help me to understand how this proposal could work in practice, and how it avoids creating another standard that is used by only a minority of network participants.

Is tor bundled with grin-wallet by default? Or is it a separate install for the user? Does the wallet not work if tor is not also running?

How does tor hidden services work from mobile devices? Is it a matter of bundling it into grin-wallet, or is there some kind of iOS or Android app that needs to run in the background?

Are mining pools and exchanges expected to run tor in order to operate their wallets and transact with their customers?

From an end-user perspective, there should no longer be a need to configure routers and firewalls to receive grins. Sending and receiving should feel like any other cryptocurrency, where a simple encoded address is all you need to share before receiving. No firewall or router configuration should be necessary.

If this is the primary benefit of going with this approach, what is the motivation for going with tor over some other communications protocol that achieves this while creating less of an external dependency?

[DavidBurkett]

Great questions. I'll try my best to answer them.

and how it avoids creating another standard that is used by only a minority of network participants.

I believe the only way to avoid this is to be opinionated about it and put together a roadmap/plan to remove all other methods of transacting from the official client.

Is tor bundled with grin-wallet by default?

I personally think it should be if possible. Ideally, we would pull in just the logic needed to access the tor network and create/use hidden services, but just bundling tor with grin-wallet along with configuration info would be a good first step.

Does the wallet not work if tor is not also running?

The wallet won't be able to send/receive, except maybe through #25. We need to be careful to make sure we're not just going to end up creating 2 separate methods again though, with some exchanges/pools/wallets implementing method 24, and others implementing 25.

How does tor hidden services work from mobile devices? Is it a matter of bundling it into grin-wallet, or is there some kind of iOS or Android app that needs to run in the background?

Android has the option of using 'Orbot' (https://play.google.com/store/apps/details?id=org.torproject.android). iOS has several options for tor apps, but not sure about hidden services. Right now, mobile wallets rely heavily on a 3rd party though (a central node or your own external node), so perhaps it could be done at that level until we're able to bundle the parts of tor we need. Any thoughts @i1skn?

Are mining pools and exchanges expected to run tor in order to operate their wallets and transact with their customers?

Yes, but unlike i2p, tor is very lightweight. Your pc/server does not have to constantly use resources to encrypt/decrypt/route traffic. It largely just sits idle in the background waiting to be used.

If this is the primary benefit of going with this approach, what is the motivation for going with tor over some other communications protocol that achieves this

The primary benefit is that while providing privacy and preventing MITM. Do you have any specific alternatives? Everything else I've seen proposed thus far would not preserve the privacy of the transacting parties.

while creating less of an external dependency?

Tor is a non-trivial dependency for sure, but it's not much larger than many of our other options (if at all). It actually seems fairly minimal considering how much it's capable of doing: https://github.com/torproject/tor
In the future, as mentioned above, we could package just what we need, or if we somehow get a huge influx of devs, maybe even implementing just the specs we would be using would be an option.

[0xmichalis]

Everything else I've seen proposed thus far would not preserve the privacy of the transacting parties.

@DavidBurkett you mentioned in Gitter that transaction building via i2p is going to be a problem for the i2p network, can you elaborate on that? Also, how does i2p fare with preserving participants' privacy? Thanks

[DavidBurkett]

@Kargakis i2p suffers network traffic disruptions when peers leave the network, so it hurts the network when a bunch of people connect for brief periods of time. Wallets, especially those that use shared nodes, are much more likely to login, send s transaction or 2, and log back out. That's not as common for nodes.

FWIW, if we really prefer to not support both, nodes on TOR likely will have fewer negative consequences than wallets on i2p, so TOR would be my personal choice.

[yeastplume]

I'm not an expert, but I believe the potential disruptions caused by i2p vs TOR have to do with how routes are determined.

In Tor, a multi-use circuit is established at the beginning of the session. This is created by each server along a route picking a random peer, which in turn picks another random peer, and so on for a few hops until an exit node is reached. Each peer in the route needs to establish a shared key with the originating node, which then uses each key in succession to onion-wrap the packet. Each node along the route then unwraps just enough information with its shared key to know where to send the packet along its next hop, until the exit node is reached and the original message is sent off into the wild. The main observation for our purposes here is that it potentially takes a little while for a Tor circuit to be established (as you can observe whenever you try and connect a tor browser,) but the circuit is built dynamically by querying the p2p network, and no central database or lookups of routes are required (there is a central DB of tor nodes, but each node will pick the next hop in the circuit from that). This takes time, but once a circuit is established it's used for the entire session or until another circuit is requested.

i2p is fundamentally a mix network, and routing isn't performed through a single pre-determined circuit. In both Tor and i2p, the chain of keys along the route needs to be known by the originator before the packet is sent in order for it to be onion (or garlic, if you want) wrapped for its journey. Tor establishes one circuit per session and uses it exclusively. Establishing this route and getting all of the needed keys back to the originator before sending is where the main difference lies. Since Tor's method of node-by-node key establishment along the network to create a circuit is time consuming (far too time-consuming for a mix network), i2p maintains an internal distributed database (i.e. its DHT) of nodes and their public keys which (in a perfect world) is available to all participating nodes, meaning that an originator can choose and plan a route and wrap the packet with all the public keys in the tunnel without having to establish shared keys one-by-one. This is naturally much faster if the database is accurate, but comes with some obvious caveats that this shared database has to be replicated and propagated throughout the network in order for the network to be reliable. If nodes are constantly being brought online and taken down, this affects the reliability of the database (as each change to the network takes time to propagate,) and packets that get wrapped with a node key that doesn't exist will simply end up dropped as their route no longer exists.

In Tor, a client can interface with an long-lived entry point, but in i2p each client on the network is itself a node with an address. So if, in our case, a wallet comes online, starts getting its identity added to the database, transacts, then leaves the network, it affects the integrity of the internal routing table. This is why short-lived nodes are damaging to the i2p network.

Happy to be corrected if any of this is wrong, as I said I'm not an expert here.

Also, on thinking about this it's becoming apparent that neither of these are 'ideal' for all of Grin's use cases. Tor's issues are the usual known attacks around entry-exit nodes (edit: we're taking about Tor hidden services here, that's fine) and the amount of time it takes to establish a new circuit. I think for the wallet's case in particular, the general properties of mix networks are much more appropriate to transaction graph obfuscation, and in i2p in particular the fact there's a built-in DHT already providing addresses and keys is a very compelling point in its favour. However, as I've outlined in my thinking above the short-lived nature of wallet interactions are definitely a problem.

I would be very interested in more research and thinking about whether these issues with i2p could be overcome, or indeed if there are any other types of networks have been proposed (in the many, many years since i2p or Tor were created) that could be more suitable for our needs. For instance, a series of long-lived provider nodes that function over i2p would seem to be a possible approach (more in tune with the grinbox method of doing things,) if you could overcome 'last-mile' issues between provider nodes and the wallets themselves. (This seems a similar problem to Tor's entry/exit nodes anyhow.) If we needed to pick a single technology (and I agree trying to maintain 2 would be a nightmare,) just offhand it seems to me this particular 'short-lived-node' issue around wallets would be slightly easier to mitigate than trying to shoehorn Tor into Grin's p2p network.

I don't have any particular answers, but I do think we need to do a lot more thinking and research around this. And we need to multiply how careful we want to be about this by 100 if we want Grin to become opinionated about enforcing a particular method of exchange, (which I don't personally believe is the right approach, but that's another philosophical argument).

[lehnberg]

This is created by each server along a route picking a random peer, which in turn picks another random peer, and so on for a few hops until an exit node is reached.

It's worth noting that the RFC proposes to use tor hidden services, which works a bit differently, in that there is no exit nodes. This paper [1] gives a detailed breakdown of how hidden services work in section 2, and also outlines some fingerprinting techniques for attacking these.

Apparently, because the way they work, hidden services are in particular vulnerable to DDoS attacks [2]. It's not clear to me how they plan to address this, and whether it's been done yet.

---

A more general problem with Tor (or I2P for that matter) from a privacy perspective is that they are vulnerable to timing, intersection, and traffic analysis attacks. Since messages are immediately forwarded without delay and the data can be distinguished through network analysis, an attacker would be able to determine "grin slate" network fingerprints, filter for those particular communications, and over time, assign a probability to which computers on the network were communicating to generate certain outputs on chain.

To their credit, I2P is transparent with their threat model and their vulnerabilities to various attacks [3].

[1] https://www.freehaven.net/anonbib/cache/fingerprinting-wpes17.pdf
[2] https://www.zdnet.com/article/tor-project-to-fix-bug-used-for-ddos-attacks-on-onion-sites-for-years/
[3] https://geti2p.net/en/docs/how/threat-model

[DavidBurkett]

Thanks for responding @yeastplume and @lehnberg.

It seems like yeast is suggesting that TOR is slower to connect to than i2p. This is not true, as cryptominji pointed out in keybase yesterday. If you open tor, it takes 5-15 seconds to establish a circuit, and then you're fully online. In i2p, it takes several minutes to connect to the network, and even longer to fully integrate. I just ran i2p locally to test, and it was just over 2 minutes before I was able to access any of the common "hidden services of interest" on the i2p launch page.

re: privacy concerns. TOR hidden services do not have the same issues as typical TOR browsing using exit nodes. Also, timing attacks are far more difficult in a heavily used network like TOR. If there's a better proposal, I'm all ears, but every previous proposal has offered something significantly worse for privacy (magic wormhole, grinbox, etc). The only networks I know of that supposedly address this lack real world usage, and thus have a much smaller anonymity set (and I'm not even convinced they do address any of the issues they claim, but that's a different topic).

re: shoehorning TOR in p2p. i2p was the one being shoehorned, imho. It was barely discussed before implementation started on it, and there are still unanswered questions, like how do we deal with network fragmentation (ie since i2p peers can't talk directly to non-i2p peers, what happens when "relays" go offline?).

re: DDoS. One of the biggest fears of a privacy coin like Grin is that a nation state will DDoS to bring the network offline. TOR is actively funded by many of the largest nations, and is therefore far less vulnerable to this type of attack. Sure, DDoS is still possible, as it is in nearly every overlay network. Privacy almost always comes with spam/flooding attacks, because the more private the network, the more difficult it is to identify and ban attackers. But TOR has an excellent team working to ensure the network's health, and they have been developing and improving TOR for around a long time. i2p OTOH, lacks any real funding, uses slow and outdated crypto primitives, and development is sort of a fragmented mess atm.

We can spend forever pointing out flaws in each overlay network. No mix network is perfect. But TOR is actually usable for short connections, has the most usage by far which is great for privacy, and is well-funded and reliable.

@yeastplume As for needing to "do a lot more thinking and research", what do I have to do to convince you that I have? It's been my #1 concern for months now, and I've been (very publicly) discussing it and seeking answers for quite a while.

[yeastplume]

It seems like yeast is suggesting that TOR is slower to connect to than i2p. This is not true, as cryptominji pointed out in keybase yesterday. If you open tor, it takes 5-15 seconds to establish a circuit, and then you're fully online. In i2p, it takes several minutes to connect to the network, and even longer to fully integrate. I just ran i2p locally to test, and it was just over 2 minutes before I was able to access any of the common "hidden services of interest" on the i2p launch page.

I didn't mean to argue that Tor is slow to connect and therefore we shouldn't use it, I was just trying to reason about the cause of short-lived i2p nodes being bad for the network. Tor in particular establishes a single tunnel for all traffic between source and destination which it then re-uses, whereas some mix networks prefer to use different routes for each packet as part of their traffic concealment. A mix network therefore needs quick access to reliable routing tables, which is difficult to propagate if a lot of nodes are quickly joining and leaving. I agree this is a major problem with i2p for the wallet use case.

re: shoehorning TOR in p2p. i2p was the one being shoehorned, imho. It was barely discussed before implementation started on it, and there are still unanswered questions, like how do we deal with network fragmentation (ie since i2p peers can't talk directly to non-i2p peers, what happens when "relays" go offline?).

I don't think anyone was trying to shoehorn i2p into Grin, as far as I know the work that's been done to date has been very exploratory, and there's no way I'd advocate the introduction of i2p anywhere into Grin either without a fairly serious RFC and convincing research and testing gone into the subject. Actually, the more I learn about i2p, the less convinced I am it's the right direction for Grin. I'm definitely not in love with it, and agree with all of the concerns around it containing a lot of outdated cryptography. I'll add that its design seems fairly clunky compared to some other mix networks I've started to take a look at.

We can spend forever pointing out flaws in each overlay network. No mix network is perfect. But TOR is actually usable for short connections, has the most usage by far which is great for privacy, and is well-funded and reliable.

@yeastplume As for needing to "do a lot more thinking and research", what do I have to do to convince you that I have? It's been my #1 concern for months now, and I've been (very publicly) discussing it and seeking answers for quite a while.

I'm only just starting to look into this, but there's a whole world of mix networks out there that aren't i2p or Tor. Granted many of them are experimental or incompletely specced, and Tor indeed has the argument that it exists already, (which is a very strong one in its favor). And no, something doesn't have to be perfect before we start experimenting with it or even include it within Grin as an optional path. As far as experimenting with Tor and seeing what benefits it could bring to the network, I don't think anyone needs convincing this is a good research direction to take. If there were a successful working prototype that at least some users would like to use, I think it would be relatively easy to argue for merging it, provided it remains optional.

However, what you're advocating here is taking a decision to adopt Tor for all Grin communications with the goal of making it the default and eventually deprecating all other forms of communication. This is a huge decision that has massive implications on Grin's future direction. As has been pointed out, a decision like this could potentially cause a lot of pain for end users and developers, and I also believe that Grin becoming opinionated on this will cut off all future explorations into other networks that might be more appropriate for Grin's use cases. For this to happen, the entire community would need to be convinced that the benefits of adopting Tor exclusively outweigh all of this potential pain. At the moment, we don't even have a prototype. I believe it's far too early and there's far too little evidence to abandon our open, experimental development path in favor of a single, untested option. This is a very big ask, and it would require a lot of convincing to get everyone on board, particularly at this stage.

I'm not trying to throw up barriers and say we need the most rigorous and academic RFC possible, but I think finding the right network for Grin is going to take quite a lot of time and experimentation. Doubly and triply so (x1000) if we were to suddenly choose one to the exclusion of all others. I think the community would need at least some level of comfort that all other options have at least been considered and also buy into whatever reasoning exists behind why a particular choice is the best possible for Grin's future direction given the current technological landscape.

[DavidBurkett]

However, what you're advocating here is taking a decision to adopt Tor for all Grin communications with the goal of making it the default and eventually deprecating all other forms of communication.

If by all Grin communications, you mean for tx building, then yes that's what I'm advocating here. Obviously, the first step is implementing it as an option, ensuring it works well for everyone, and then deprecating other options. Without the intention of deprecating other options though, this just makes Grin even more confusing. Any approach that still leaves HTTP(s) tx building indefinitely, and therefore requires non-technical users to learn how to open ports, is not a solution worth pursuing imho. If that's not what we're trying to accomplish, then Grin's usability problems will certainly remain.

[minjimw]

I don't know why I2P is being mentioned here. I2P is not the correct tool for the job and anybody who has spent 10 minutes using it would know that.

Downsides of building transactions via Tor hidden services are:

1. It adds latency to transactions, but this usually wouldn't be any more than, say ~10 seconds.

2. Tor is more susceptible to DOS than http(s) over IP, but that would be true for any layer over IP. In the case of building transactions, I deem this an acceptable risk given that Tor has been running strong for years. Worst case scenario is that easy transaction building temporarily breaks, it wouldn't harm the network itself.

3. It adds some complexity to wallet development. I don't think this is problematic.

But what we get in return is:

1. Improved UX compared to existing http(s) schemes. No port forwarding is required for this.

2. Improved privacy. I cannot believe a privacy project in 2019 encourages exposing your IP to everyone you interact with. Tor is practically the established solution to unlinking your IP and actions on the internet. If people have issues with Tor because it's not private enough, they are probably looking for something that doesn't exist. Tor is the best existing solution for network level obfuscation that meets our requirements. Lack of network level obfuscation (current situation) is clearly on the opposite side of the spectrum and should be considered harmful.

My understanding is that the only blocker for this is irrational fear of being opinionated on the topic of transaction building. If we aren't opinionated, there will be indefinite fragmentation between wallets. Or, even worse, one wallet captures a majority of Grin users and forces their own standard on everyone. That is a risk because their own standard may be harmful to privacy.

[lehnberg]

Various questions/feedback raised in the last dev meeting:

• What's the adoption path? Is it preferred method on some platforms?
• Any peculiarities of service configurations?
• What's the key derivation path used for service naming?
• How are addresses handled? Can addresses be reused? Are they stored or managed somewhere?
• What's the rationale for bundling / non-bundling tor, and which approach is recommended?

Updates to the RFC which answer some of these questions can improve the chances for support of the proposal.

[yeastplume]

Current thoughts on this:

• What's the adoption path? Is it preferred method on some platforms?

Depending on how users get on with it once the optional functionality is merged into master, I think it's to become preferred where available... all going well we'd ultimately want to prefer this with a grinbox-based fallback for mobile or other configurations where Tor isn't easily available. We still need to do a full survey of where this won't be a runner, and include findings in this RFC, for instance:

• iOS Devices (although they don't HTTP listen at present anyhow)
• Situations where Tor is blocked on a corporate network, or blocked by a state at a wider level
  ...?

• Any peculiarities of service configurations?

Don't know of anything specific we need to note around here for the time being

• What's the key derivation path used for service naming?

Good question, at present the wallet's parent accounts are at

m/0/0 -> default
m/1/0 -> account_1
m/2/0 -> etc

So keys are created at (in account_1 above, for instance:)
m/1/0/0
m/1/0/1
m/1/0/2 etc

So as a first suggestion I might suggest hidden service addresses derived from (again using account_1 above)

m/1/1/0
m/1/1/1
m/1/1/2

Where the private key of the derived onion address is BLAKE2(key_at_path(deriviation_path))

So the first address above would be
BLAKE2(key_at(m/1/1/0))

• How are addresses handled? Can addresses be reused? Are they stored or managed somewhere?

Since they're coming off of a particular derivation path, they don't need to be stored other than perhaps an index of the last derivation path that was used (though this wouldn't survive a full restore).
However, like derivation paths addresses would be consistent between restores.. it might be a possibility to leave it up to the individual wallet to decide how many derived tor addresses it would like to have active at once, but the APIs should have functionality to output new addresses or regenerate keys and start onion services for a particular block of derived keys.

• What's the rationale for bundling / non-bundling tor, and which approach is recommended?

On platforms with strong package manager support (e.g linux distros), I think we should be 'bundling' via providing proper distro packages. This is something I think we need to address sooner or later anyhow as providing nothing but executables via a build can only take you so far. I think we'd want to be bundling grin and grin-wallet together anyhow and just keep tor as a dependency in this case.

On windows, I think we'd need to include a pre-built tor binary as part of an install package.

Mac is kind of a middle case, (perhaps we provide brew packages that can install the tor dependency)

[0xmichalis]

FWIW, Wasabi in Bitcoin ships w/ Tor bundled and the whole thing runs seamlessly. Very smooth experience.

https://wasabiwallet.io/

EDIT:
Also bootstrapping to Tor was pretty fast:

$ ./wassabee 
2019-10-08 22:55:56 INFO	Program (43)	Wasabi GUI started

Oct 08 22:56:06.000 [notice] The current consensus contains exit nodes. Tor can build exit and internal paths.
Oct 08 22:56:07.000 [notice] Bootstrapped 57%: Loading relay descriptors
Oct 08 22:56:07.000 [notice] Bootstrapped 62%: Loading relay descriptors
Oct 08 22:56:07.000 [notice] Bootstrapped 69%: Loading relay descriptors
Oct 08 22:56:07.000 [notice] Bootstrapped 78%: Loading relay descriptors
Oct 08 22:56:07.000 [notice] Bootstrapped 80%: Connecting to the Tor network
Oct 08 22:56:08.000 [notice] Bootstrapped 85%: Finishing handshake with first hop
Oct 08 22:56:08.000 [notice] Bootstrapped 90%: Establishing a Tor circuit
Oct 08 22:56:08.000 [notice] Tor has successfully opened a circuit. Looks like client functionality is working.
Oct 08 22:56:08.000 [notice] Bootstrapped 100%: Done

[lehnberg]

Yes, Wasabi does that, and IIRC allows you to turn "wasabi tor" off in settings if you're already have a circuit established through some other tor you got running. In my experience that was a manual process, which is probably "good enough" for a first implementation.

It would be useful to have a survey of various applications and how they approached the question, and how it is handled across different platforms and OSes.

[CircusDad]

For what it's worth... Tangram is a still incomplete project BUT their testnet was up and functional with tor. It may be helpful to look at their code for those with the proper skill: https://github.com/tangramproject/

[yeastplume]

All good, and once again thanks for putting all of this together and pushing us in this direction. After having implemented this in grin wallet and seen the benefits, I'm convinced this is the right way to go for the time being.

Just a few updates to reflect further thinking we've had, as well as a couple of definitions around derivation paths and generation of keys, and I think this is ready for final comment period.

[yeastplume]

In accordance with our RFC process, as of November 29th, 2019 this can be considered in Final Comment Period with a disposition to merge.