Bounty: Route BitcoinJ over Bitsquare's JTorProxy

[ManfredKarrer]

Requirements for taking that bounty:

• Expertise with Java
• Expertise with Tor
• Expertise with BitcoinJ
• Expertise with the usage of JTorProxy and the related libraries/network module in Bitsquare

We used to run BitcoinJ over Tor with its built in Tor support (Orchid) but that was not very stable, so we deactivated that. As we use a native Tor instance for the P2P network it would be better to route BitcoinJ over our JTorProxy.
BitcoinJ uses nio (netty) which does not provide an API for sockets, so that might be difficult to get working with Socks5Proxy. But it also supports a BlockingClientManager (no nio) which is also used when BitcoinJ runs over Orchid.
We would need to pass our socket (or socketFactory) to the BlockingClientManager.
We maintain our own fork (https://github.com/bitsquare/bitcoinj) of BitcoinJ so changes need to be added there.

If you want to work on that bounty it requires close communication. I need to be sure that you fulfill the above requirements! Also the final scope of the bounty will be decided during work on it. The bounty amount will be paid out for a working implementation where BitcoinJ is routed over our Tor instance and it does not conflict with the Bitsquare P2P network. The work might require more effort as it is covered by the bounty, if work relationship turns out positive we can arrange alternative reward schemes.

Bounty address (initial funding 1 BTC):
https://blockchain.info/address/1LkUscnxVAeNsuRMu9Veuo6MLaoDEG9JXw

Related issues: #357

[dan-da]

I was surprised to notice that bitsquare is connecting to mainnet directly and not through tor, since the p2p operates through tor. I came to github to find if there is an open issue or else to create one, so I'm glad to find this writeup.

one question I have. It is easy to route bitcoin-core through a tor proxy simply by setting a couple config settings (proxy, onlynet). Does bitcoinJ not support proxies? And if it does support a proxy, then couldn't users at least configure it to route through their O/S tor proxy? Or alternatively through bitsquare's tor if it exposes an open proxy port? I'm just thinking that could be a way to use existing plumbing rather than integrate at a lower level.

[ManfredKarrer]

I don't know how easy it is to route it from the outside but from the BitcoinJ side it should not be too hard, but needs some dev effort I don't have at the moment.
As Bitsquare works with zero config I am not very interested in some manual setup how to get it routed over Tor. It needs to work out of the box for non-techies. I had Tor activated earlier with the Orchid Tor implementation in BitcoinJ but that was not very stable and as we already run a native Tor instance it would make more sense to re-use that also for BitcoinJ instead of running 2 Tor instances.

[dan-da]

yeah definitely better to use internal tor. Still ability to use external if desired could be a nice option, even if only accessible via a config file or env var.

fwiw, I found out how to set socks proxy in bitcoinj:

System.setProperty("socksProxyHost", "127.0.0.1");
System.setProperty("socksProxyPort", "9050");
peerGroup = new PeerGroup(params, chain, new BlockingClientManager());

I'm not sure if using that would be much different from your proposed solution. Anyway, my next task is to get bitsquare built from source and then perhaps can dig deeper.

[ManfredKarrer]

Yes that BlockingClientManager need to be used. Would be great if yoou have time to check it out.

[dan-da]

I made some progress on this. I was able to connect with peers over Tor using the system service Tor (localhost:9050). I did this with a subclass of WalletAppKit. No change required to bitcoinj library. Tested by shutting down system tor and restarting. Peers dropped off and came back as expected.

Question: how to delay bitcoin network connections until bitsquare tor is up and running? A pointer could save me lots of digging around.

[ManfredKarrer]

In MainViewModel is the startup process. For BitcoinJ there is a listener as well as for other subsystems like the P2P network. You could add here some delays/handling for Tor is ready...

Great that you could make it work that way, though for Bitsquare I need a no-config solution, so it requires using our internal Tor instance.
If you like to dig into that, you can check out the network module.
TorNetworkNode (createSocket() and startServer are the methods where you get the Tor sockets). The Server and Connection classes are the one using those sockets.
I am not sure how to plug a socket to the BitcoinJ nio version (the standard and faster one) but with the nonblocking BitcoinJ classes it should work. Though better would be to get it working with nio.

[dan-da]

update.

I got it wired up so that the jtorproxy proxy host and port are passed to the code that sets socksProxyHost and socksProxyPort for bitcoinj (per my July 5 comment) -- after p2p service is up.

However when I run it this way I get errors like:

Jul-10 13:08:32.143 [BlockingClient network thread for dnsseed.bluematt.me/88.198.60.110:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/88.198.60.110:8333: SOCKS: Connection not allowed by ruleset

Tracking down the exact origin or meaning of this "Connection not allowed by ruleset" has proven difficult.

Some interesting points:

1. If I configure a webbrowser to use the jtorproxy host/port, I am able to browse the web fine with it.
2. If I configure socksProxyHost/socksProxyPort to a regular socks5 proxy (eg ssh's proxy) it is able to connect normally.

So both of those things work ok in isolation, but not when used together.

socksProxyHost/socksProxyPort are System/JVM properties. I think setting those may be causing the proxy to try to connect to itself, or something like that.

[dan-da]

ok, I'm getting a strong feeling that using socksProxyHost/socksProxyPort is the wrong path. Because those settings are VM-wide.

This page is helpful. http://docs.oracle.com/javase/7/docs/technotes/guides/net/proxies.html

Now looking at bitcoinj to see if I can integrate java.net.Proxy. I was hoping it wouldn't be necessary to modify bitcoinj at all.

[dan-da]

update.

I added a sub-class of SocketFactory called ProxySocketFactory to pass to BlockingClientManager. This replaces socksProxyHost vm-wide settings. Even so, I still get the "Connection not allowed by ruleset" error when trying to use internal socks proxy. But external proxies work fine.

Still scratching my head over that.

Also: Using the SocketFactory subclass mechanism, it should be possible to pass internal sockets directly to bitcoinj without need to connect to internal proxy at all. But it requires more investigation.

[ManfredKarrer]

The sub-classing SocketFactory approach sounds good. No idea about that "Connection not allowed by ruleset" message :-(.
Do you have commited the changes already? I did not find a repo.

[dan-da]

No I haven't forked the bitsquare repo yet. maybe I can do that today.

[dan-da]

@ManfredKarrer I pushed some work in progress code in case you want to take a look. It's at https://github.com/dan-da/bitsquare/tree/route_bitcoinj_over_jtoryproxy.

it's rough and some things are not in the right place, I will refactor later after basic concept is proven.

I have not solved the "connection not allowed by ruleset issue yet". Suggestions welcome.

To route bitcoinj over system Tor instead of jtorproxy, uncomment the following line in MainViewModel.java:

//            nodeAddressProxy = new NodeAddress("localhost", 9050);

Also, I just tried modified approach where I create a new JTorSocketFactory class that uses SocksSocket similar to what TorNode does, using Socks5Proxy class, etc. It compiles now, but I get an error connecting to proxy when the first socket is created. still debugging that.

Assuming this can be made to work, I think that cleanest final approach would be to refactor TorNode class to use JTorSocketFactory internally. Then, this same SocketFactory subclass can be passed directly to bitcoinj and we have just one class responsible for creating tor sockets.

[dan-da]

I ran some tests using an external client (curl) and it appears to be related to the way that jtorproxy handles non-standard ports. Using an external client rules out some sort of integration weirdness inside bitsquare between jtorproxy and bitcoinj.

I should preface this by stating that I have not yet tried connecting to .onion bitcoin nodes. That might very well work. So far I've been trying to connect to the standard nodes via tor exit nodes. I think that ideally it should be possible to use either .onion or standard nodes.

== Tests Against jtoryproxy inside bitsquare ==

Connect to a bitcoin peer.

$ time curl -sSf --socks5-hostname localhost:53318 80.212.163.202:8333 > /dev/null
curl: (7) Can't complete SOCKS5 connection to 0.0.0.0:0. (2)

real    0m0.022s
user    0m0.008s
sys     0m0.008s

This is very quick and seems to be denied without going over tor network.

Connect to http website port 80 ( google.com )

$ time curl -sSf --socks5-hostname localhost:53318 google.com > /dev/null

real    0m0.768s
user    0m0.012s
sys     0m0.008s

This takes longer and returns data.

The difference in responses of the above two requests demonstrate that I can use jtorproxy as a proxy for firefox successfully but bitcoinj requests will fail.

== Tests Against standard Tor system service ==

Connect to a bitcoin peer.

$ time curl -sSf --socks5-hostname localhost:9050 80.212.163.202:8333 > /dev/null
curl: (52) Empty reply from server

real    0m0.652s
user    0m0.008s
sys     0m0.012s

Got a different response than from jtorproxy and took significantly longer. Actually, this response is cached, the very first time took over 3 seconds. This indicates to me that the request actually traversed tor network. note: "Empty reply from server" is not really unexpected because we are making an http request to a bitcoin p2p service.

== Next Steps ==

1. try to connect to .onion bitcoin peers over jtorproxy. ( low hanging fruit. )
2. investigate further into why jtorproxy doesn't appear to like certain ports. ( is there a policy file somewhere? )

[dan-da]

connect to an onion bitcoin peer, over jtorproxy

time curl -sSf --socks5-hostname localhost:53318 boa3km6cince3nqo.onion:8333 > /dev/null
curl: (52) Empty reply from server

real    0m7.563s
user    0m0.008s
sys     0m0.008s

Now we get the same result as with standard tor. this is encouraging!

== Next Steps ==

1. try to connect bitcoinj to .onion bitcoin peers over jtorproxy
2. investigate further into why jtorproxy doesn't appear to like certain ports. ( is there a policy file somewhere? )

I don't have any time to work on this again until Monday.

@ManfredKarrer what are your thoughts about connecting bitcoinj only to .onion bitcoin peers vs to standard peers over tor? Assuming both can be done, do we need a config option for it?

Also, any idea why jtorproxy might be blocking ports?

[ManfredKarrer]

@dan-da Wow, that is already great progress to get closer to a solution!

Regarding connection to Bitcoin hidden services nodes:
It would be cool to support both and let the users decide what he want to use. Of course a default decision should be done by us.
I am still not sure if Bitcoin over Tor is a good idea. There have been a paper which listed a few attack vectors when using hidden services for Bitcoin, but that was criticized by some Bitcoin devs, so that might have been a false alarm.
Hidden services nodes would be faster as we don't need the exit nodes which are usually the bottleneck. So that would be an argument in favor of HS.

Regarding the problem in jtorproxy: I also don't have an idea what can cause it. If it would have failed with a normal webpage request as well I would have thought there is some config allowing hidden services only, but as that suceeded it must be something else.

Btw. I should add a prog. arg for passing a custom bitcoin node. As the bloom filters are pretty weak reagarding privacy connection to your own full node would be the only good solution atm to fix that. Bitsquare should also run bitcoin nodes (clearnet and HS) so those could be used as well in favor of public nodes, if the user prefers that.

[dan-da]

Success!

The problem was the SafeSocks 1 is set in torrc. This setting permits hostname lookups only and rejects IP lookups. They assume that any IP lookups are because the client called DNS itself.

After recompiling with SafeSocks 0, my test curl request works and also bitcoinj connects successfully and quickly found 11 peers without error. All appears normal in the app.

bitcoinj has various Peer Discovery strategies and I need to play around with them more. I believe that in some cases peers are provided as IP only... anyway we need to be able to support that. So I suggest that we permanently set SafeSocks 0, but set TestSocks 1, at least during development, so it will warn of any raw IP connections but permit them.

Docs from https://www.torproject.org/docs/faq.html.en#SocksAndDNS

How do I check if my application that uses SOCKS is leaking DNS requests?

These are two steps you need to take here. The first is to make sure that it's using the correct variant of the SOCKS protocol, and the second is to make sure that there aren't other leaks.

Step one: add "TestSocks 1" to your torrc file, and then watch your logs as you use your application. Tor will then log, for each SOCKS connection, whether it was using a 'good' variant or a 'bad' one. (If you want to automatically disable all 'bad' variants, set "SafeSocks 1" in your torrc file.)

Step two: even if your application is using the correct variant of the SOCKS protocol, there is still a risk that it could be leaking DNS queries. This problem happens in Firefox extensions that resolve the destination hostname themselves, for example to show you its IP address, what country it's in, etc. These applications may use a safe SOCKS variant when actually making connections, but they still do DNS resolves locally. If you suspect your application might behave like this, you should use a network sniffer like Wireshark and look for suspicious outbound DNS requests. I'm afraid the details of how to look for these problems are beyond the scope of a FAQ entry though -- find a friend to help if you have problems.

[ManfredKarrer]

Ah good find!!! Congrats!
I think there is no case in Bitsquare for DNS lookups, so I think it is safe that we change that settings.
I was planning a new release soon. Maybe we can get it into that release. I can add the UI stuff for letting the user decide to use Tor or not for BitcoinJ. It will be stored as a boolean in the Preferences (useTorForBitcoinJ).
I will also add a param to set custom btc seed node addresses.
Do you know if there is a kind of Seed server which delivers onion addresses, or if the default seed nodes are delivering those as well with a certain flag? Never looked into that so far...

You can send me also your bitcoin address for receiving the bounty. I consider it as solved. But of course you are very welcome to help more (refactoring the handling of the socketfactory).

There is also another related issue open (#522), would be great if we get that solved as well. I will add another 0.5 BTC to that if you can solve that.

Thanks a lot so far!!!!

[dan-da]

Do you know if there is a kind of Seed server which delivers onion addresses, or if the default seed nodes are delivering those as well with a certain flag? Never looked into that so far...

I'm not certain if .onion seed addresses are available from DNS or not. bitcoinj has a class TorDiscovery that performs DNS lookups and there is a comment "// TODO handle .onion CNAME replies" that seems to indicate maybe it is. research needed. I think what we can do for a first release is create a PeerDiscovery subclass with a list of known Tor seed nodes. That, plus user configurability should be sufficient.

You can send me also your bitcoin address for receiving the bounty. I consider it as solved. But of course you are very welcome to help more (refactoring the handling of the socketfactory).

Thanks. I don't like to leave things messy, so I will clean up and submit a proper pull request, and include a BTC address at that time.

There is also another related issue open (#522), would be great if we get that solved as well. I will add another 0.5 BTC to that if you can solve that.

Yeah, I will take a look at that. separate branch, pull request.

I was planning a new release soon. Maybe we can get it into that release.

Sounds good. I should be able to have a pull request ready by mid-week.

[ManfredKarrer]

Cool! I added quite a lot of changes recently to the dev branch, so if u branched from that maybe good to update...
Btw. I think its better to do another release for the Tor stuff once all is in place. It would be too much rushing to that release, which is already postponed way too long.

Appreciate a lot your help!

[dan-da]

update: I just cleaned up a little and pushed.

There is an exception being thrown when the app exits. Seems like the p2pService is closing the proxy port before bitcoinj closes, so it tries to send over closed socket. I'm not familiar with the exit/cleanup code, so that would be research for me.

Now I am merging changes from your dev branch, and will test with that and see what's left to do, eg integration of seed nodes via program args?

[ManfredKarrer]

Cool! Yes the shutdown handling needs prob. some adoption but I can do that once I merge it into dev branch.
I just pushed (to dev branch) support for seednodes passes as prog args and useTor flag.
I think the seednodes are passed via walletAppKit.setPeerNodes() but not 100% sure. Maybe u can check that out...
Also re-activated use tor checkbox in settings/network and in preferences. So user can change there without prog args as well. Currently is uses Orchid Tor when activated.

[dan-da]

ok, I will take a look at that.

I was just trying to add .onion seed nodes, but it turns out tricky to do for two reasons:

1. The default seed nodes are specified according to network (mainnet, testnet, etc) and those are buried inside bitcoinj.
2. The bitcoinj peer discovery code is not friendly to this idea. It is requiring IP addresses instead of hostnames. Certain parts of it use InetAddress class instead of InetSocketAddress.

Anyway, it is certainly not plug/play and It is unclear that it is possible to achieve without modifying bitcoinj, something I've been able to avoid so far, and I think is getting outside the scope of this issue/bounty. So I am dropping that for now.

[dan-da]

actually, I was not aware of setPeerNodes(). Maybe there is something helpful there. I will look.

[dan-da]

update. just pushed some code.

I am now utilizing the useTor flag. If on, jtorproxy is used and walletservice init is deferred until tor is ready. If off, neither tor is used and walletservice init is immediate.

I started to wire up the seedNodes var, but there are a couple issues:

1. seedNodes is always empty. I tried running like:

java -jar gui/target/shaded.jar --seedNodes=119.9.126.111:8333

and

java -jar gui/target/shaded.jar --seedNodes 119.9.126.111:8333

--help shows the --seedNodes usage, but I never receive the value (in WalletService). Not sure if I'm doing something wrong, or what.

2) setPeerNodes() accepts PeerAddress which requires a resolved IP ( or else it tries to resolve for you, over DNS ). This means that: - .onion addresses won't work at all. - regular hostnames would be resolved by InetAddress outside tor, thus leaking user's IP. The second issue (leaking IP) can be worked around by performing lookups ourselves over Tor first. The first seems like it requires significant re-working of bitcoinj internals, though I haven't determined full scope yet.

My feeling is that for first release, we simply do not support .onion peers. It should be simple if bitcoinj supports them, but as far as I can tell it does not.

On second thought, I'm not sure about the above. bitcoinj might correctly use the proxy for this, since it was already given the BlockingClientManager. I will test.

Also, before making a pull request, I plan to run with wireshark to make sure we are not leaking IP in unexpected ways, especially DNS.

[dan-da]

So I tested by hardcoding a .onion address in the seedNodes var. I get a null pointer exception because it is not resolved and PeerAddress really wants it to be. Using a static IP or regular hostname does work. Indicating that the DNS is being resolved locally, not over Tor.

I am getting a bad feeling that bitcoinj just simply leaks DNS queries, at least for some operations.

This quote from Mike Hearn last year is not really comforting:

We need to move the Orchid TorClient object into Context, because there are several parts of bitcoinj that make network requests which should potentially be Torified, not just PeerGroup.

here is the exception:

java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:210)
        at org.bitcoinj.core.PeerAddress.<init>(PeerAddress.java:78)
        at org.bitcoinj.core.PeerAddress.<init>(PeerAddress.java:89)
        at org.bitcoinj.core.PeerAddress.<init>(PeerAddress.java:100)
        at io.bitsquare.btc.WalletService.initialize(WalletService.java:293)
        at io.bitsquare.gui.main.MainViewModel.initWalletService(MainViewModel.java:504)

will look into that more tomorrow.

[ManfredKarrer]

The prog arg is: --btcSeedNodes not --seedNodes. seedNodes is used for P2P network seed nodes.

Yes I agree, lets ignore the onion support for the first version. That can be worked on in a second step. It is also still not clear how to get the onion addresses of seed nodes.
Also I would like ot run then bitcoin nodes myself both clearnet and tor and best would be if they are configured to connect to both networks to avoid isolation attacks. But I will not have time for that soon...

I think we can also only support IP addresses for btcSeedNodes. That would be probably the default usage anyway I assume.

[dan-da]

Unfortuately java networking classes do not support resolving hostname over socks proxy. See for example this bug: https://bugs.openjdk.java.net/browse/JDK-8028776

I was able to work around this by using SocksSocket to perform a connect over Tor, after which the IP address is available from socket object. Right now I have it working for the btcSeedNodes args. Next step will be to make it into a Discovery subclass so it will work for the default bitcoin seeds used by bitcoinj.

Presently it freezes the UI during each connect, so I guess the lookup should be moved into its own thread. My java-fu is not that strong, so I intend to leave that to you.

It is also still not clear how to get the onion addresses of seed nodes.

I think I see a way to make .onion addresses work, via a Discovery subclass, at least for the default seeds. I've already made a successful connection. Wiring up the --btcSeedNodes would be trickier because PeerAddress throws null pointer exception if no IP address found.

Here are the .onion seeds used in bitcoin.

3ffk7iumtx3cegbi.onion:8333
3nmbbakinewlgdln.onion:8333
4j77gihpokxu2kj4.onion:8333
546esc6botbjfbxb.onion:8333
5at7sq5nm76xijkd.onion:8333
77mx2jsxaoyesz2p.onion:8333
7g7j54btiaxhtsiy.onion:8333
a6obdgzn67l7exu3.onion:8333
ab64h7olpl7qpxci.onion:8333
am2a4rahltfuxz6l.onion:8333
azuxls4ihrr2mep7.onion:8333
bitcoin7bi4op7wb.onion:8333
bitcoinostk4e4re.onion:8333
bk7yp6epnmcllq72.onion:8333
bmutjfrj5btseddb.onion:8333
ceeji4qpfs3ms3zc.onion:8333
clexmzqio7yhdao4.onion:8333
gb5ypqt63du3wfhn.onion:8333
h2vlpudzphzqxutd.onion:8333
n42h7r6oumcfsbrs.onion:4176
ncwk3lutemffcpc4.onion:8333
okdzjarwekbshnof.onion:8333
pjghcivzkoersesd.onion:8333
rw7ocjltix26mefn.onion:8333
uws7itep7o3yinxo.onion:8333
vk3qjdehyy4dwcxw.onion:8333
vqpye2k5rcqvj5mq.onion:8333
wpi7rpvhnndl52ee.onion:8333

comments from OnionSeedPeers class I was working on:

        // this list comes from
        //    https://github.com/bitcoin/bitcoin/blob/master/contrib/seeds/nodes_main.txt
        // which is used by:
        //    https://github.com/bitcoin/bitcoin/blob/master/contrib/seeds/generate-seeds.py
        // which generates output included in:
        //    https://github.com/bitcoin/bitcoin/blob/master/src/chainparamsseeds.h

more later.

[ManfredKarrer]

Ah great progress!
IF you want/need to change anything in BitcoinJ feel free. I have already changed quite a bit. And in future if we have enough strong BitcoinJ devs we might do even more (inverted Bloomfilter project by @Giszmo).

[Giszmo]

@ManfredKarrer with Transaction Finder I don't intend to touch bitcoinJ but I'm confident now that with 1.5GB of data you will have a good index of the total blockchain, allowing you to quickly (on my old laptop it's 38s for 3 addresses but I assume it's io bound, so it's basically the same for 1000 addresses) find all blocks relevant to your SPV wallet, with a false positive rate of 0.0001%.

[ManfredKarrer]

Would be an interesting sub project to integrate that with BitcoinJ so the BitcoinJ bloom filters are not used anymore and the transactions are requested from the full nodes directly. But for sure a larger project...

[ManfredKarrer]

Thanks @dan-da for the PR!
Bounty paid out:
https://blockchain.info/tx/096314f54d17b419385258c7f5cb1cd738ea0925301784dc9adec209dfe1c0ac

Need still a bit for testing, got strange performance issues, but need to work first on another task.

[dan-da]

Thanks. I have noticed that there are sometimes some perf and/or lookup issues during startup when using DNS over Tor. If you want to try running with local DNS lookup instead, just comment out this line in WalletAppKitBitsquare.java:

this.setDiscovery( new SeedPeersSocks5Dns(socks5Proxy, params) );

There are likely still some improvements that can be made for this operation mode. I included a comment in the code that we could include a user pref for user to decide whether to run DNS over Tor or not.

If still issues, you could try running the commit before I introduced DNS over Tor, which seemed pretty solid to me: 89ab87d

If either or both of the above changes improve/fix your perf issues, that can help isolate the problem.

anyway I will test some more when I can find time.

[dan-da]

related, I find messages like this at startup and periodically while running:

Jul-22 08:39:31.760 [BlockingClient network thread for dnsseed.bluematt.me/85.25.195.142:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/85.25.195.142:8333: SOCKS: Connection refused 
Jul-22 08:39:31.852 [BlockingClient network thread for dnsseed.bluematt.me/85.25.195.142:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/85.25.195.142:8333: SOCKS: Connection refused 
Jul-22 08:39:31.853 [BlockingClient network thread for dnsseed.bluematt.me/85.25.195.142:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/85.25.195.142:8333: SOCKS: Connection refused 
Jul-22 08:39:31.890 [BlockingClient network thread for dnsseed.bluematt.me/85.25.195.142:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/85.25.195.142:8333: SOCKS: Connection refused 
Jul-22 08:39:31.891 [BlockingClient network thread for dnsseed.bluematt.me/85.25.195.142:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/85.25.195.142:8333: SOCKS: Connection refused 
Jul-22 08:39:31.891 [BlockingClient network thread for dnsseed.bluematt.me/85.25.195.142:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/85.25.195.142:8333: SOCKS: Connection refused 
Jul-22 08:39:31.987 [BlockingClient network thread for dnsseed.bluematt.me/85.25.195.142:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/85.25.195.142:8333: SOCKS: Connection refused 
Jul-22 08:39:32.000 [BlockingClient network thread for dnsseed.bluematt.me/85.25.195.142:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bluematt.me/85.25.195.142:8333: SOCKS: Connection refused 
Jul-22 08:45:05.074 [BlockingClient network thread for dnsseed.bitcoin.dashjr.org/98.209.219.85:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bitcoin.dashjr.org/98.209.219.85:8333: SOCKS: TTL expired 
Jul-22 08:45:05.075 [BlockingClient network thread for dnsseed.bitcoin.dashjr.org/98.209.219.85:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bitcoin.dashjr.org/98.209.219.85:8333: SOCKS: TTL expired 
Jul-22 08:45:05.192 [BlockingClient network thread for dnsseed.bitcoin.dashjr.org/98.209.219.85:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bitcoin.dashjr.org/98.209.219.85:8333: SOCKS: TTL expired 
Jul-22 08:45:05.192 [BlockingClient network thread for dnsseed.bitcoin.dashjr.org/98.209.219.85:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bitcoin.dashjr.org/98.209.219.85:8333: SOCKS: TTL expired 
Jul-22 08:45:05.218 [BlockingClient network thread for dnsseed.bitcoin.dashjr.org/98.209.219.85:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bitcoin.dashjr.org/98.209.219.85:8333: SOCKS: TTL expired 
Jul-22 08:45:05.218 [BlockingClient network thread for dnsseed.bitcoin.dashjr.org/98.209.219.85:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bitcoin.dashjr.org/98.209.219.85:8333: SOCKS: TTL expired 
Jul-22 08:45:05.218 [BlockingClient network thread for dnsseed.bitcoin.dashjr.org/98.209.219.85:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bitcoin.dashjr.org/98.209.219.85:8333: SOCKS: TTL expired 
Jul-22 08:45:05.336 [BlockingClient network thread for dnsseed.bitcoin.dashjr.org/98.209.219.85:8333] ERROR o.b.n.BlockingClient: Error trying to open/read from connection: dnsseed.bitcoin.dashjr.org/98.209.219.85:8333: SOCKS: TTL expired 

I just tried to telnet over clearnet to these two hosts, and I get the same behavior. connection refused for one and indefinitely long wait for the other. But I only see the errors in bitsquare when Bitcoin over Tor is enabled, so apparently BlockingClient is logging these types of errors where the Nio client is not. You might want to change that behavior in bitcoinj to have cleaner output, eg make it a debug msg rather than error.

The troubling thing to me though is the timestamps on these log messages. The first 8 are within 1 second of eachother for the same IP. Is bitcoinj really trying to open the same connection 8 times, or is something else going on? The TTL expired message also repeats 8 times in the same second, which seems to indicate something else. I doubt this would be related to a perf issue, but might be.

[dan-da]

fyi, I just pushed a commit that cleans up the SeedPeersSocks5Dns class and should fix an intermittent null exception I was seeing in PeerAddress.

[dan-da]

update.

There were definitely perf issues related to the DNS discovery. With it enabled, I would get intermittent freezes. Not every time I ran the app, but often. With it off, I do not seem to get them.

I'm not sure that DNS discovery over Tor can be made to work well, or at least it requires substantial effort. Basically the java framework just fights DNS over proxy at every step. The way that bitcoinj does DNS discovery uses several classes and a threading architecture for best performance. I went with a much simpler approach, so maybe that is why. I still don't understand why it would cause CPU to spike though... maybe something else in another thread gets upset waiting while we block on DNS request. I'm not really a java guy, so I haven't been sure how to track down the CPU usage ( eg profiler ).

For now, I think it is best to go with just using IP seed nodes rather than DNS. Or allow user to choose DNS based, but not over Tor. ( This would be useful in case the hard-coded seed nodes become stale. )

I have been working on getting IP seeds to work via the SeedNodes class. Unfortunately it seems that all the IPs in MainNetParams are obsolete, so none accept connections. To that end, I've written a PHP utility script that queries DNS and retrieves IPs as big-endian hex integers which is the way that seed IPs are encoded into bitcoinj for some reason. With that in place, I get 11 connections quite quickly.

There is just some cleanup now, so I should have something pushed by end of day.

[ManfredKarrer]

I will disable tor support for bitcoinj by default but users can enable it in the settings view or via prog arg.
If and when we get the BitcoinJ shutdown exceptions solved and have it more tested I might change it to enabled by default. But better to see first how well it works and consider it experimentat for now.
There are also very few exit nodes, so I am not very sure about the security and reliability of using tor with bitcoinj.

Bitsquare might run own Bitcoin nodes in future both clearnet and hidden service and we might offer those to users. So that would reduce the privacy issues bdue bloom filters as well.
Though not sure about that as it would introduce centralization points (thought it would be optional and users can change between the public seed nodes, Bitsquare nodes or custom nodes). Maybe we get a group of trusted devs who will run those nodes and we can offer that as a semi-trusted alternative... But anyway other things to do first...

[dan-da]

yeah, that makes sense. At least it is there as an option for anyone that cares. If/when I can find time I will look at the shutdown issue. hopefully something simple.