Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bounty: Investigate solution how to solve blocked Tor connection in China #483

Closed
ManfredKarrer opened this issue Jun 11, 2016 · 116 comments · Fixed by #1893
Closed

Bounty: Investigate solution how to solve blocked Tor connection in China #483

ManfredKarrer opened this issue Jun 11, 2016 · 116 comments · Fixed by #1893

Comments

@ManfredKarrer
Copy link
Contributor

ManfredKarrer commented Jun 11, 2016

Bitsquare cannot be used inside China mainland because Tor is blocked by the (not so) great Firewall.

We want to set up a bounty for finding the best technical solution to overcome that problem.
It might be possible to solve by using a VPN, though it would still be a big hurdle for many users (https://forum.bitsquare.io/t/using-bitsquare-in-mainland-china). If we find a way to integrate a VPN inside Bitsquare so the user has no setup costs, that might be an interesting approach.
If VPN does not solve that issue, Tor Bridges or Tors pluggable transports might be solutions.
Please note that we use Tor Hidden Services, so Bitsquare Tor traffic is never leaving the Tor network to the clearnet (via exit nodes), so it is a different use case as usual.

@meapistol
Copy link

I add 1 BTC to this bounty. I was in fact considering this bounty myself but you beat me to it. VPN might not be necessary if Bitsquare is hosted outside China. Tor Bridges or Tor pluggable transport will most likely not work. I tried all these things without success in Wuhan. VPN is throttled back by the Chinese authorities as well.

@ManfredKarrer ManfredKarrer changed the title Investiage solution how to solve blocked Tor connection in China (Bounty) Bounty: Investiage solution how to solve blocked Tor connection in China (Bounty) Jun 11, 2016
@ManfredKarrer
Copy link
Contributor Author

Got first input from a friend that it should work by extending the config file with Bridges. If that does not work then pluggable Transports should work but that needs inclusion of extra binaries.

@agardener
Copy link

@ManfredKarrer I have been in the process of testing modifications of the torr config as well as testing over hyperboria for most of the day. What is required to take this bounty?

@ManfredKarrer
Copy link
Contributor Author

@agardener: A solution which works for mainland China. We need some testers there to verify that it works.

@ManfredKarrer ManfredKarrer changed the title Bounty: Investiage solution how to solve blocked Tor connection in China (Bounty) Bounty: Investiage solution how to solve blocked Tor connection in China Jun 12, 2016
@ManfredKarrer ManfredKarrer changed the title Bounty: Investiage solution how to solve blocked Tor connection in China Bounty: Investigate solution how to solve blocked Tor connection in China Jun 12, 2016
@agardener
Copy link

agardener commented Jun 12, 2016

If someone in China wants to test bridges in Bitsquare you can use these steps:

  • run Bitsquare (Bitsquare should be running during this entire process)

  • Goto https://bridges.torproject.org and get some fresh bridges

  • choose 'none' on pluggable transports option (for now)

  • open ~/.local/share/Bitsquare/mainnet/tor/torrc in a text edditor and add the bridges that you got from bridges.torproject.org at the bottom of the file, it should look something like:

    AvoidDiskWrites 1
    ControlPort auto
    CookieAuthentication 1
    CookieAuthFile /home/bob/.local/share/Bitsquare/mainnet/tor/.tor/control_auth_cookie
    DataDirectory /home/bob/.local/share/Bitsquare/mainnet/tor
    GeoIPFile geoip
    GeoIPv6File geoip6
    HiddenServiceDir /home/bob/.local/share/Bitsquare/mainnet/tor/hiddenservice
    HiddenServicePort 9999 127.0.0.1:57169
    PidFile pid
    RunAsDaemon 1
    SafeSocks 1
    SocksPort auto

    UseBridges 1
    bridge 63.142.157.162:9001 9C27FABC13D868F94F7377D17C22998683AFCCAD
    bridge 107.23.146.44:8443 08D75007901C33900733E2D0CF51B6F196DA2824
    bridge 185.22.52.223:443 BA3AACD41D8F7EE69350CC3769BC0C9730049AEB

  • Now we need to restart tor, find the PID by running something like ps -aux|grep torrc or top. You will see a line that looks like (the PID below is 1361):

    bob 1361 2.0 3.7 20384 19016 ? S Jun11 0:35 /home/bob/.local/share/Bitsquare/mainnet/tor/tor -f /home/bob/.local/share/Bitsquare/mainnet/tor/torrc __OwningControllerProcess 1319

  • Now we send the SIGHUP signal to Tor in order to reload the config

    kill -SIGHUP 1361

  • In order to see if a connection has been made run netstat using one of your bridge IPs

    netstat -an|grep 63.142.157.162

  • or view all of your tcp connections

    netstat -an|grep tcp

  • Check status of BitSquare window

This is a quick way to get up and testing Bitsquare with bridges, I am happy to provide more support to any users willing to test. Also if we discover that basic bridges do not function I can provide additional configuration information for pluggable transports, but as ManfredKarrer pointed out it will be as simple as including additional binaries. It is not currently state aware as it appears tor is creating a new config every time Bitsquare restarts - I believe this could be fixed by creating a patch that reads torrc and runs controlConnection.setConf.

I can also help build/document other layers(like cjdns or vpn) if needed however if bridges and transports are not working in China any longer I suspect most public(free) networks will also be caught in the same web.

@ManfredKarrer
Copy link
Contributor Author

@agardener Thanks a lot!

I think I will do the following to make it as easy for users as possible:
Bitsquare starts up. If Tor connection does not work a screen gets displayed to ask user if he wants to use Bridges. Unfortunately the Brigdes webpage use terrible captchas, maybe i find a way to provide the user directly current addresses, otherwise the user has to do that manually.
After the Bridge addresses are added, Bitsquare restarts and use those addresses in the torrc at the next start. Alternatively a internal restart of tor might be even better.
Do you have an idea how to get the bridge addresses automatically (avodi user to enter captcha and go to webpage but load them form bitsquare)? Probably not possible as it is designed to avoid that, so they cannot get blocked easily....

@ManfredKarrer
Copy link
Contributor Author

ManfredKarrer commented Jun 12, 2016

@agardener Do you have any technical contact inside China who might be available for testing it?

@MarcSte
Copy link

MarcSte commented Jun 12, 2016

@ManfredKarrer oakpacific?

@agardener
Copy link

agardener commented Jun 12, 2016

No problem! Open communications ftw!

@ManfredKarrer I think you have the right idea with allowing the user to set their bridges or alternative connection methods at startup. Socks5 proxies might be another helpful option for example (a user may wish to hide their tor traffic) - this is also defined in the torrc.

Do you have an idea how to get the bridge addresses automatically (avoid user to enter captcha and go to webpage but load them form Bitsquare)? Probably not possible as it is designed to avoid that, so they cannot get blocked easily....

Its intentionally designed to not be easily scraped. I also think collecting a list of bridges and hardcoding them would be counter productive, unless perhaps they were private bridges run by the Bitsquare team as a regular bridge might be stable today but 6 months down the road some of the users could be left in the dark . Having a solution where the user sets their own bridges and transports may provide more stability/decentralization.

I don't know anyone off hand. If no one can test I can try and see if there is a VPS available... not sure how easy that is to find though I have seen availability in hong-kong.

@ManfredKarrer
Copy link
Contributor Author

@agardener To run our own bridges might be an interesting option indeed. I could ping them and if they are reachable will use those, otherwise ask the user. I want to make it as easy as possible for users (they are lazy...). Users who want/are willing to run their own bridges are even better, but I assume not many will do that, though I could offer that option as well (plan that also for a private full bitcoin node to connect to instead of seednodes).
I am not sure how quickly I can work on that, have a long list of other stuff, but hope the next 1-2 weeks.

Would you be available to help to set up a bridge?

@meapistol
Copy link

meapistol commented Jun 12, 2016

Having been in China and seen the efficiency of blocking tor also when bridges and pluggable transport is used I have my doubts that any method involving tor inside China will work. Hong Kong is very different and I guess tor is allowed there. A few years ago bridges were supplied to trusted people in China by e-mail who then distributed the adresses. Hopefully tor has been able to solve these issues lately, but China has more than 30 000 people employed to harden their Firewall so it will not be easy. I can mention that I did manage to get Bitsquare to start and find seed nodes over VPN, very very slowly, when I was in Wuhan this March so Chinas deep packet inspection is not perfect. I never got it to run fully though. I don't want to be negative here and I think you should try since this issue is so important. It will be necessary to have a skilled tester in the mainland.

@agardener
Copy link

agardener commented Jun 12, 2016

@ManfredKarrer Definitely. I could write the server configuration into some basic anisble playbooks. Not now, but later this week

@meapistol What do you think about private bridges? Think they are likely stand more of a chance?

@meapistol
Copy link

@agardener Maybe private bridges work. Maybe tor can run over "high quality" VPN. I used 12VPN. I also tried www.vpngate.net which is dedicated to China but their servers did not work at all. They did work a year ago though. Somebody from tor should be able to answer these questions. Unfortunately all my friends in China are not good at this so I cannot ask them. In fact most Chinese are satisfied with their local alternatives, Baidu, Wechat etc.

@ManfredKarrer
Copy link
Contributor Author

I think to use private bridges might be a good start. As long Bitsquare is not relevant in China we will probably be under the radar. Later we can improve. Multipe bridges and a more dynamic address allocation might be the next step. More input from Tor devs would be definitely helpful.

@agardener
Copy link

@meapistol I think before VPN I would suggest using something like the hyperboria network, setup with internal bridges to the Tor network. This mesh format creates more decentralization. But it maybe hard to integrate into Bitsquare.

@ManfredKarrer I think that relevancy is the biggest thing to note. If Bitsquare becomes 'noticed' I believe it will have to be just as agile as Tor in allowing users to set their connections.

@ManfredKarrer @meapistol I have been involved generally with Tor/mesh R+D and QA for around 10 years. While Im not a programmer per say I may be able to answer questions, or at least research an answer.

@ManfredKarrer
Copy link
Contributor Author

@agardener I was assuming you have deep background with Tor/networking stuff :-) Great to have you here!
I have no experience with hyperboria, but as you said that might be an extra challenge to integrate into Bitsquare. Usability (no setup effort for users) is a must-have for Bitsquare if there are any alternative solutions.
And yes I don't assume Bitsquare will get too soon into their radar, so we will have time. Also Bitsquare is not that directly political as other Tor usage, so even when they discover it they might tolerate it.

So my plan for the first version:
Setup a private bridge (maybe @ivilata can help here), use that hardcoded for torrc in case normal tor does not succeed.
Find someone in mainland China to test.
If that succeeds the bounty is fulfilled for the first goal.

Later we can improve.

@agardener
Copy link

agardener commented Jun 12, 2016

@ManfredKarrer Thanks! I don't mind setting up and running a VPS for a bridge if the costs could be covered(I could share login credentials somewhere for someone to foot the bill). Alternatively if someone can provide ssh access I can start there. Something to note, as I have not done a ton of development in relation to the not-so-GFW it maybe true that it has a live black list. So if our initial bridges are noticed and blocked this may mean we need to move to a new IP before testing pluggable transports. I don't want to loose the bounty, but the sun is shining here and Im afraid most people are probably sleeping in china.

How can @ivilata help?

@agardener
Copy link

agardener commented Jun 13, 2016

To answer HostFat's question obfs4 can be tested in much the same way as regular bridges.

I used the information located at the obsf4 github, I tested on debian 8

  • First install go for debian do

aptitude install golang

AFAIK everything else should not be debian specific

  • Now create the $GOPATH

    export GOPATH=$HOME/go
    export PATH=$PATH:$GOROOT/bin:$GOPATH/bin

  • Then run the go get command to install obsf4, this also install the needed depends

    go get git.torproject.org/pluggable-transports/obfs4.git/obfs4proxy

  • After a minute the binary should be ready to use, copy obfs4proxy from ~/go/bin to /usr/local/bin

    cp /home/bob/go/bin/obfs4proxy /usr/local/bin/.

  • After this the setup is much the same as normal bridges as posted above, your torrc should look something like (make sure to select obsf4 bridges on bridges.torproject.org):

    AvoidDiskWrites 1
    ControlPort auto
    CookieAuthentication 1
    CookieAuthFile /home/bob/.local/share/Bitsquare/mainnet/tor/.tor/control_auth_cookie
    DataDirectory /home/bob/.local/share/Bitsquare/mainnet/tor
    GeoIPFile geoip
    GeoIPv6File geoip6
    HiddenServiceDir /home/bob/.local/share/Bitsquare/mainnet/tor/hiddenservice
    HiddenServicePort 9999 127.0.0.1:57169
    PidFile pid
    RunAsDaemon 1
    SafeSocks 1
    SocksPort auto

    ClientTransportPlugin obfs4 exec /usr/local/bin/obfs4proxy
    UseBridges 1

    bridge obfs4 195.132.208.182:42679 69C... cert=pGG... iat-mode=0
    bridge obfs4 195.132.208.105:43666 108E... cert=EXz8... iat-mode=0
    bridge obfs4 212.218.216.127:10575 D2A.... cert=fVvl... iat-mode=0

  • Now we need to restart tor, find the PID by running something like ps -aux|grep torrc or top. You will see a line that looks like (the PID below is 1361):

    bob 1361 2.0 3.7 20384 19016 ? S Jun11 0:35 /home/bob/.local/share/Bitsquare/mainnet/tor/tor -f /home/bob/.local/share/Bitsquare/mainnet/tor/torrc __OwningControllerProcess 1319

  • Now we send the SIGHUP signal to Tor in order to reload the config

    kill -SIGHUP 1361

  • In order to see if a connection has been made run netstat using one of your bridge IPs

    netstat -an|grep 195.132.208.182

  • or view all of your tcp connections

    netstat -an|grep tcp

  • Check status of BitSquare window

@ManfredKarrer
Copy link
Contributor Author

@agardener Adding obfs4proxy would require that we add that binary, so that would be more effort. Lets start first with the easiest solution and see if we succeed.

I talked to @ivilata (the dev who joined to work on the Bitsquare network module, he is more experienced with sys admin and network stuff than me...) and he agreed to handle the bridge installation with you. I leave it up to you how to proceed. I assume the VPS costs are < 20 EUR for the period we need it for testing, so no worry I will cover that. After successful test I will run my own bridges (with @ivilata's help).

I need to get a few other things done first, but then I will try to build a version with bridge support. Hope in about 1 week I have that. I might start with some stable bridge addresses and later use our private bridge.

What are the system requirements for such a VPS? I assume they are very low (1GB ram). Do u have a recommendation for a hosting?

@agardener
Copy link

@ManfredKarrer OK, no problem.

Sounds good, I would suggest going with a host that provides per minute pricing and multiple regions, instead of getting a single VPS for the month. This way the account balance can be loaded up and if we want to switch hosting region or re-deploy for a new IP there wont be any issues. @ivilata let me know where to start.

OK, in app support will be cool.

Requirements are generally bound by RAM, which is tied to transfer/users/etc. For testing I think all of this usage will be really low and being a private bridge helps. I think we would be safe with starting in the low range(~512mb) working our way up or adding other nodes as required to do so.

I don't have a specific host in mind but there are a few hosts here that provide per minute hosting and with multiple regions: http://www.cryto.net/~joepie91/bitcoinvps.html .

Any testers in China yet?

@ManfredKarrer
Copy link
Contributor Author

@agardener Cool!
No I have no testers yet, but at least 2 or 3 contacts who I will ask as soon we have a binary ready for testing. I will release a new Bitsquare version soon and maybe I can directly add it to that version already.

@ManfredKarrer
Copy link
Contributor Author

@agardener Is it ok for you that you setup your own VPS and hand over the setup scripts/configuration after testing succeeded so @ivilata can install it on my VPS. If you want I can pay you in advance the estimated costs for VPS, lets say for 1 month.

@agardener
Copy link

@ManfredKarrer Sure.

You can use this address 19q8sKjicr4SKzB5FbK85HiqqJoRWvEUiS. What ever you send I will forward to the hosting services, I think it will cost about 0.0057 per hour.

@ManfredKarrer
Copy link
Contributor Author

It was part of the master branch until 0.4.9 I guess. But it did not help so we removed it. It needs Meek/Obfusc4 but we have not start working on that. If you are experienced in that get in touch.

@ABISprotocol
Copy link

This is the thing I gather? @ahf

@CR0CKER
Copy link

CR0CKER commented May 26, 2017

The only reliable way to connect to Tor without VPN from China is the meek-amazon pluggable transport. This has been the case for a long time. Bridges don't work for longer than a couple weeks before they are blocked, and are difficult for the average user to obtain without VPN. Happy to test if someone integrates meek-amazon and meek-azure into Bitsquare.

@ManfredKarrer
Copy link
Contributor Author

@CR0CKER: Thanks a lot for the info and your offer to help testing. As soon we have resources to work on it we will come back to you.

@ghost
Copy link

ghost commented Jul 31, 2017

New to Bisq by way of Dash. I'm based in Mainland China with software testing experience, willing to help testing new builds to get Bisq working in China. I agree that domain fronting using the meek pluggable transport is the most promising approach here.

@meapistol
Copy link

meapistol commented Jul 31, 2017

Just some information. I got Bisq to work in Wuhan, China, using 12VPN (https://12vpn.net/) first and then starting another VPN to my job inside 12VPN followed by Bisq. It worked for at least a week until I left China. Using one VPN did not work.

@hanzac
Copy link

hanzac commented Sep 18, 2017

I've been able to connect to the network by changing the below files:
C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\torrc:
ControlPort auto
CookieAuthentication 1
DisableNetwork 1
AvoidDiskWrites 1
PidFile pid
RunAsDaemon 1
SafeSocks 0
SOCKSPort 9050
Socks5Proxy 127.0.0.1:1080

C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\socks.properties:
port = 9050
proxy = 127.0.0.1:9050

The main idea here is using a socks5 or http or https proxy provided by VPN or ShadowSocks. I'm running on the Windows.
Hope these configuration can be moved to outside or there will be some UI or documentation.

@ManfredKarrer
Copy link
Contributor Author

@hanzac: Why you could not connect by default? Is Tor blocked in your country? If not maybe a firewall or blocking ISP was the reason?

@hanzac
Copy link

hanzac commented Sep 19, 2017

@ManfredKarrer Yes. Tor is blocked and I need to use proxy to connect to Tor network.

@kdaye
Copy link

kdaye commented Sep 23, 2017

Thanks @hanzac
I have add Socks5Proxy 127.0.0.1:1080 to C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\torrc:,and it is working with shaodowsocksR.
can not connect tor network resolved .

@ManfredKarrer
My country blocked Tor network ,but Tor bridge obfs4 is working.
and the bisq not working default mode.

@taylorshuang
Copy link

@hanzac, Tor is blocked in my country, i follow kdaye's method , just add Socks5Proxy 127.0.0.1:1080 to C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\torrc, and i can't connect to Tor network . Do I need to modify SOCKSPort 9050 in C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\torrc and C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\socks.properties:?

@ManfredKarrer
Copy link
Contributor Author

If anyone wants to work on pluggable transports integration for Bisq, here is the lib from @JesusMcCloud : https://github.com/JesusMcCloud/netlayer/
That would need to be integrated in bisq and replace the current tor libs. And would need a lot of testing of course...

@seanlis
Copy link

seanlis commented Sep 27, 2017

FYI.
I could use bisq in China with a VPN set up on my US based VPS.

@taylorshuang
Copy link

@seanlis, Hi, dude, can you tell me how to connect Bisq use US vps. I have a virtua machine on amazon cloud, and use shodowsocks as a VPS, but i can't to connect Bisq in China.

@taylorshuang
Copy link

I have solved my problem with a VPN , thanks!

@kdaye
Copy link

kdaye commented Sep 28, 2017

@taylorshuang you forgot open the local port(1080) for Bisq ,yep?_

@hanzac
Copy link

hanzac commented Sep 28, 2017

@taylorshuang I modified C:\Users\NT_USER_NAME\AppData\Local\Bisq\app\Bisq.jar\socks.properties, because I make tor to serve at 9050 port.

@taylorshuang
Copy link

@hanzac, OK, Thanks. It is good idea to directly buy a VPN and no need modify any.

@raidsan
Copy link

raidsan commented Jun 27, 2018

each time I launch bisq-desktop.bat, the following file will be re-create
C:\Users\username\AppData\Roaming\Bisq\btc_mainnet\tor\torrc
C:\Documents and Settings\username\AppData\Roaming\Bisq\btc_mainnet\tor\torrc

It prevent I add proxy setting in the torrc file, how to make it re-use exist torrc file and not refresh it?

@ManfredKarrer
Copy link
Contributor Author

@raidsan It is not supported yet but planned.
See JesusMcCloud/netlayer#7

@JesusMcCloud
Copy link

@raidsan @ManfredKarrer torrc overrides are supported, using an existing tor is also (basically) working (see JesusMcCloud/netlayer#7).
If you just need to add something to your torrc, you can user torrc overrides are the way to go (though I'm not sure how Bisq handles it).
If you want to use an external tor instance, check JesusMcCloud/netlayer#7

@estoniah
Copy link

bisq on Linux:
I can't connect bisq network from china, Tor bundle work well through meek-amazon,
But meek-amazon of bisq can't work?

@joncamfield
Copy link

I came across this thread in a random search, you might be interested in the various options of using different transports (and some self-managed bridge options) over at https://www.pluggabletransports.info/

@ManfredKarrer
Copy link
Contributor Author

We support pluggable transports. You can enable it either in the settings/network or when the app cannot connect for 4 min. it will show up a popup.

@msq6323013
Copy link

msq6323013 commented Sep 24, 2019

MacOS:
cp Bisq-x-x-x-.jar /yourDirectory
jar xvf Bisq-x-xx-.jar
mv torrc
// add Socks5Proxy 127.0.0.1:1080
jar uf Bisq-x-xx-.jar torrc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.