Signal's TLS Proxy Failed to be Probing Resistant and seems leaky

[DuckSoft]

Links

• Signal Post: https://signal.org/blog/help-iran-reconnect/
• Their Repository: https://github.com/signalapp/Signal-TLS-Proxy
• Our Original Issue: https://github.com/signalapp/Signal-TLS-Proxy/issues/3

Why Here?

⚠️ emotional. don't read.

So I've studied this with @studentmain and found it problematic about 4 hours ago.
We immediately reported this through a GitHub Issue, with PoC and advice attached, without even sleeping (it's about 4:00 am in local time)
But @moxie-signal (@moxie0) from @signalapp just closed our issue, saying this:

Hey everyone! Thanks for the interest in this. We normally don't use GH issues for this type of discussion, though, and prefer to have that happen over on the Signal community forum. Please check out https://community.signalusers.org/ if you want to get involved. Thanks!

It seemed that they just don't care about probing resistant and simply shut people's mouths up instead of fix this issue.

And guess what's next? THEY THEN JUST CLOSED THE WHOLE ISSUE FUNCTION!

THEY CLAIMED TO HELP PEOPLE IN CENSORSHIP, BUT THEY IN TURN CENSOR WHISTLEBLOWERS.

And here's what has happened if I go to "Signal community forum" to post the issue:

Okay, fine. You guys have really an awesome security!

Some updates: They even banned me from their GitHub organisation! What an honour!

---

Extra notes for those guys who think I was aggressive and not considerate enough:

You guys are correct. I ain't saint. I got my emotions and temper.
The weapon of criticism obviously cannot replace the criticism of weapons. Material force must be overthrown by material force. But theory also becomes a material force once it has gripped the masses.
So long as they were ignoring the weapon of criticism, I think it fair to put some criticism of weapons -- Or I can completely ignore everything: Don't post it at all, and grab some popcorn to watch -- Just wild guessing, Iranian being caught using Signal and sent to jail?

---

Updates:

So that's why I'll post it here.

Original Issue

Issue Body

I am not a specialist on Censorship Circumvention, so if I'm wrong, I'd appreciate if I could be corrected.

I've quickly scanned the code, so this is just a TLS tunnel set up to forward inner TLS traffic with signal SNIs as-is.
But what if the censor actively probe the suspected proxies? For example, connect and then use real signal app traffic and non-signal app traffic? The difference can be detected. And if a proxy can be detected, it can be blocked.

Thanks my friend @studentmain for the PoC.

PoC from @studentmain

PoC Source Code:

package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"os"
	"time"
)

func send(addr, server, sni string) int {
	c0, e := net.Dial("tcp", addr)
	if e != nil {
		log.Fatal(e)
	}

	c1 := tls.Client(c0, &tls.Config{
		ServerName:         server,
		InsecureSkipVerify: true,
	})

	c2 := tls.Client(c1, &tls.Config{
		ServerName:         sni,
		InsecureSkipVerify: true,
	})
	c2.SetDeadline(time.Now().Add(2 * time.Minute))
	s := fmt.Sprintf("GET / HTTP/1.1\r\nHost: %s\r\nUser-Agent: curl/7.68.0\r\n\r\n", sni)
	//b := make([]byte, 4096)
	l, _ := c2.Write([]byte(s))
	log.Println(l)
	if e != nil {
		return 0
	}
	log.Printf("%s->%s->%s\n", addr, server, sni)
	return l
}
func main() {
	if len(os.Args) != 3 {
		log.Fatalln("usage: main.exe server_name addr_port")
	}
	server := os.Args[1]
	addr := os.Args[2]
	l1 := send(addr, server, "updates.signal.org")
	l2 := send(addr, server, "telegram.org")
	if l1 != 0 && l2 == 0 {
		log.Fatalln("It is a Signal TLS Proxy")
	}
	log.Println("It is not a Signal TLS Proxy")
}

PoC Example Run from @studentmain

Here is an example run of the PoC from @studentmain:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

C:\src\signalTlsPoc>main.exe
2021/02/05 03:40:15 usage: main.exe server_name addr_port

C:\src\signalTlsPoc>main.exe your.proxy.server.example.com your.server.ip.address:443
2021/02/05 03:40:34 69
2021/02/05 03:40:34 your.server.ip.address:443->your.proxy.server.example.com->updates.signal.org
2021/02/05 03:40:35 0
2021/02/05 03:40:35 your.server.ip.address:443->your.proxy.server.example.com->telegram.org
2021/02/05 03:40:35 It is a Signal TLS Proxy

C:\src\signalTlsPoc>main.exe baidu.com 39.156.69.79:443
2021/02/05 03:43:09 0
2021/02/05 03:43:09 39.156.69.79:443->baidu.com->updates.signal.org
2021/02/05 03:43:10 0
2021/02/05 03:43:10 39.156.69.79:443->baidu.com->telegram.org
2021/02/05 03:43:10 It is not a Signal TLS Proxy

C:\src\signalTlsPoc>

(modified to protect my personal information)
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEdbyjO018JNZhjd2SqnhRnCCMh0IFAmAcT0AACgkQqnhRnCCM
h0Lx6RAAlx6C6R7tZROlj8I+1uu2Owde9EjOA3BTYw7aC8pr4cd2LuTkmAY07TuZ
mbQRueUIKOXx2ZlTPRL1yOAW60nePC5SRmlOkNssafWzM+8zyqfHdRe64OX0ZA5V
KPgSVHOz2JKfh650oEqlaHb/6N/Nj++617TcHmmYoy/xsiDy/fD7IoifD8bT5N3+
4znZ2Sr2+IirKxqvpVDl5AwRBR5EfXUI54c0OjKB8Sq+VPlhAmAcj/eZgSz7A4rV
EWSJjwbN1fp7g2C4jNtjh0Z3jyvvA9yGNThylcMPbXcrc1a9sOzhbCgGTtF1413n
tXSjy3vRBEnwvlVs78ItQljCZV1S+jwpXlkjFPGAIYyRRQbvaQ4Biaki/4g3ofVu
nfqEnc5m4vSsde2r5GLBqxSkbsKuZjGBbd1KRE+5LZPvkVrA5wP400tirThnvNU9
G7S5Noz/8Npwazag0beDlio45e8sBafTBRJy/xuq6HVGdFkPa1C7EUMuDxVVnPdg
mOgyqrjXju+4y/pl39U6JmDt6/ZSn34/1EFVchRWS5Kq0gPDGdjvrjI88aOtqhRH
tT8F6dU8rwRa8tjMHv3gNVTQY+jQZIi4SHNB54DRc1K0AKzIkAOgD/ROhcU4LkXq
vudKRTfhbtL6l8FIo8WCzGVONK1dY+1alKXKc8cVJNrye/hm5t4=
=H1r/
-----END PGP SIGNATURE-----

Suggestion 1 from @DuckSoft

One crude idea is to use preshared tokens and fallback mechanism.

In this case, we can hide a preshared key within SNI. For example, we can preshare a key 538fd09a066a, and then mix this with original SNIs, where cdn.signal.org could become 538fd09a066a.cdn.signal.org. We then only accept whitelisted SNIs mixed with our preshared key, and deny all other SNIs including the original one cdn.signal.org. This way without knowing the preshared key, the censor is unable to active probe any longer. Since the SNI is actually sent inside the TLS tunnel after establishing a TLS connection with the proxy (TLS over TLS), the inside SNI is virtually invisible to the censors (unless root certificates got hijacked, etc.).(Edited: this is only feasible when those domains are transparent about SNIs and hosts, thus not operatable) And, since the tunnel is over TLS, you can virtually hide the preshared key anywhere inside, even making this as a customized TLS ClientHello Extension, or just masquerade as a plain HTTP request but with special headers, etc.

It is also possible to utilize WebSocket over TLS on the outside, so that unless the exact WebSocket endpoint path is known (which also works as a preshared key), the proxy cannot be probed by the censors without knowing the path - maybe they can only see a sad 404 page or your blog index page. But one thing must be clear: From the ClientHello ALPN extension, censors can deduce that you are using WebSocket techniques, since your ALPN cannot be h2/http/1.1 in this case, and WebSocket don't work when there's a h2 connection. You can turn off the h2 support on the server side and proceed in this situation, but normal browsers don't tend to connect WebSocket just all the time and they will prefer the use of h2 unless clearly denied.

For the fallback mechanism, when the traffic is found invalid, we should act the same as those innocent servers. When innocent servers is sent with TLS handshake inside TLS tunnel, the server tends to disconnect with bad request response. Then we should act the same. And if a valid HTTP request is received, we can then fallback to an innocent website, such as blog, minigames or something. This will significantly increase the difficulty for the censors to block.

Suggestion 2 from @DuckSoft

Also there's a weak temporal feature in this project, where the certificates tend to be newly signed. If possible, relay hosts should try to reuse existing certificates rather than brand new ones, which can be far more suspicious to the censors.

Suggestion 3 from @DuckSoft

Besides, for large portions of TLS traffic, there would be an ALPN in the Client Hello frame. If this proxy runs TLS over TLS, then the ALPN would possibly not be commonly seen ones like h2 and http/1.1. Instead, the application tend not to send this extension, which creates a passive feature of this proxy.

But what if we pretend to be h2/http/1.1? Well, you can actually do it, though you are breaking the rules. I am not sure about the exact behavior, but this could create problems when you want to be compatible with existing web servers.

Suggestion 4 from @DuckSoft

By the way, DPI (Deep Packet Inspection) techniques can detect TLS in TLS traffics easily (there are lots of papers available, if you do a search). Although not 100% TLS in TLS traffic is our lovely proxy, you can still risk being caught using a proxy when censorship is there.

DPI might be expensive, but code like @studentmain is not. The current status of probing resistance is really a problem.

Archive

⚠️_Failure_to_be_Probing_Resistant_with_PoC_·Issue#3_·_signalapp.zip

What Leaky?

https://community.signalusers.org/t/traffic-not-routed-to-tls-proxies-can-expose-users-to-censors/27479

[wkrp]

Thanks for making this post. To me it looks like your analysis is correct: a countermeasure to this kind of proxy is to watch for outgoing TLS connections and then connect to the same TLS servers, sending a tunneled TLS stream for one of the signal.org domains. The proof-of-concept prober works for me, using example.com and a proxy I found posted online. I know it's frustrating to have your bug reports deleted, but I would ask let's please try to be considerate and constructive. We are all on the same side.

A TLS-in-TLS tunnel with restricted endpoints is not a bad design for circumvention, in my opinion. It hides the endpoint address while protecting the client's end-to-end security. You are correct, though, that to prevent active probing, the proxy must require the client to prove knowledge of some secret. A simple way to do it is to have the client send a password or token when it connects, like NaïveProxy does with probe-resistant HTTP authentication, HTTPT does with a URL path, and Rosen does with an HTTP header. The key idea is that there must be more information required to use the proxy than just the IP address / domain name, because those pieces of information are visible to a censor.

I had a look at the Signal-TLS-Proxy repo, though, and it's not a custom piece of software: it's a Docker configuration that installs two instances of Nginx, an internal instance that handles the forwarding to signal.org servers, and an external instance that terminates the outer layer of TLS and forwards to the internal instance. It's actually a pretty clever way of quickly constructing a proxy that's easy to install. But it means that one cannot just hack in a custom client authentication scheme. I am not too familiar with Nginx configuration files, but @DuckSoft's idea of a secret subdomain prefix may be workable, if there's a way to strip the subdomains before forwarding. (Edit: I realized later that the idea of stripping subdomains from SNI is nonsense, as end-to-end integrity checks would prevent it. I was concerned about the secret serving as a persistent and externally visible user identifier, but as long as all users of a proxy use the same secret, it's not any more identifying than the proxy IP address.) Something like this would increase the complexity of deployment, though, as every proxy would need to have a unique nginx-relay/nginx.conf file, and there would need to be a way to share the secret as part of a signal.tube or sgnl:// link.

I think HTTPT's faux WebSocket forwarding design is clever. It enables the external TLS layer to be HTTPS, which provides places to communicate a secret. The payload does not actually need to be WebSocket—the external HTTPS server just forwards the contents without actually trying to interpret them as the WebSocket protocol. But it still requires a custom server program to absorb the forwarded HTTP headers and then proxy the remainder of the connection. It's definitely not as simple as an Nginx proxy.

[proletarius101]

If I didn't make it wrong, we are just trying to reinvent https://www.v2ray.com/. Maybe a custom variant of v2ray is a solution, as we are using docker. It even has built-in sni sniffing.

[sergeyfrolov]

Back when we were working on The use of TLS in Censorship Circumvention paper, we found some issues in TLS ClientHello messages, generated by Signal client. You can find more info in the paper.
We sent the signal team a couple e-mails, privately informing them about our findings. After getting no response in a couple months, we submitted a github issue. Here it is: https://github.com/signalapp/Signal-Android/issues/7337

So, I understand how you feel about the issue, and it is unreasonable that Signal suspended your forum account after telling you to create a post there, but I must agree with @wkrp reminder about civility: we are on the same side. Calling their approach stupid on their forum and naming PoC github repo "fuck-signal-tls-proxy" doesn't help either. If Signal does bury their heads into the sand and does not address the issue, let's just grab some popcorn and see how long it takes them to get blocked.

[DuckSoft]

@sergeyfrolov Well, I got to clarify something first.

1. The repo is composed and named by @studentmain, where I respect the freedom of his naming his own project;
2. I am not the very author who've put "stupid" in their forum, and back then my account was suspended. You will see the post was by Teleposter rather than DuckSoft.

Anyway, let's see what will be on. Thanks for your reply.

[wkrp]

I like to try to set an encouraging tone. It is, of course, fair play to point out security vulnerabilities, but let's also give credit where it is due. The blog post says Signal deployed this new system in about 10 days, which is a remarkably fast response. It calls the new simple TLS proxy an "interim solution" and says "we're also continuing to investigate other techniques that are more automated and convenient." So far as that's true, the simple TLS proxy may be good enough, for now, to deal with the blocking in Iran. If the censors there do not already have an active probing system set up (and I don't know of any evidence that they do), they won't be able to create one instantly. (It would be a different story in China, of course, because the GFW's existing active probing of Tor and Shadowsocks, among other protocols, could probably easily be adapted for Signal proxies.)

@DuckSoft and @studentmain also deserve credit for analyzing the proxy and finding the flaw so quickly. Not everyone has the knowledge and skill to do something like that. I value your participation on this forum, and your help in joining together different groups of circumvention developers. People like us are accustomed to recognizing common pitfalls such as susceptibility to active probing. We gained that knowledge through experience, and it's the same for any other developer. Let's continue to look for flaws while also encouraging and supporting our colleagues who have good intentions.

[ghost]

they won't be able to create one instantly

Time is money. At the moment, our only chance (not only for Signal, I mean everyone) is develop and deploy new solution before they deploy new system. Luckily, small teams can response way faster than a huge organization. If it works now, then it works now, just use it. Then let's build something which works in the darker future. And knowledge is power. If they know the sad stories in this forum at the beginning, they can just learn from it, avoid these problem without try and error.

[ghost]

Perdí la confianza en esta aplicación, lista negra +1

---

Translation added by @wkrp:
I lost confidence in this application, blacklist +1

[luckydonald]

To bad the issue was never crawled to archive.org in the original repository.
That makes it sadly difficult to verify the statement of actually making them aware.

[DuckSoft]

To bad the issue was never crawled to archive.org in the original repository.
That makes it sadly difficult to verify the statement of actually making them aware.

Also you can go to GitHub GraphQL Explorer and perform the following query, where you can see the original post and responses:

query { 
  repository(
    name:"Signal-TLS-Proxy", 
    owner:"signalapp"
  ) {
    issue(number: 3) {
     	author {
        login
      }
      body
      comments(first: 9) {
        nodes {
          author {
            login
          }
          body
        }
      }
    }
  }
}

[exabrial]

There are so many better ways to make important points without resorting to personal attacks, ultimatums, or veiled threats. I think that should be talked about.

Take this sentence:

It seemed that they just don't care about probing resistant [sic] and simply shut people's mouths up instead of fix this issue.

Even if it were on purpose, why not consider a de-escalation first; why go on the offensive? Undoubtedly if you are passionate about something, eventually you will become frustrated. It's part of growth and progress, but part of that two is not giving in to feelings of frustration, anger, or rejection.

I think there are two things that could be applied here that resulted in the missed high five.

• First: Seek first to understand, then to be understood. Essentially, seek to understand why they closed the ticket, rather than reacting as if it were an attack. https://en.wikipedia.org/wiki/The_7_Habits_of_Highly_Effective_People#5_-_Seek_first_to_understand,_then_to_be_understood

• Second: Assume positive intent. When someone does something unexpected or unpredicted, assume they actually were acting with good intentions, and that it was not an attack on you. This is highly related to the first point. In the The 5 Actions of Smart Trust they talk about this and I'm always reminded of that when dealing with over-worked devs on open source projects.

I hope this all shakes out for the positive. I'm very thankful for the Signal team and the people that take their time to audit the security and submit bugs.

[benlivengood]

Don't put secrets in SNI or any other plaintext before the TLS session is established with the proxy. Anything in plaintext can be replayed by an adversary.

[wkrp]

Don't put secrets in SNI or any other plaintext before the TLS session is established with the proxy. Anything in plaintext can be replayed by an adversary.

I think you misunderstand the proposal. There are two layers of TLS in the tunnel: the outer layer that terminates at the simple TLS proxy, and the inner layer that is end-to-end between the client and signal.org servers. The proposal is to include an authentication secret with the inner layer of TLS, where it is invisible to an adversary. The outer TLS layer would remain as normal. It's a sound model, used by many existing and proposed circumvention systems, including those mentioned in #60 (comment). There are many ways to do it, but I believe in this thread we were brainstorming ways that would fit easily with the Docker setup.

[ghost]

in this thread we were brainstorming ways that would fit easily with the Docker setup.

Proxy in Docker is not something new, you know. Outline is the most famous one. I'm thinking about is there something only they can do? Like, use their already existed account system, protocol, etc, to provide authentication support, detect bad users, etc.

[petermolnar]

The more I read the more this whole setup resembles port forwarding over SSH, and via connecting TLS over the tunnel.

[wkrp]

The more I read the more this whole setup resembles port forwarding over SSH, and via connecting TLS over the tunnel.

In the sense of tunneling one protocol in another, yes. SSH is probably an inferior cover protocol (less costly to block) than TLS, though.

The naive and insecure way of constructing a TLS proxy is to have the client's TLS session terminate at the proxy, and the proxy re-encrypt the stream in a separate TLS session to the destination. The problem with that is that the proxy may inspect and tamper with the traffic between the client and the destination. Signal's simple TLS proxy avoids that pitfall with its TLS-in-TLS design. The outer layer to the proxy is for circumvention only: it hides the SNI and other destination identifiers such as the destination IP address. The inner layer, which the proxy cannot tamper with, is for end-to-end encryption and authentication with the Signal servers.

[abschluss24]

We sent the signal team a couple e-mails, privately informing them about our findings. After getting no response in a couple months, we submitted a github issue. Here it is: https://github.com/signalapp/Signal-Android/issues/7337

I just got a page not found (archive) by visiting this link. No idea what was wrong.

If the issue @sergeyfrolov opened was just deleted (archive.org keeps only one archive of that page so I cannot be sure), then it implies that the Signal team is watching this thread, without actively participating in the conversation.

[DuckSoft]

We sent the signal team a couple e-mails, privately informing them about our findings. After getting no response in a couple months, we submitted a github issue. Here it is: https://github.com/signalapp/Signal-Android/issues/7337

I just got a page not found (archive) by visiting this link. No idea what was wrong.

@abschluss24 It's surely deleted, my friend.
Signal just do ostriches and refuse to handle the problem, which is pretty sad.

[ghost]

That's true.

I can visit both
signalapp/Signal-Android#7336
and signalapp/Signal-Android#7338

Since GitHub PR and Issues have an self-increment unique ID, the only reason to the 404 https://github.com/signalapp/Signal-Android/issues/7337 would be the post deletion.

Update: See signalapp/Signal-Desktop#4513

[DuckSoft]

the Signal team is watching this thread, without actively participating in the conversation.

Maybe, yeah. I just don't understand why they don't process the problem itself, rather than "process" us whistleblowers.

[ghost]

Hi, Signal team, if you are watching this, thank you. Keep watching, and reading other issues, learn from them, then update your code, create a perfect solution for your user, you can do it.

[ghost]

as far as I know

If can’t solve the problem, then "solve" the one who raised the problem

This is the style of the Chinese government

Yeah, i got it

[ghost]

as far as I know

If can’t solve the problem, then "solve" the one who raised the problem

This is the style of the Chinese government

Yeah, i got it

Is there a possibility that this is not a bug
its just a pre-preserved backdoor?

[DuckSoft]

Is there a possibility that this is not a bug
its just a pre-preserved backdoor?

I'd try my best to resist resorting to conspiracy, but... I don't know.
I just felt sad.

[ghost]

as far as I know
If can’t solve the problem, then "solve" the one who raised the problem
This is the style of the Chinese government
Yeah, i got it

Is there a possibility that this is not a bug
its just a pre-preserved backdoor?

No conspiracy theory, please.

[SekiBetu]

There is no way for a proxy server to discriminate between "users of Signal who are government censors" and "users of Signal who aren't"

it's simple, we just need to add a key to verify.

it's like your door is locked, you copied your key and give it to your friend, you and your friend can come in, but other dude can't
and when the key is leaked, we just change the lock and get a new key before it's too late.

but currently, they don't even have a door.

and here is another interesting idea about the key that i recommend you to read:
shadowsocks/shadowsocks-org#178
shadowsocks/shadowsocks-org#178 (comment)

[wkrp]

"Probing resistant" is not some defined property of a proxy system; anything that's a public open proxy is probe-able by definition: if it works to talk to the (public) service, then it's a proxy. There is no way for a proxy server to discriminate between "users of Signal who are government censors" and "users of Signal who aren't". This means that anyone trying to use it will be able to determine immediately if it can be used to talk to Signal.

This isn't a vulnerability, this is just how an open-to-anyone proxy works. If you think about it, there's no other way it could work, as "probing" users can be assumed to have the same Signal accounts that you and I and any other potential proxy user does.

Intuition can be misleading on this topic. At first it seems impossible—how do you distinguish legitimate proxy users who have learned the proxy's address through legitimate means, from a censor who has learned the address by watching the wire? But it's not insoluble—the key idea is to authenticate the client inside the proxy protocol, as @SekiBetu and others on this thread have suggested. Probing resistance is a property that a proxy system may have or not have, and modern circumvention systems include active probing attacks in their threat model. This is not least because the Great Firewall of China has been extensively documented to use active probing for proxy discovery since as early as 2010. It is not a new or theoretical attack. I encourage you to read at least at least the introduction of "ScrambleSuit: A Polymorphic Network Protocol to Circumvent Censorship" from 2013 to gain some background.

You correctly understand how the basic active probing attack works. A client makes a connection to a secret proxy server, using whatever cover protocol—in this case TLS. The censor, who can monitor all traffic going in and out of its network, watches the connection happen and identifies the protocol (TLS) as being one that may be used to connect to secret proxies. The censor guesses what proxy protocol is in use, then sends its own probe "speaking" that protocol—as if it were an ordinary client—to the same server the client connected to. If the server responds to the probe as expected for the proxy protocol, then the censor adds its IP address to a blocklist. There are various possible refinements to the basic attack: the censor can reduce its active-probing load by prefiltering on side channels such as the TLS fingerprint, as @studentmain has demonstrated for the Signal proxy's client traffic. And the censor does not have to guess a singular proxy protocol; it may send probes of multiple protocols at once and process them in parallel.

The thing to note about the attack is that the censor is able to probe the server knowing only its IP address and port number (and whatever other information the proxy protocol exposes, such as the TLS SNI). The critical requirement of a probing-resistant proxy system, then, is that knowledge of only the proxy's network address must not be sufficient to make use of the proxy. A putative client must provide proof that it knows some secret, or else the proxy will deny it service, ideally in some inconspicuous way. (How to respond to unauthenticated clients is itself a subtle matter, but orthogonal to the basic need for client authentication.)

This is why, when you ask BridgeDB for an obfs4 bridge for Tor, it doesn't just give you an IP address and port number, it also gives you a cert parameter, which is how the proxy will identify you as an authorized proxy user. And why, when you set up a V2Ray/V2Fly server, the configuration file includes a secret UUID which clients use to authenticate. These and other proxy protocols have been designed to be probe-resistant, so they include secret information apart from the proxy's network address.

Knowledge of the secret shows that the client learned of the proxy through an intended out-of-band channel, and not by traffic sniffing. (This implies that such out-of-band channels must themselves resist easy enumeration by the censor—this is why BridgeDB shows a captcha and why Signal recommends distributing proxy address via DM rather than posting them publicly. But that is a separate problem from probing resistance.) The problem with Signal's TLS proxy is that it is easily detectable by a censor who knows nothing more than the proxy's domain name.

I'm taking the time to explain these things in detail because I know this thread is getting outside attention, and some readers who are not researchers or practitioners in censorship circumvention will appreciate the added background. I care about censorship circumvention and I want to increase the general level of understanding of it. Below, there is a list of recommended research papers on the topic of active probing. This forum's "reading group" label additionally has short summaries of other recent research papers (and the "Reading list" wiki page has a list of papers that are not yet summarized).

• "ScrambleSuit: A Polymorphic Network Protocol to Circumvent Censorship", 2013.
• "Examining How the Great Firewall Discovers Hidden Circumvention Servers", 2015.
• "Detecting Probe-resistant Proxies", 2020. (Summarized here.)
• "HTTPT: A Probe-Resistant Proxy", 2020. (Summarized here.)
• "How China Detects and Blocks Shadowsocks", 2020. (20-minute video.)

[sneak]

Yeah, but that's not how the Signal one is intended to work at all. Adding an extra layer of authentication, so that one is not only shuttling around proxy addresses but also authentication information, is quite impractical. The released code is really simple, and pretending that the lack of this misfeature (authentication) is a security bug is silly and misleading.

They were right to ban someone from their forum for being misleading in this way; it's classic FUD.

[ewust]

@sneak Adding a secret wouldn't be impractical here. Signal already uses the clever signal.tube/domain link scheme to deliver arbitrary data (the domain) to the Signal app; this could be easily changed to make the link be signal.tube/domain/secret to include an authentication secret. Users would still distribute these links through private channels as they do now, and the app learns both the domain and secret needed to authenticate to the proxy. The censor would not learn the secret, and so wouldn't be able to get it from watching network traffic alone, and would be unable to probe the proxies.

This does require a change on the server side, and Signal is within their right to say "cool, but we don't have the bandwidth to implement such a feature" or ask for a PR implementing it for them. But I don't think it's fair to say this particular issue is FUD: We know censors like China have used this kind of technique to block proxies before, so it's within the realm of possible for Iran to do so as well.

What I think may be happening here is:

1. Signal doesn't have enough engineering resources to fix "could be" problems like this, even if they are "could be very soon" problems
2. Signal would prefer not to have active vulnerabilities and PoC exploits hanging around in public places

If this is the case, both Signal and researchers could do better here:

• Researchers should use responsible disclosure (e.g. private email) to report these kinds of problems instead of trying to post them in public right away
• Signal should be more clear about the reason behind a #wontfix: is it because they don't believe it to be a problem (in which case perhaps researchers can educate if there's a wrong assumption) or is it because they don't have time to implement the fixes (in which case researchers can help make PRs/come up with low-cost solutions)?

[sneak]

This isn't an issue of "researchers" or "disclosure" as this isn't a security vulnerability. Naming something a "PoC" or an "exploit" is inaccurate and misleading; it is neither.

A proxy getting blocked is no more a problem than the existing issue: Signal's first-party servers getting blocked in Iran. The "problem" scenario is the same as if the volunteer proxy didn't exist: the proxy gets blocked too. There's no security issue here.

[DuckSoft]

No personal attack intended, this would be the worst response I could expect from @moxie0.

Twitter Link: https://twitter.com/moxie/status/1358835853867425792?s=20

To me it's more like "there is no problem at all and I won't fix it".
Well if that is the case, I could only wish people still using this "TLS Proxy" a better luck.

[SekiBetu]

@sneak

this isn't a security vulnerabilit

i see, you guys are first time fighting against things like GFW right? why not listen to people who are doing this for 10+ years, it may not be a security vulnerabilit, because you guys are good at defining things, but things like signal proxy will 100% put users to jail in the near future both in china and iran, that's what i can tell.

remember, we are not enermy, we are together here to fight against censorship.

[Herohtar]

I have been told that someone else have submitted: https://vuldb.com/?id.169364

That listing is complete nonsense. The TLS proxy isn't "firewall software", there is no "weak encryption vulnerability", and the technique used in the probing isn't T1552 (unsecured credentials).

[DuckSoft]

I have been told that someone else have submitted: https://vuldb.com/?id.169364

That listing is complete nonsense. The TLS proxy isn't "firewall software", there is no "weak encryption vulnerability", and the technique used in the probing isn't T1552 (unsecured credentials).

Yeah. I didn't submit this and I feel weird too. Just don't put downvotes at me.

[mholt]

I've been following this thread for a while, and have taken an interest in improving the current situation. I think we can easily make some substantial improvements on what Signal has initially released.

Caddy is an extensible web server that uses TLS automatically and by default, including automatically renewing certificates and stapling OCSP responses. It also uses strong cryptographic settings by default. It's commonly used with v2ray by our Chinese friends. Caddy is statically compiled and has no external dependencies like libc, Certbot, or Docker. It's a very compelling alternative to nginx for use cases like this. (Disclaimer: I am the Caddy author.)

Several months ago, I released a Layer 4 module for Caddy that works with raw TCP/UDP streams.

Last evening, I was able to successfully get a Signal proxy working that is powered by Caddy, using this config (and with the layer4 module plugged into my Caddy binary):

(Updated for Caddy 2.6)

{
	"apps": {
		"layer4": {
			"servers": {
				"signal_proxy": {
					"listen": [
						":443"
					],
					"routes": [
						{
							"handle": [
								{
									"handler": "tls"
								}
							]	
						},
						{
							"match": [
								{
									"tls": {
										"sni": [
											"chat.signal.org",
											"ud-chat.signal.org",
											"textsecure-service.whispersystems.org",
											"storage.signal.org",
											"cdn.signal.org",
											"cdn2.signal.org",
											"api.directory.signal.org",
											"contentproxy.signal.org",
											"uptime.signal.org",
											"api.backup.signal.org",
											"sfu.voip.signal.org",
											"updates.signal.org",
											"updates2.signal.org",
											"cdsi.signal.org"
										]
									}
								}
							],
							"handle": [
								{
									"handler": "proxy",
									"upstreams": [
										{
											"dial": ["{l4.tls.server_name}:443"]
										}
									]
								}
							]
						}
					]
				}
			}
		},
		"tls": {
			"certificates": {
				"automate": [
					"example.com"
				]
			}
		}
	}
}

(Apologies for GitHub's obscenely huge 8-wide tabs)

(Replace example.com with your own domain name.)

That's it, that's the whole setup.

Though not perfect -- nor a solution to the probe resistance issues discussed in this thread -- this setup is already an improvement because:

• Caddy manages certificates for you, much better than sleep() + certbot can.
• The certificate used is not necessarily new: it can be the same one used by an existing site. (See Suggestion 2 above.) This is a powerful idea I'll explore below.
• Caddy staples OCSP responses by default. That means censors can't block OCSP servers to take a service offline. Nginx does not do this without extra config, and its stapling implementation is not good. Caddy's is state-of-the-art.
• No external dependences are required: no Docker, no libc, no Certbot, no cron, etc. Just run Caddy: caddy run --config caddy.json (you will want to daemonize it, probably).

This is not a probe resistant solution. However... it's a much better start. This config can be augmented with an actual HTTPS server config so that Caddy can both serve a website at example.com and a Signal proxy at example.com -- same TLS ServerName (SNI), same IP address, etc. -- and it can reuse the existing certificate.

The config that runs an HTTP server alongside the Signal proxy is very similar; mostly, we add a http app and proxy connections that aren't Signal:

{
	"apps": {
		"http": {
			"servers": {
				"my_site": {
					"listen": ["127.0.0.1:8080"],
					"protocols": ["h1", "h2", "h2c", "h3"],
					"routes": [
						{
							"handle": [
								{
									"handler": "file_server",
									"root": "/var/www"
								}
							]
						}
					]
				}
			}
		},
		"layer4": {
			"servers": {
				"signal_proxy": {
					"listen": [":443"],
					"routes": [
						{
							"handle": [
								{
									"handler": "tls"
								}
							]
						},
						{
							"match": [
								{
									"tls": {
										"sni": [
											"chat.signal.org",
											"ud-chat.signal.org",
											"textsecure-service.whispersystems.org",
											"storage.signal.org",
											"cdn.signal.org",
											"cdn2.signal.org",
											"api.directory.signal.org",
											"contentproxy.signal.org",
											"uptime.signal.org",
											"api.backup.signal.org",
											"sfu.voip.signal.org",
											"updates.signal.org",
											"updates2.signal.org",
											"cdsi.signal.org"
										]
									}
								}
							],
							"handle": [
								{
									"handler": "proxy",
									"upstreams": [
										{
											"dial": ["{l4.tls.server_name}:443"]
										}
									]
								}
							]
						},
						{
							"handle": [
								{
									"handler": "proxy",
									"upstreams": [
										{
											"dial": ["127.0.0.1:8080"]
										}
									]
								}
							]
						}
					]
				}
			}
		},
		"tls": {
			"certificates": {
				"automate": [
					"example.com"
				]
			}
		}
	}
}

What this config does is establish 3 apps:

• The http app starts an HTTP server that serves your website, with a socket accessible only on the local machine. H2C is enabled since we'll want to allow HTTP/2 over plaintext in our case (we terminate TLS earlier, see next point.)
• The layer4 app runs the TCP proxy that terminates TLS, then if the next bytes are another TLS handshake, it proxies those to Signal's servers. Otherwise, it proxies the remainder of the connection to our HTTP server (over plaintext, but is localhost, so meh -- I also got this working over TLS but seemed unnecessary).
• The tls app tells Caddy to manage and use a certificate for example.com -- replace with your actual domain name.

The next steps are to improve on more of the shortcomings of the current offerings discussed above, and of course to have more people test and vet it for problems before we recommend it more widely.

[DuckSoft]

@mholt Yes, since you won't get a CONNECTION_RESET again when opening the website in the browser, where you get a true website instead, this is better than the original implementation. I'd be curious on how to integrate the authentication part inside to eliminate the active probing issue. Do you have any idea?

Also, passive detection issue may persist if Signal doesn't take some time with their networking stuff, which generally includes leaky DNS and API requests, missing ALPN (due to the content inside is not HTTP/HTTP2) and TLS in TLS packet length sequence features. Seems to me both the proxy itself and the integration with the proxy needs redesigning or resigning.

Anyway, this is a good start - people are starting to work together to discuss solutions, rather than making stuffs behind a shut door (Chinese: 闭门造车). Thanks for your contribution!

[abschluss24]

I don't think we can realize an authentication part without modifying Signal's client code. As Signal is improving the design/implementation of its TLS proxy (I hope it is!), our design will also need to adapt to future Signal releases, and dirty/smart hacks might not be able to survive such changes.

[mholt]

@DuckSoft

I'd be curious on how to integrate the authentication part inside to eliminate the active probing issue. Do you have any idea?

I am not sure that active probe resistance is possible without the client supporting some sort of pre-shared secret. I may need to re-read the above discussion in more detail to fully understand some of the proposals, but in principle, active probe resistance is akin to authentication, and that'll require client support too. Unfortunately we don't have control over the Signal clients.

missing ALPN (due to the content inside is not HTTP/HTTP2)

Re: Suggestion 3 above, about advertising a typical ALPN, I am not 100% sure what the requirements are here, but I tested this modified TLS termination handler in the configs I posted above, and this still works:

{
	"handler": "tls",
	"connection_policies": [
		{
			"alpn": ["http/1.1", "h2"]
		}
	]
}

I'm not sure how to improve the situation much more though without changing the Signal client.

[DuckSoft]

I don't think we can realize an authentication part without modifying Signal's client code.

Agreed. We need to unite developers of Signal to make out the plan.

[yicixingzhanghao001]

我关注到了你在hackernews上的提交。hackernews是由商业组织运行的，内容会被审查，还有很多网络间谍也活跃在上面，请小心行事。

---

Machine translation added by @wkrp:
I followed your submission on hackernews. hackernews is run by a commercial organization and content is censored, there are also many cyber spies active on it, so please proceed with caution.

[chagai95]

can you add examples for the traffic that is being monitored, perhaps a pcap file or just the text of the relevant packets?

[DuckSoft]

@chagai95 Yes, sure. I can provide you guys with the TLS ClientHello Packet.
Hold on a sec and I will edit this message in-place.

TLS ClientHello of Signal TLS Proxy

TLS ClientHello of Signal (without Proxy)

some "leaked" DNS queries (source)

---

ps: It's about 20 days, still Signal didn't make any direct response. I've seen the attitude and am not in any hope of their any further action. As stated in RFC 8890, the Internet is for End Users, which, I think, the same is true for this report (or whatever you call it). Signal may ignore this report, and it's totally fine -- as long as Iranian users finally know what's going on.

I've informed as many Iranians as I can about the fact and discouraged them from using Signal's immature TLS Proxy, and hopefully it would help.

Last but not least, let's take this incident as a unique case to study, rather than something to argue where who's correct or wrong -- it's pointless. Let's make Signal TLS Proxy a typical educational case, and learn from it, hopefully.

[sneak]

Unfortunately we don't have control over the Signal clients.

You absolutely do; the Signal clients are GPL.

[wkrp]

I was going to suggest Caddy with the forwardproxy plugin as a probe-resistant, mostly drop-in replacement for what the Signal TLS proxy does now with Nginx and stream_ssl_preread. forwardproxy was designed for this purpose: it works like a typical HTTPS CONNECT proxy with an allowlist of proxy destinations, except that it does not respond to unauthenticated proxy requests with 407 Proxy Authentication Required (that's the probing resistance).

• Source code
• Documentation
• Blog post

But I was looking at it today, and it's not clear that forwardproxy works with Caddy v2. I do not see it in the list of optional packages at https://caddyserver.com/download, and the documentation link https://caddyserver.com/docs/http.forwardproxy gives me 500 Internal Server Error and an empty page. @sfrolov, @mholt: is forwardproxy meant to work with the current version of Caddy? Alternatively, is it considered safe to use an older release of Caddy with which forwardproxy is compatible? I found caddyserver/forwardproxy#72 regarding Caddy v2 support, and its pull request caddyserver/forwardproxy#74, but it looks like it has not been merged or released yet. I see also that @klzgrad has made a fork of forwardproxy to use with NaïveProxy.

Anyway, here is how it could work with Signal. The Dockerfile downloads Caddy and forwardproxy, and generates a Caddyfile with random basic_auth credentials that are unique to this installation. (The username can be static, the password random.) The Caddyfile additionally sets probe_resistance (no need for a secret link in this use case) and has an acl block with allow for the necessary Signal domains, followed by deny all. The password can be shared with users, as @ewust suggested, in a signal.tube link, something like signal.tube/#domain.example;password.

[klzgrad]

I made a fork of forwardproxy to do one thing - obfuscate a few TLS-in-TLS packet sizes. Without it forwardproxy can be easily detected as a generic TLS-in-TLS proxy because TLS handshakes have only a few predictable packet sizes.

Something has to be here, but the obvious answer of introducing Pluggable Transports probably involves too much complexity to mholt's liking.

[cjhenck]

One of the exciting possibilities with ForwardProxy is the potential of doing HTTP CONNECT over QUIC. I don't think that's currently possible, though?

[mholt]

The forward proxy plugin is indeed an excellent option.

@wkrp That pull request to support Caddy 2 is just about finished, there's just one or two tests that has started failing recently (actually, hanging indefinitely) and I'm not sure why; even reverting recent changes doesn't make the problem go away, but I know that the last time I worked on it (about a month or two ago) all the tests were passing.

@sergeyfrolov wrote the forwardproxy plugin and its tests, and knows the code base better than I do. I won't have a lot of time to work on it in the near future. If anyone is able, contributions would definitely be welcomed, especially to get the tests passing. Please help if you can!

@klzgrad We'd be happy to consider your patch. Caddy 2 is all about pluggable transports in its reverse_proxy, but I don't know if/why obfuscating a few TLS packet sizes would require an entirely different transport -- submit an issue or PR and we can discuss it.

@cjhenck Feel free to submit a pull request to add that if you'd like to have it! But maybe after we get the Caddy 2 PR merged.

[pheki]

Unfortunately we don't have control over the Signal clients.

You absolutely do; the Signal clients are GPL.

Although anyone can fork and change the Signal clients, if you do so you're not allowed to use the official Signal servers, in other words, you can't communicate with anyone using the official Signal client.

See LibreSignal/LibreSignal#37 (comment)

[sneak]

A common misconception. Changing the clients and publishing a modified/forked client that connects to the official Signal servers is entirely permissible. The client is GPL, and you can put whatever URLs or server hostnames in your fork you wish, including the official Signal servers. Signal has no rights to dictate which lines of code you are required to change in your fork: the whole client is GPL.

The Signal TOS applies to users of the service. The GPL applies to the code.

If Signal has an issue with legitimate Signal users using forked client code, that is an issue between Signal and those users, not a software publisher. It is common these days to conflate software and services, but the Signal TOS for the Signal web service does not apply to the Signal source code, only the GPL does.

The GPL allows you to fork the client, and the GPL allows you to retain the official Signal servers in that forked client.

Imagine if Google said you weren't allowed to visit google.com using Chromium: we'd rightly point that out as preposterous.

[pheki]

I guess you're technically right: legally anyone can modify the client.

Whether users of the fork are welcome in the service and may be banned if detected, though, is another story.