Mobile data not working on 053j

[puelp]

I upgraded from 053i to 053j and apps can no longer access the Internet on mobile data. (Wi-Fi works fine.)

Everything was working fine in 053i, so the problem seems to be somewhere in the code of 053j.

I had to uninstall 053j and downgrade to 053i. Upon setting up 053i, I realized that there was no way to bulk-block apps, so I had to downgrade again to 053h, which I am currently on right now.

I am on a Samsung Galaxy device, One UI 4.1, Android 12.

[ignoramous]

I had to uninstall 053j and downgrade to 053i. Upon setting up 053i, I realized that there was no way to bulk-block apps, so I had to downgrade again to 053h, which I am currently on right now.

In v053i we introduced support for IPv6, and that has a few corner cases where apps think there's IPv6 when in reality there may not be (and hence loss of internet connectivity).

As a workaround, in both v053i and v053j, change Choose IP Version (from Rethink Settings) to IPv4 to fallback to how v053h worked.

---

I upgraded from 053i to 053j and apps can no longer access the Internet on mobile data. (Wi-Fi works fine.)

Are you on T-Mobile? When you say "apps", even browsers didn't work or was it only certain apps?

[didi8517]

I have the same experience, switching from auto to IP4 worked for me. Hopefully in the next update there'll be a fix, great job!!! Keep it up!

[afontenot]

@didi8517 @puelp could either of you say what carrier you're on? I believe that would help the devs.

For Google Fi / T-Mobile specifically (which might be the source of all these bugs), see #554

[didi8517]

I'm in Europe, so I dunno if that will tell you much, it's called KPN ... I know they've added IPv6 support to all mobile sims, but a quick search didn't give me much more, at least not more recent. Is there anything you want to do, run some tests or so ...?
I've got an LG Velvet 5G Android 12, btw. If you need any other info, or something to try out, give us a shout!

[afontenot]

Edit: sorry, I tagged someone from another project I've been working on recently by accident. Fixed.

@ignoramous Best I can tell from this thread KPN is using NAT64 + DNS64. Similar to my situation, although probably (?) not using 464XLAT - I think "pure" NAT64/DNS64 is supposed to be common in Europe.

Hope this helps.

@didi8517 I'm not a Rethink dev, ignoramous will tell if you if they need something specific, but can you confirm whether you can access any IPv6 enabled sites when in "auto" mode in Rethink? (Try several different browser apps and sites, some will assume the site is down if they try IPv4 first and can't get a connection.) One good site to try is https://v6.ipv6test.app/ because that subdomain doesn't have a IPv4 A record. Accessing IPv6 sites frequently works for me on my 464XLAT carrier in the United States.

[didi8517]

Hi @afontenot ,

thanks for your input. I tried the ipv6 test site and some more. I noticed problems yesterday that when auto was on, and I'd use the Google assistent to listen to the news, that when I was on data, most of the podcasts wouldn't work, whereas when connected to Wifi there was no problem. Someone on the telegram group pointed me in this direction, and I tried switching from auto to only IPv4, and the problem was gone (on data). That's why I posted here also.
Now about trying out your link (on Firefox and Kiwi, i'm sorry I didn't try more)
A) with RDNS off

1. on wifi:
   as to be expected no problem.

2. on just data:
   as to be expected no problem.

B) with RDNS on:

1. on wifi:
   with setting set to
   -IPv4: as to be expected doesn't connect (other websites work)
   -IPv6: works
   -auto: works

2. on data:
   with setting set to
   -IPv4: won't connect (other websites work)
   -IPv6: works
   -auto: works

I then tried all of these steps again with the news, where I initially encountered the problem, and the results are rather inconsistent... It could of course be that when it does connect when I'd expect it not to, that it was playing from cache, but I doubt so, looking at the logs in RDNS, seeing that the connection sometimes doesn't go through (it has a ? as a symbol, and clicking on it the info says "No Answer", instead of the IP connected to it says "NXDOMAIN". Those are always IPv6 requests, the IPv4 to the same server are always fine. At first I got the impression that only if I used auto that those domains don't work, but if I either choose IPv4 or 6, all news get played... But right now it seems to sometimes work, sometimes not... But my general impression is, also looking at the logs, that on auto, it goes wrong... if I pick either, then some of those news servers don't give/get an answer...
And like I said, it doesn't happen with all of them. The problematic ones are:
podcast-mp3.dradio.de
cdn.nos.nl
static.bnr.nl
feeds.soundcloud.com
I'll see if I can make some more tests ... and get something clearer and more consistent... Hope this still helps

[didi8517]

Here's another one: I have smart lamp using the tuya protocol. They have a free web api you can send HTTP POST/GET commands to controll the things. The server is px1.tuyaeu.com. If I'm on data, using Termux, and have setting set to:
IPv4: then ping and curl work (ping6 obviousbly not)
IPv6: then ping6 and curl work (ping obviousbly not)
auto: ping says "connect: no route to host"; ping6 says "unknown host"; curl says "(7) couldn't connect to server".
RDNS logs are similar to what I described in last post: All ipv4 connects to the server are ok, all IPv6 "No Answer" with "?"
I noticed it because I use Llamalab Automate to control the lamp and there I got an EAI_NODATA (No address associated with hostname) error, which led me to... I've been trying to get the IPv6 address of the server to send the commands directly, but kinda can't figure out how to, I don't know how to put the IPv6 address in so it'll work. It keeps saying it's invalid, or takes the numbers after the first ":" to be a port... not even with IPv4, I get 404... But I haven't tried all the IPs that RDNS log has, I just used the one I get from the ping...
Edit: Ok, but I copied the IPv6 from RDNS log and whilst set to "auto" I ping6-ed that and it works, but pinging the hostname doesn't work. So I thought it's a DNS problem, but switching to System DNS didn't make any difference. Hope this still helps
EDIT: And now their server won't answer at all after all the pinging I've done, not even from behind a VPN lol

[afontenot]

@didi8517 Thanks, that seems helpful to me. The behavior on data seems to be the same as what I see on my 464XLAT carrier, which suggests that you will probably need the same fix.

I doubt so, looking at the logs in RDNS, seeing that the connection sometimes doesn't go through (it has a ? as a symbol, and clicking on it the info says "No Answer", instead of the IP connected to it says "NXDOMAIN".

Are you using one of the DoH DNS resolvers, like Cloudflare, etc? I saw this behavior a lot in this case. What appears to be happening is that Rethink will pick an IP address for the resolver at random (it has a list of them stored). If it picks IPv4, the connection will fail, and then Rethink will be unable to resolve the domain. That's one of several failure modes and can give the impression that connections are working / failing at random.

There are also two other effects.

• Rethink seems to briefly cache resolved domains locally (not sure how long), and in this case if you have a domain cached it will still resolve even if you change your settings to something that in previous testing would break DNS. This makes it hard to confirm the effects of different settings.
• Some apps, if they try IPv4 first and get a IPv4 address from the resolver, will present you with a "could not connect to server" error when that connection fails instead of then trying IPv6 (which would work). This adds a bit more randomness because it's hard to guarantee which protocol will be used unless you disable one of them, use curl, or try a site that only supports IPv6.

These are just my observations - I haven't read the code closely enough to confirm that this is exactly how it behaves / how it's supposed to behave.

[didi8517]

Thx @afontenot

Are you using one of the DoH DNS resolvers, like Cloudflare, etc? I saw this behavior a lot in this case. What appears to be happening is that Rethink will pick an IP address for the resolver at random (it has a list of them stored). If it picks IPv4, the connection will fail, and then Rethink will be unable to resolve the domain. That's one of several failure modes and can give the impression that connections are working / failing at random.

No, I was only using the RDNS Plus servers. I just tried switching to Cloudflare and seems to give same results. Even when I set it to System DNS (though that also didn't work well in general, after some time I'd get "Failing" or "waiting", and even with RDNS after that at first...)

[ignoramous]

You folks see any improvements in v054c / v055?

At first I got the impression that only if I used auto that those domains don't work, but if I either choose IPv4 or 6, all news get played... But right now it seems to sometimes work, sometimes not... But my general impression is, also looking at the logs, that on auto, it goes wrong...

We added connectivity probes when Rethink is in Auto mode to help with 4to6 networks using 464Xlat and DNS64. Since Indian providers use DS-Lite, we're unable to test it for real ourselves.

Rethink seems to briefly cache resolved domains locally (not sure how long), and in this case if you have a domain cached it will still resolve... This makes it hard to confirm the effects of different settings.

Rethink starting v055 doesn't respond from cache if it detects connectivity loss.

What appears to be happening is that Rethink will pick an IP address for the resolver at random (it has a list of them stored).

Rethink's DNS-over-HTTPS implementation does try to probe ALL IPs (both IPv6 and IPv4) available to it. May be there's a bug if this isn't happening.

Here's another one: I have smart lamp using the tuya protocol. They have a free web api you can send HTTP POST/GET commands to controll the things. The server is px1.tuyaeu.com. If I'm on data, using Termux, and have setting set to:
IPv4: then ping and curl work (ping6 obviousbly not)
IPv6: then ping6 and curl work (ping obviousbly not)
auto: ping says "connect: no route to host"; ping6 says "unknown host"; curl says "(7) couldn't connect to server".

I must note that ping is unsupported on Rethink for IPv6 (as in, it won't work the way you expect it to); see: #450

[didi8517]

Hello ignoramous. I haven't actually tried again, since, considering it kept saying "experimental". I'll try to do some experiments in the coming days, when I get a chance.
Thanks!

[ignoramous]

No worries. Keep us posted!

I haven't actually tried again, since, considering it kept saying "experimental".

It'll keep saying that unless one of you folks who are on 464Xlat / DNS64 networks confirm Auto isn't giving you pain (:

[didi8517]

Hi again ignoramous,

sorry it took so long. I've been traveling and when I came back I couldn't really remember what the precise issue was. After reading through the posts here, what I've tried now was:

1. Setting Rethink to Choose IP version to auto and switch over to data only (wifi off)
2. then I found out that the mobile APN setting was set to IPv4 only, so set that to both
3. I've tried the above given test website https://v6.ipv6test.app/ (and others: https://test-ipv6.com/, including a Dutch one) and they all give me a full ok, that IPv6 is working, (except for ipv6-test.com, sometimes doesn't really load, once it only finds ipv4, then only ipv6 ...?! but it also doesn't show ipv4 on my laptop over wifi...)
4. I've tried listening to the news, since that was where I first noticed problems: all went well.

I'm not sure that this means it's alls solved now...? If there's anything else that I ought to check and try out, let me know.

Thanks

[ignoramous]

Thanks. Final set of sanity checks before I close this:

then I found out that the mobile APN setting was set to IPv4 only, so set that to both

Can you confirm Rethink reports protos: IPv4, IPv6 in the bottomsheet that comes up when you tap on the down-arrow next to the START / STOP button on Rethink's homescreen?

I've tried the above given test website https://v6.ipv6test.app/ (and others: https://test-ipv6.com/, including a Dutch one)

Can you please confirm if you are able to access https://ip6.me on mobile data? And that it shows you your IPv6 address same as on those other websites?

Also, do you see IPv6 addresses in the Network Logs, too?

I've tried listening to the news, since that was where I first noticed problems: all went well.

If you use Facebook or Facebook Messenger, can you confirm if those work on mobile data, too?

[didi8517]

Can you confirm Rethink reports protos: IPv4, IPv6 in the bottomsheet that comes up when you tap on the down-arrow next to the START / STOP button on Rethink's homescreen?

Yes it states that exactly.

Can you please confirm if you are able to access https://ip6.me/ on mobile data? And that it shows you your IPv6 address same as on those other websites?

Yes to all questions.

Also, do you see IPv6 addresses in the Network Logs, too?

Yes, they usually resolve ok. One exception was the ipv4.test-ipv6.arauc.br which didn't get an answer on any protocol (ipv6, ipv4, http service binding, send_fail and no answer it says for all), but that shouldn't reflect on this, then, I guess

If you use Facebook or Facebook Messenger, can you confirm if those work on mobile data, too?
I don't use those on my phone, but I could give it a try, if that really helps?

[afontenot]

I didn't initially reply to this because I switched carriers recently, and I'm not sure if my current one uses 464Xlat or not (seems likely in the US). Gave it a quick test and I'm not seeing any issues, and I appear to be able to access IPv4 sites without problems. I see appropriate external IPv6 and IPv4 addresses as well.

Rethink is reporting both protocols. Network switching appears to work as well - I had significant issues with that in the past.

Thanks for your work on this!

[ignoramous]

I don't use those on my phone, but I could give it a try, if that really helps?

That's fine. I wouldn't want to force anyone to use Facebook products (:

Thanks a lot for running tests for us. In India,¹ no mobile network operator implements DNS64/NAT64 nor 464Xlat, so it was hard for us test any of the fixes we were rolling out.

I'm closing this issue, but @puelp if it isn't fixed for you, please feel free to reopen.

Footnotes

1. Indian operators use DS-Lite, which perhaps works the best of most 6to4 solutions. ↩

[ignoramous]

Rethink is reporting both protocols. Network switching appears to work as well - I had significant issues with that in the past.

Glad its sorted. tbh, I'm not really pleased with our current fix; but our inability to perform tests hindered just how cleanly we could fix it.

I'm not sure if my current one uses 464Xlat or not

Think that can be verified with ifconfig or ip commands via adb shell. On Android, clatd opens up a 464Xlat tun network interface (viz. clat4 or v4-rmnet0) on a private IPv4 net (usually, 192.168.255.1 or 192.0.0.1).

[ignoramous]

Btw, I just recalled that Rethink tries some funny things to determine presence of DNS64/NAT64 (ex). Not sure if it works as expected. Do any of you folks see IPv6 addresses begining with 64: in your Network Logs and those connections going through without issues (tapping on these 64: entries would bring up a bottomsheet at the footer of which error messages are shown, if any).

[didi8517]

@ignoramous,

Think that can be verified with ifconfig or ip commands via adb shell

In ifconfig I have a "v4-rmnet_data2 Link" with a 192.0.0.4 address, that would be the clatd you mentioned, right? So that should mean it's running 464xl, right?

Do any of you folks see IPv6 addresses begining with 64: in your Network Logs

I'm not sure if that's what you meant, but I searched for exactly this, "64:" and in neither of the logs anything showed up. But considering that I only just turned on IPv6 on my data, I'll have look in a couple of days again, and let you know.

[didi8517]

Sorry for the long silence, totally forgot to get back about this....
Just had a search for "64:" in the logs and no addresses starting that way to be found. Otherwise it seems to be running well with IP version set to auto on either network type.
Thanks

[didi8517]

@ignoramous and all,

sorry for coming back on this issue, though it was marked as completed.
I recently swapped for another phone, a variant of what I had, a LG Velvet (no 5G). Although it's very similar in most things, I'm again having trouble running Rethink with IPv6 (or auto), and not only when on data, but also WiFi now!

I've looked at the APN and set it to ip4/ip6 (both) and if I stop Rethink then I can load all the ipv6 test pages and everything's fine... But when I activate Rethink then IPv6 breaks... And it says "protos: ip4, ip6".
My ifconfig returns the "v4-rmnet_data0 Link" with the same IP as on the old phone "inet addr:192.0.0.4 P-t-P:192.0.0.4" (there are also two more rmnet links, one data0 and another ipa0). When Rethink is active, its tun0 Link has two ip6 addresses that are different from the one's in the rmnet data0 link.
If I connect to WiFi it has 4 ip6 addresses (in ifconfig), two global scope, one link scope. None of them is the same as Rethink's tun0 ip6. But as opposed to when on data, Rethink only has one (link scope, no global).
I've also tried swapping between Sky and Max DNS, no difference.
The IP6 requests show up in the logs, and would seem to resolve no problem.

Any ideas, suggestions...?

Many thaks in advance!

[ignoramous]

When Rethink is active, its tun0 Link has two ip6 addresses that are different from the one's in the rmnet data0 link. If I connect to WiFi it has 4 ip6 addresses (in ifconfig), two global scope, one link scope. None of them is the same as Rethink's tun0 ip6. But as opposed to when on data, Rethink only has one (link scope, no global).

Will you please share the output of these commands (make sure to truncate out some part of the public IPs)?

[didi8517]

Hi @ignoramous and thanks for the quick reply!

Here's the output for
a) data (no wifi connection)

rmnet_ipa0 Link encap:UNSPEC
          UP RUNNING  MTU:2000  Metric:1
          RX packets:82204 errors:0 dropped:0 overruns:0 frame:0
          TX packets:72018 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:262109221 TX bytes:34060242

tun0      Link encap:UNSPEC
          inet addr:10.---.1  P-t-P:10.---.1  Mask:255.255.255.0
          inet6 addr: fd66:---::1/120 Scope: Global
          inet6 addr: fe80:---:1bf1/64 Scope: Link
          UP POINTOPOINT RUNNING  MTU:1500  Metric:1
          RX packets:580 errors:0 dropped:0 overruns:0 frame:0
          TX packets:608 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:82542 TX bytes:448640

wlan0     Link encap:UNSPEC    Driver icnss
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:1280720 errors:0 dropped:2 overruns:0 frame:0
          TX packets:572402 errors:0 dropped:48 overruns:0 carrier:0
          collisions:0 txqueuelen:3000
          RX bytes:1242192188 TX bytes:189901050

v4-rmnet_data1 Link encap:UNSPEC
          inet addr:192.0.0.4  P-t-P:192.0.0.4  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1472  Metric:1
          RX packets:6 errors:0 dropped:0 overruns:0 frame:0
          TX packets:52 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:168 TX bytes:8634

dummy0    Link encap:UNSPEC
          inet6 addr: fe80:---:c8cb/64 Scope: Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3167 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 TX bytes:788264

lo        Link encap:UNSPEC
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope: Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:50369 errors:0 dropped:0 overruns:0 frame:0
          TX packets:50369 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:5531026 TX bytes:5531026

rmnet_data1 Link encap:UNSPEC
          inet6 addr: 2a02:---:d4e1/64 Scope: Global
          inet6 addr: fe80:---:d4e1/64 Scope: Link
          UP RUNNING  MTU:1500  Metric:1
          RX packets:119713 errors:0 dropped:0 overruns:0 frame:0
          TX packets:84396 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:120759728 TX bytes:21530676

b) No data, but Wifi (!!! It was a typo when in my earlier post when I said 4 ip6's, it's 3)

p2p0      Link encap:UNSPEC    Driver icnss
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3000
          RX bytes:0 TX bytes:0

rmnet_ipa0 Link encap:UNSPEC
          UP RUNNING  MTU:2000  Metric:1
          RX packets:82251 errors:0 dropped:0 overruns:0 frame:0
          TX packets:72071 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:262120235 TX bytes:34071054

tun0      Link encap:UNSPEC
          inet addr:10.---.1  P-t-P:10.****.1  Mask:255.255.255.0
          inet6 addr: fe80:---:efe3/64 Scope: Link
          inet6 addr: fd66:---:1/120 Scope: Global
          UP POINTOPOINT RUNNING  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:0 TX bytes:0

wlan0     Link encap:UNSPEC    Driver icnss
          inet addr:192.---.1  Bcast:192.---.255  Mask:255.255.255.0
          inet6 addr: 2a02:---:3a59/64 Scope: Global
          inet6 addr: 2a02:---:20a5/64 Scope: Global
          inet6 addr: fe80:---:6581/64 Scope: Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1281763 errors:0 dropped:2 overruns:0 frame:0
          TX packets:573347 errors:0 dropped:48 overruns:0 carrier:0
          collisions:0 txqueuelen:3000
          RX bytes:1242826155 TX bytes:190267052

dummy0    Link encap:UNSPEC
          inet6 addr: fe80:---:c8cb/64 Scope: Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3209 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 TX bytes:795376

lo        Link encap:UNSPEC
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope: Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:51110 errors:0 dropped:0 overruns:0 frame:0
          TX packets:51110 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:5615132 TX bytes:5615132

I forgot to mention in my earlier post another difference between these two models is that this one has dual sim, and I have two sims in it, but only the main one has data activated. Not sure this might be part of the trouble.

I also noticed just now, experimenting some more, that IPv6 only NOT works when both FW and DNS are active. If either is active, the ip6 address gets recognized fine... ???!!! But perhaps thats also a cache problem...

In the network settings I have nothing activated. In the DNS settings I use RDNS Plus over Max, but with on device blocklists, and all other options activated. I've also tried without DNS boost or leaking prevention, no change.

Anything else?
Thanks!