WIFI can no longer use hostname in HTTP URL #9712
Comments
|
Works fine here. tested daily with |
|
Using 3.0.0 final and 30 second loop time, fails after 3 to 12 iterations. Following is debug output: 21:51:23.421 -> [167419][V][HTTPClient.cpp:230] beginInternal(): url: http://hector:8080/ Haven't been able to workout what -54 means. |
|
Explicitly called getaddrinfo. This returns 202 when above error occurs. |
|
How is that getting resolved in the network? mdns or does the dns server have a default domain? |
|
The router provides DNS. It does have a default domain. Using the FQDN makes no difference. |
VojtechBartoska
added
Area: WiFi
|
DNS fails because the software start trying to send the DNS request to the routers IPV6 address. I don't see any trace of this packet using Wireshark Before DNS starts to fail, dns_set_server() get called twice, both times with an IPV6 address. After that, DNS fails. Some time later, dns_set_server() is called with an IPV4 address and DNS starts working again. I suspect that the software is seeing some DHCPV6 packet that it should ignore but I haven't verified this. I do see a DHCPv6 advertise packet at about the same time as the bogus dns_set_server calls. |
|
Have you tried enabling IPv6? |
|
Doesn't seem to make any difference. Not sure I've done it correctly. void WiFiEvent(WiFiEvent_t event) { Appears dchp6.c is unrelated as it doesn't seem to be calling dns_set_server. Can you tell me what version of lwip is used in the current board? None of the releases of lwip that I downloaded appear to match. |
WiFi.enableIPv6(true);
WiFi.begin(ssid, pass); |
|
That’s weird. Mostly this works but my software regularly switches between using IPv4 and IPv6 for DNS.
After boot, if the first DNS request works, then DNS keeps succeeding. If the first DNS fails, then DNS keeps failing.
On boot, it mostly works but it fails about 20% of the time.
Strangely, if I connect to my PCs hotspot rather than to my Wifi Access Point, DNS uses Ipv4 exclusively.
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows
…________________________________
From: Me No Dev ***@***.***>
Sent: Wednesday, June 12, 2024 3:00:47 PM
To: espressif/arduino-esp32 ***@***.***>
Cc: thammr ***@***.***>; Author ***@***.***>
Subject: Re: [espressif/arduino-esp32] WIFI can no longer use hostname in HTTP URL (Issue #9712)
WiFi.enableIPv6(true);
WiFi.begin(ssid, pass);
—
Reply to this email directly, view it on GitHub<#9712 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGKOCO7RXZOQ4CLX2V3EPA3ZG7MIPAVCNFSM6AAAAABINVKRV2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRSGEZTQMJRGA>.
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
|
I had at least a very similar problem with my ongoing ESP32 project. After a certain period, no more data was transmitted to an external server because the hostname could no longer be resolved. Sometimes the problem and the following output appeared after just half an hour, sometimes only after one or two hours:
As @thammr in #9712 (comment)_ pointed out that this issue is related to IPv6, and because @me-no-dev in #9712 (comment)_ suggested enabling IPv6, I explicitly enabled IPv6. My project is working with WifiMulti, where the corresponding option can or must be set directly to true in WiFiMulti.cpp:
Thank you for the solution approach. The happy result is that there are indeed no more connection drops now. In my view, it is still a strange behavior when it was possible to resolve the hostname with IPv4 for half an hour before, and then suddenly, no connection can be made because apparently only IPv6 is to be used to reach the DNS server. At least my router has IPv6 enabled, which is probably why this solution works well for me. I do not know if this solution will also work if only an IPv4 network is available. Perhaps my contribution will help solve this case for someone. Best regards |
|
maybe we need to check WiFiMulti, but in general when you switch to IPv6 (having global address that can reach internet) the DNS cache will be purged and new request would be made to get the possible new IPv6 address of the host. This is true since 3.0.0 for using the regular WiFi interfaces. Did not check Multi though |
Thank you very much for the troubleshooting and your work here on GitHub, @me-no-dev. Just a quick note: I hadn't looked into WiFiMulti in detail yet, but I just read in the description that it is mainly used when multiple Access Points are available and when a connection to the strongest available Wi-Fi network should be established. In my project, I don't really need this function, but WiFiMulti was used in the example project "BasicHttpsClient", which I initially chose as the basis for my new project. I can imagine that other users might also use WiFiMulti for this reason, and that they might encounter the same issue, or that they could maybe use this solution to solve the problem with connection dropping because this IPv6/DNS issue as described. Best regards. |
|
After much investigation, I am convinced that the DNS problem is not due to any deficiency in my LAN appliances. Just to summarize my findings. If there is no ICMPv6 traffic on the LAN, the software uses IPv4 without problem. If I connect to the hotspot on my Windows 11 PC, I see no ICMPv6 packets and IPv4 works perfectly. So there shouldn't be any problem on an IPv4 only network. However, if there is ICMPv6 traffic on the LAN, the ESP32 decides to join the fun. Every 400 seconds, it sends an ICMPv6 Router Solicitation and then provides the resultant IPv6 address to the dns module. After that, no DNS requests are sent and consequently DNS just fails. At some stage, the dns module is told to use the routers IPv4 address again and DNS starts working until the next 400 second time slice expires and DNS start failing again. While DNS is failing, MQTT and HTPP continue to work fine. If I enable IPv6, DNS uses IPv6 but HTTP continues to use IPv4. This solution would be viable apart from the fact that DNS fails completely on every 4th or 5th reboot. It appears that something is not being initialized correctly. When DNS fails, no DNS requests are sent. Furthermore, I can find no documentation on enabling IPv6 and in my humble opinion, IPv6 needs to be optional anyway. Consequently, I have reverted to 2.0.17. |
It is optional. You need to call |
|
That’s the point. The software tries to use IPv6 for DNS without calling WiFi.enableIPv6() or even if I call WiFi.enableIPv6(false).
I don’t think the software should be sending ICMPv6 messages unless WiFi.enableIPv6() is called.
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows
…________________________________
From: Me No Dev ***@***.***>
Sent: Tuesday, June 25, 2024 2:58:46 PM
To: espressif/arduino-esp32 ***@***.***>
Cc: thammr ***@***.***>; Mention ***@***.***>
Subject: Re: [espressif/arduino-esp32] WIFI can no longer use hostname in HTTP URL (Issue #9712)
IPv6 needs to be optional anyway
It is optional. You need to call WiFi.enableIPv6() to make it work. Do you have this happen without calling it?
—
Reply to this email directly, view it on GitHub<#9712 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGKOCO7R24A55L7L46GMGF3ZJD5Y5AVCNFSM6AAAAABINVKRV2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBYGAYDSMRZGQ>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
unfortunately we have no control, other than |
|
The official statement of the IDF team is that there is no way for the ESP to respond to any IPv6 traffic, unless a Link Local address is assigned to the interface, which can also only happen when |
|
Wireshark begs to differ. In the following figure, packet 10623 is sent by the ESP32. The Wireshark filter is the ESP32s MAC address.
***@***.***
The packets source MAC address is my ESP32s MAC Address.
***@***.***
Following shows the same transaction with the filter off. This shows the router responding to the Router Solicitation.
***@***.***
After the above transaction, DNS fails.
Either the IDF team is wrong or I have something on my network spoofing the MAC address of any ESP32 whose software is built using version 3.
This was captured using 3.0.2 of the board. The code definitely is not calling enableIPv6.
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows
|
I reckon this issue might have already been covered in espressif/esp-idf, too. ;) The error described in the above post might stem from the same behavior. But of course, this is just an initial suspicion. |
|
Well. so much for being able to respond by email. That removed all my attachments.
Hope this bloody works! |
|
I am facing a similar problem.
I have always used DNS names there. Since switching to 3.02 this works for the first time, after that DNS is no longer resolved. The script does not use IPv6 either. But IPv6 packets are travelling in the connected network. |
|
@peff74 did you try |
|
@me-no-dev 1 loop() == 1 Sec. |
|
@peff74 that seems to be a different issue. Your DNS is working well |
|
@me-no-dev |
|
This thread is about DNS issue. In your log, DNS is working fine, therefore is a different problem. Your DNS issue was "fixed" by enabling V6, now you have another one |
|
@me-no-dev any ideas anyway, what can cause the dns issue? is it idf related? |
|
@me-no-dev I want to be clear that this is not a criticism of your work, which is great, but there is a problem here with DNS in Core 3.x.x. |
|
@peff74 we are tracking the DNS problem, but your last log shows DNS working fine, then something else not working. Please differentiate between DNS not working and then your client not connecting. Two separate things. @cziter15 trying to pursue IDF's network team to accept that there is such issue and look at it. It would have been great if I can reproduce it, but I can't. It definitely needs some particular environment case that I can not simulate here. Work continues... |
|
@me-no-dev, when it comes to the environment I wasn't able to reproduce it on ESP32S3 and my home router (sagemcom cs 50001 - with IPv6 enabled on the router). In my parant's house there's D-LINK DWR921 with OpenWRT flashed. The issue occured in my parent's house. There's a device I've built to monitor their central heating kettle. I'm hosting MQTT server for all the devices. Reproduction was easy - just reboot the router in my house - it triggers no-ip to update the IP and then these devices should retry to reconnect. ESP32 was unable to reconnect. W/o worharound it eventually comes back online but after 4-5h. Other ESP8266 devices come back almost instantly. |
|
I've been trying to diagnose why my app was losing connection to a telegram bot due to DNS failure when compiled with core v3.x when it worked fine with v2.x. Having read this thread and added |
You're right, so I only tested DNS. I have resolved both IPv6 and IPv4. And if DNS takes too long... of course does not work connect(), too. |
|
Does it continuously take multiple seconds, or does it "fix" itself after some time? Maybe TTL of the record is about to expire in the cache and it does not know which DNS to ask, so query fails and it returns the cached result. |
|
No, as soon as the error starts, it persists endless. here is the LOG how the DNS works again by reconnect |
|
I have now also performed a test with a public host that is only accessible via IPv4. |
|
I have been able to replicate the issue (V4 DNS fails, when V6 is advertised)! Stay tuned! |
|
Fix is on the way! Will be fixed in 3.0.3 (hopefully tomorrow) |
|
Hi all, thanks a lot! Just wanted to confirm, that this fix works! Good job! I had this exact issue on all my ESP32 (ESP32-D0WDQ6) while IPV6-DualStack also enabled on my router (Swisscom Internet Box 3 / Arcadyan VRV951A FW 14.20.24). Only way to get around this issue was to disable IPV6 on my router. Today i recompiled my sources with 3.0.3 and enabled IPV6 on my router (not in my arduino source), and i no longer get a IPV6 DNS, it stays at IPV4. |
|
I ran my DNS IPv4 / IPv6 script with Core 3.0.3 tonight, with the ETH.enableIPv6(true);. So there still seems to be a problem. Now I'll test client.connect(); with DNS and IPv4 only without ETH.enableIPv6(true). |
this is what was actually fixed. Why your queries start to take more time in a sudden is not yet clear |
|
I can confirm that now client.connect() works perfectly with DNS and only IPv4 (i.e. without ETH.enableIPv6(true)). Are you still working on the other topic, or should another ticket be opened for it? |
|
what other topic? |
|
well, the issue that when IPv6 is activated, the queries for IPv4 take an extremely long time after a few hours. |
|
i could not reproduce this here even after two days |
|
Are you trying internal or external hosts to resolve? One network was connected with Fritzbox as router/DNS and FTH. |
|
My router does not provide local DNS and I use MDNS to access internal hosts, so no way to test the same way as you. You sure your local DNS is not limiting you? What if you don't bombard it with requests every second? And why is cache not working? What is the TTL of the records that your local DNS is providing? Anyway... I will not fault the ESP for that. As you see your external hosts are fine, but at the same time, the ESP does not care or know that it's not a remote host, so also no way to act different based on it. |
|
I'm pretty sure it's not my DNS server, especially since the W2k19 provides a company network with AD, which gets bored with one query per second. but without success |
|
It looks as if the DNS is "forgotten" after a short time. The example code runs for several minutes until Activating #include "Arduino.h"
#include "WiFi.h"
#include "WiFiClient.h"
WiFiClient client;
char ssid[] = "*****";
char pwd[] = "*****";
char host1[] = "as-hls-ww-live.akamaized.net";
char host2[] = "stream.revma.ihrhls.com";
IPAddress dns1(8, 8, 8, 8);
IPAddress dns2(8, 8, 4, 4);
uint32_t timer;
void setup() {
WiFi.begin(ssid, pwd);
while(WiFi.status() != WL_CONNECTED){;}
log_i("WiFi connected");
timer = millis();
}
void loop(){
int res;
// WiFi.setDNS(dns1, dns2);
vTaskDelay(5000 / portTICK_PERIOD_MS);
res = client.connect(host1, 80);
if(res) log_i("%is host1: success", (millis() - timer) / 1000);
if(!res) log_i("host1: connection failed");
vTaskDelay(5000 / portTICK_PERIOD_MS);
res = client.connect(host2, 80);
if(res) log_i("%is host2: success", (millis() - timer) / 1000);
if(!res) log_i("host2: connection failed");
}
|
Board
ESP32S2 Development Board
Device Description
ESP32S2 Development Board
Hardware Configuration
Two VL53L1Xs
Version
v3.0.0
IDE Name
Arduino IDE
Operating System
Windows 11
Flash frequency
OTA
PSRAM enabled
no
Upload speed
N/A
Description
My ESP32S2 POSTs a data packet to InfluxDb every 30-40 seconds. After recompiling my code today, now POSTing the data fails after about 5-7 packets. Wireshark shows that the ESP32 starts trying to send packets to 0.0.0.0. However, if I replace hostname with the IP, everything works fine. While the ESP32S2 is failing to POST data to InfluxDb, all the other networking stuff works OK- MQTT, OTA, ping.
The board manager shows version 3.0.0-rc3. I tried downgrading to rc2 - same result, Downgraded to 2.0.16 - works OK now.
I have at least 6 ESP32s sending data to InfluxDb without problem. All use hostname in the URL.
uri = String("http://") + host + ":8086/api/v2/write?org=" + org + "&bucket=" + bucket + "&precision=s";
The following code running on a Tiny Pico starts returning -1 after several minutes, Some time later, it will start returning a valid status (405 in my case). This pattern repeats.
Sketch
Debug Message
Other Steps to Reproduce
No response
I have checked existing issues, online documentation and the Troubleshooting Guide
The text was updated successfully, but these errors were encountered: