Added support for W5500 via. esp_netif_t interface.

[OekoSolveMG]

Description of Change

The Ethernet library already supports W5500, the main problem is that it does not use the underlying esp_netif_t client that the WiFiClient uses, but instead a custom workaround because the ESP base library did not add esp_netif_t support W5500 until Version 4.3.0 or higher.

As this has been added for nearly a year now and makes it possible to use the base esp_https_ota library over ethernet via. the esp_netif_inherent_config_t more specifically the if_name attribute. I've added an additional constructor to the ETH library that initalizes the W5500.

I first tried to integrate it into the current constructor, but the parameters passed were that different and the construction requires a lot of additional steps that I think makes using a seperate constructor more useful.

All these changes overall now allow to easily update over Ethernet with the additionally updated HttpsOtaUpdateClass library.

I also added examples for all the new features and adjusted the documentation in the readme for the new HttpsOtaUpdateClass.begin() method.

Tests scenarios

I have tested my Pull Request on Arduino-esp32 core v2.0.4 with ESP32 and M5 Stack Core 2 with a W5500.

The test scenarios were connecting to google.com on Port 80 as well as doing an OTA update over the modified HttpsOtaUpdate.

Illustration

Related links

Some of theese have already been closed most of them tough, because there is an alternative library for ethernet over W5500, that does not use the underlying esp_netif_t client interface.

#5913 (Ethernet with W5500, now possible even when using integrated base ethernet in the library, based on esp_netif_t)
#5482 ""
#2637 ""
#6403 (Possible with updated HttpsOtaUpdateClass)
#4553 ""

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[VojtechBartoska]

Thanks for the PR @OekoSolveMG! We really appreciate your contribution, I'm adding this to next major release 2.1.0 as it needs further testing and investigation between Arduino's and ESP-IDF APIs.

[OekoSolveMG]

It seems I understand now why this hasn't been integrated until now. The esp_netif_t way to integrate the w5500 uses the underlying spi bus, which uses an spi driver by esp-idf. That works perfectly alone.

The problem is this is incompatible with arduino-esp32, because Arduino does not use the spi driver by esp-idf and therefore the same xSemaphore lock but instead uses another spi driver that creates it's own xSemaphore, meaning that the device crashes if the Ethernet and another device on the SPI is communicated with and it is done over the Arduino SPI library instead of the spi driver by esp-idf.

[VojtechBartoska]

Status update:

• we need to test if we are able to run this with current Arduino SPI API,
  -- if yes, this can be implemented in Bugfix relese
  -- if not, we will handle this for 3.0.0 milestone

[ruihe1117]

@OekoSolveMG

Hello, thank you for your selfless contribution; The code you modified can be compiled on ESP32, and the ESPAsyncWebServer example can be run well (it runs well when WiFi STA, WiFi AP and Ethernet are enabled at the same time, and the httpserver and WebSocket services can be executed, and the three IP addresses can be accessed at the same time).

However, it is not good on ESP32-S2, ESP32-C3 and ESP-S3. An error is reported when compiling. The use of ArduinoIDE and PlatformIO are the same problem. It happens that these chips do not have ETH interfaces and need SPI Ethernet functions.

ESP32-S3 has USB-HOST and more pins, but no ETH. If the W5500 can be used, it will change perfectly; Look forward to your repair.

Compilation errors are reported as follows:

.../tools/sdk/esp32s3/include/esp_common/include/esp_attr.h:124:35: error: conflicting declaration of C function 'constexpr spi_ll_trans_len_cond_t operator~(spi_ll_trans_len_cond_t)'
.../tools/sdk/esp32s3/include/esp_common/include/esp_attr.h:125:35: error: conflicting declaration of C function 'constexpr spi_ll_trans_len_cond_t operator|(spi_ll_trans_len_cond_t, spi_ll_trans_len_cond_t)'
.../tools/sdk/esp32s3/include/esp_common/include/esp_attr.h:126:35: error: conflicting declaration of C function 'constexpr spi_ll_trans_len_cond_t operator&(spi_ll_trans_len_cond_t, spi_ll_trans_len_cond_t)'

It seems that it is caused by：
...\tools\sdk\esp32c3\include\hal\esp32c3\include\hal\spi_II.h
// Flags for conditions under which the transaction length should be recorded

typedef enum {
    SPI_LL_TRANS_LEN_COND_WRBUF =   BIT(0), ///< WRBUF length will be recorded
    SPI_LL_TRANS_LEN_COND_RDBUF =   BIT(1), ///< RDBUF length will be recorded
    SPI_LL_TRANS_LEN_COND_WRDMA =   BIT(2), ///< WRDMA length will be recorded
    SPI_LL_TRANS_LEN_COND_RDDMA =   BIT(3), ///< RDDMA length will be recorded
} spi_ll_trans_len_cond_t;
FLAG_ATTR(spi_ll_trans_len_cond_t)

It seems that it is caused by:
...\tools\sdk\esp32s3\include\esp_common\include\esp_attr.h

// This allows using enum as flags in C++
// Format: FLAG_ATTR(flag_enum_t)
#ifdef __cplusplus

// Inline is required here to avoid multiple definition error in linker
 #define FLAG_ATTR_IMPL(TYPE, INT_TYPE) \
 FORCE_INLINE_ATTR constexpr TYPE operator~ (TYPE a) { return (TYPE)~(INT_TYPE)a; } \
 FORCE_INLINE_ATTR constexpr TYPE operator| (TYPE a, TYPE b) { return (TYPE)((INT_TYPE)a | (INT_TYPE)b); } \
 FORCE_INLINE_ATTR constexpr TYPE operator& (TYPE a, TYPE b) { return (TYPE)((INT_TYPE)a & (INT_TYPE)b); } \
 FORCE_INLINE_ATTR constexpr TYPE operator^ (TYPE a, TYPE b) { return (TYPE)((INT_TYPE)a ^ (INT_TYPE)b); } \
 FORCE_INLINE_ATTR constexpr TYPE operator>> (TYPE a, int b) { return (TYPE)((INT_TYPE)a >> b); } \
 FORCE_INLINE_ATTR constexpr TYPE operator<< (TYPE a, int b) { return (TYPE)((INT_TYPE)a << b); } \
 FORCE_INLINE_ATTR TYPE& operator|=(TYPE& a, TYPE b) { a = a | b; return a; } \
 FORCE_INLINE_ATTR TYPE& operator&=(TYPE& a, TYPE b) { a = a & b; return a; } \
 FORCE_INLINE_ATTR TYPE& operator^=(TYPE& a, TYPE b) { a = a ^ b; return a; } \
 FORCE_INLINE_ATTR TYPE& operator>>=(TYPE& a, int b) { a >>= b; return a; } \
 FORCE_INLINE_ATTR TYPE& operator<<=(TYPE& a, int b) { a <<= b; return a; }

 #define FLAG_ATTR_U32(TYPE) FLAG_ATTR_IMPL(TYPE, uint32_t)
 #define FLAG_ATTR FLAG_ATTR_U32

 #else
#define FLAG_ATTR(TYPE)
#endif

[VojtechBartoska]

@OekoSolveMG Just to let you know, milestone have been changed to 3.0.0 release. We need to investigate this a bit more, we have eyes on it :)

[strange-v]

Hi @VojtechBartoska. Have you heard any news about the topic?
Native support of W5500 would be a great feature.

[VojtechBartoska]

Hi @strange-v, we don't have any updates yet, this is still waiting for us in the backlog.

[me-no-dev]

We will be looking into this with the IDF ETH team this and next week. SPI ETH should use Arduino's SPI driver, or it's separate SPI bus

[yellobyte]

We will be looking into this with the IDF ETH team this and next week. SPI ETH should use Arduino's SPI driver, or it's separate SPI bus

Pls speed things up folks, native W5500 support is in demand. As board creator I do my part, please do yours as well. Thanks

[atanisoft]

Pls speed things up folks, native W5500 support is in demand. As board creator I do my part, please do yours as well. Thanks

Being demanding is a quick way to get nowhere. There is also absolutely nothing preventing you from using the underlying ESP-IDF code which already supports the functionality you are demanding, which is what a lot of people do when Arduino-esp32 doesn't provide an "arduino" wrapper.

https://github.com/espressif/esp-idf/blob/release/v4.4/examples/ethernet/basic/main/ethernet_example_main.c is a rather generic example for how to use the esp_eth component from ESP-IDF. It doesn't show specifically using the W5500 but you can use it by using esp_eth_phy_new_w5500 instead of the similarly named calls to initialize the interface.

Note that the above example should work pretty much as-is in arduino-esp32.

[yellobyte]

Pls speed things up folks, native W5500 support is in demand. As board creator I do my part, please do yours as well. Thanks

Being demanding is a quick way to get nowhere. There is also absolutely nothing preventing you from using the underlying ESP-IDF code which already supports the functionality you are demanding, which is what a lot of people do when Arduino-esp32 doesn't provide an "arduino" wrapper.

https://github.com/espressif/esp-idf/blob/release/v4.4/examples/ethernet/basic/main/ethernet_example_main.c is a rather generic example for how to use the esp_eth component from ESP-IDF. It doesn't show specifically using the W5500 but you can use it by using esp_eth_phy_new_w5500 instead of the similarly named calls to initialize the interface.

Note that the above example should work pretty much as-is in arduino-esp32.

Granted. However, the situation obviously doesn't allow the board getting added to the official board lib as stated in platformio/platform-espressif32#1123. Which is disappointing for me as board creator.

[atanisoft]

However, the situation obviously doesn't allow the board getting added to the official board lib as stated in platformio/platform-espressif32#1123. Which is disappointing for me as board creator.

It doesn't need to be any board library list... the support IS already there and available for those user(s) who want to use it. Perhaps wrapping it with an "arduino" like wrapper is a solution for your board library.

[yellobyte]

However, the situation obviously doesn't allow the board getting added to the official board lib as stated in platformio/platform-espressif32#1123. Which is disappointing for me as board creator.

It doesn't need to be any board library list... the support IS already there and available for those user(s) who want to use it. Perhaps wrapping it with an "arduino" like wrapper is a solution for your board library.

For the time being I provide board description files *.json with my software examples and suggest to use Ethernet(2) lib which is sufficient for most applications. It's a bit like chicken-egg problem I guess. It might get solved with ESP32 Arduino Core 3.0.0 as indicated.

[me-no-dev]

SPI Ethernet is not a simple task to add to Arduino... the driver needs to be able to use Arduino's SPI and IDF's SPI (depending on configuration). We are not obliged to cater to all boards that users make and somehow move our timelines to fit such boards. As @atanisoft said, IDF support is there and you are free to use it or add it as example. If we add "Arduino" support, we must make sure that it's done properly and that users will not shoot themselves in the foot. Tomorrow afternoon we will have a meeting with IDF's SPI-ETH team and discuss how we can move forward.

[yellobyte]

Thanks folks for all your constant efforts and energy you put into this. I acknowledge the amount of work necessary to make it all waterproof for Arduino users. Hope you'll find a way to move successfully forward. Regards

[kapyaar]

@me-no-dev Another show of hand for interest in w5500 spi support. I know it takes time. I have a quick question. I got the spi part to work, and the only thing that is kinda messing up is the secure client. There are some cases were I could make ethernet client work with sslclient wrapper, but it seems to require custom certs. Wificlientsecure has the option to setInsecure which is a great tool for testing. Is there any known way / inside example that can guide us in somehow using the same wrapper for wificlientsecure to be used for the ethernet wrapper?

I downloaded the arduinobearssl library, and got it to work on its own, but when I put it in a bigger project, there are too many conflicts from original esp32 files, causing "first defined here" errors.

I am sure your plate is full. Until the full spi support sees day light, any small help/ guidance is much appreciated.

Regardless, thank you :)

UPDATE: I got a working code based on what atanisoft mentioned, and with this, w5500 initializes, gets ip, can connect to client etc. Now, like what oekeSolveMG mentioned, If something using arduino spi is used along side, The SPI hardware (I am testing with an lcd display) seems to work, but the ethernet part loses connectivity. For sake of testing, I downloaded the mod'ed files submitted by @OekoSolveMG OekoSolveMG (compiler throws an undefined error on spi.cpp#L163 (SPI.setSpiDeviceHandle(ETH.getSpiDeviceHandle(), ETH.getSpiHost());) Sorry, no compile errors. But the second spi instance still does not work. One thing I noticed is that, If I call the ethernet action first, and then the lcd, the first spi keeps working, but the second does not. I will keep troubeshooting.

UPDATE2: With the updated files from PR by @OekoSolveMG , there are no assert errors or anything happening, but somehow the Arduino SPI is not working as intended. See image of what the screen looks like.

https://ibb.co/XSDDxCD

Any suggestions to try much appreciated.

[kapyaar]

SPI Ethernet is not a simple task to add to Arduino... the driver needs to be able to use Arduino's SPI and IDF's SPI (depending on configuration). We are not obliged to cater to all boards that users make and somehow move our timelines to fit such boards. As @atanisoft said, IDF support is there and you are free to use it or add it as example. If we add "Arduino" support, we must make sure that it's done properly and that users will not shoot themselves in the foot. Tomorrow afternoon we will have a meeting with IDF's SPI-ETH team and discuss how we can move forward.

@me-no-dev Any chance you could or someone knowledgeable on both SPI drivers check on the files included in this pull, and make it work with any second SPI hardware that uses esp32 arduino SPI? The above pull seems pretty close. I understand that releasing the feature is one thing. But something that works for the community can only do good, that by the time when you really get to the actual release, a lot of us could test and cover more ground. I got the spi display library I needed converted to use esp-idf spi, and that seems to work for my needs, but being able to use SPI.h would be a good step forward.

[me-no-dev]

SPI Ethernet is now supported in 3.0.0

[VojtechBartoska]

covered in this Pull Request: #8712