streamFile and serveStatic for ESP8266/ESP32 boards #22

AC8L · 2021-01-26T03:47:24Z

Per your request refactored the code to server only ESP8266/ESP32 platforms. I spent whole day to create a class which works but request handlers are broken. So, ended up separating the code.

Used bufferization.

modified namespace to mime_esp to avoid conflicts with existing mimetable.h file

Added check to RequestHandlersImpl.h to include the code only on ESP8266 and ESP32 platforms.

…platforms On other platforms relevant code breaks, so it will be included only if ESP platforms is detected.

khoih-prog · 2021-01-26T20:25:49Z

Thanks for your good efforts. I'll have a more detailed look at the new PR within several days.

BTW, could you also provide the example code the test the new ESP streamFile feature to save me some time to write it to verify? I'll include it in the examples, with your credit, if the PR OK to be merged later.

AC8L · 2021-01-27T02:59:19Z

Done with the example sketch.

khoih-prog · 2021-01-27T19:46:21Z

Wow, you've spent a lot of time on the code.

I just had a preliminary look and test compiled the simplest example AdvancedWebServer and still there are many compiler errors

1. For boards different from ESP8288/ESP32, this is simple to fix by not using `mimetable.cpp` (using #if)

File mimetable.h

#ifndef __MIMETABLE_H__
#define __MIMETABLE_H__

#if (ESP32 || ESP8266)
...

#endif
#endif

File mimetable.cpp

#if (ESP32 || ESP8266)

...

#endif

The compiler errors

1.1 For SAM DUE

/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/esp_detail/mimetable.cpp:2:22: fatal error: pgmspace.h: No such file or directory
 #include "pgmspace.h"
                      ^
compilation terminated.
Error compiling for board Arduino Due (Programming Port).

1.2 AVR Mega

/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/esp_detail/mimetable.cpp:2:10: fatal error: pgmspace.h: No such file or directory
 #include "pgmspace.h"
          ^~~~~~~~~~~~
compilation terminated.
exit status 1
Error compiling for board Arduino Mega or Mega 2560.

1.3 SAMD51

/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/esp_detail/mimetable.cpp:2:10: fatal error: pgmspace.h: No such file or directory
    2 | #include "pgmspace.h"
compilation terminated.
exit status 1
Error compiling for board Adafruit ItsyBitsy M4 (SAMD51).

Teensy 4.0

/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/esp_detail/mimetable.cpp:2:22: fatal error: pgmspace.h: No such file or directory
compilation terminated.
exit status 1
Error compiling for board Teensy 4.0.

2.Even compiler errors for ESP8288/ESP32, this is also simple to fix by using only ESP libb64, and not using libb64 of this library. Could you check and fix

I'm using ESP8266 core 2.7.4 and ESP32 core 1.0.4 with Arduino IDE v1.8.13

1.1 ESP8266

/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:13: multiple definition of `base64_init_encodestate'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:44: first defined here
/home/kh/.arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/2.5.0-4-b40a506/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: core/core.a(cencode.cpp.o): in function `base64_encode_value':
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:25: multiple definition of `base64_encode_value'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:50: first defined here
/home/kh/.arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/2.5.0-4-b40a506/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: core/core.a(cencode.cpp.o): in function `base64_encode_block':
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:41: multiple definition of `base64_encode_block'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:60: first defined here
/home/kh/.arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/2.5.0-4-b40a506/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: core/core.a(cencode.cpp.o): in function `base64_encode_blockend':
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:97: multiple definition of `base64_encode_blockend'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:137: first defined here
/home/kh/.arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/2.5.0-4-b40a506/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: core/core.a(cencode.cpp.o): in function `base64_encode_chars':
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/cores/esp8266/libb64/cencode.cpp:118: multiple definition of `base64_encode_chars'; libraries/EthernetWebServer-serveStatic_dev2/libb64/cencode.c.o:/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/libb64/cencode.c:161: first defined here
collect2: error: ld returned 1 exit status
exit status 1
Error compiling for board NodeMCU 1.0 (ESP-12E Module).

ESP32

In file included from /home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/RequestHandlersImpl.h:192:0,
/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/ESP_RequestHandlersImpl.h:28:17: fatal error: Uri.h: No such file or directory
compilation terminated.
exit status 1
Error compiling for board ESP32 Dev Module.

khoih-prog · 2021-01-27T19:58:05Z

It's OK now for ESP8266, just wrap

#if !(ESP32 || ESP8266)

...
#endif

around cencode.c and cdecode.c solves the ESP compile issue.

Can you fix and update the PR?

khoih-prog · 2021-01-27T20:01:38Z

Still having issue with ESP32.

In file included from /home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/RequestHandlersImpl.h:192:0,
/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/detail/ESP_RequestHandlersImpl.h:28:17: fatal error: Uri.h: No such file or directory
compilation terminated.
exit status 1
Error compiling for board ESP32 Dev Module.

khoih-prog · 2021-01-27T20:06:26Z

Fix now by comment out #include "Uri.h" in ESP_RequestHandlersImpl.h

#include "RequestHandler.h"
#include "esp_detail/mimetable.h"
#include "FS.h"
#include "WString.h"
//#include "Uri.h"
#include <MD5Builder.h>
#include <base64.h>

Pls update the PR.

khoih-prog · 2021-01-27T20:30:40Z

The example serveStatic, I think, is written for ESP32 and deprecated SPIFFS. I'll rewrite to include LittleFS for both ESP32 and ESP8266, also remove all unnecessary #defines.

AC8L · 2021-01-27T21:18:33Z

Made all suggested changes except for cencode.h which was not necessary. Tested by compiling AdvancedWebServer for SAMD and ESP boards without errors. Feel free editing the example file. After all it will be a good example for LittleFS as well.

khoih-prog · 2021-01-27T22:20:28Z

I had to modify cencode.c and cdecode.c to solve the ESP8266 compile issue. I think you compile only for ESP32, not ESP8266.

Can you check to be sure using the latest ESP cores and Arduino IDE ?

AC8L · 2021-01-27T22:56:50Z

On ESP8266 I am getting different error:

AdvancedWebServer:183:14: error: 'class EthernetClass' has no member named 'init'
Ethernet.init (USE_THIS_SS_PIN);
^
Multiple libraries were found for "Ethernet.h"
Used: /Users/sardar/Library/Arduino15/packages/esp8266/hardware/esp8266/2.7.4/libraries/Ethernet
Not used: /Applications/Arduino.app/Contents/Java/libraries/Ethernet
exit status 1

I am using Arduino IDE 1.8.14, ESP8266 core 2.7.4.
But for ESP32 I am using 1.0.5-rc6, since it fixes the python issue on. Mac OS Big Sur I am using.

AC8L · 2021-01-27T23:05:30Z

I tested the previous error - I am getting it even with the original code on ESP8266.

khoih-prog · 2021-01-27T23:15:09Z

Everything is OK here now. You have to check your settings / installation

I'm writing an example to permit you to upload the files (e.g. page1, etc.) into LittleFS/SPIFFS so that you can use with the staticServe example. Already working, will ready for next release, with your example and PR.

Also more complicated FS_EthernetWebServer example with Web editor.

Still have to test everything to see where else it breaks.

khoih-prog · 2021-01-27T23:19:24Z

The ESP8266 compile result (IDE v1.8.13, ESP8266 core 2.7.4)

In file included from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:0:
sketch/defines.h:228:4: warning: #warning Use ESP8266 architecture [-Wcpp]
   #warning Use ESP8266 architecture
    ^
In file included from sketch/defines.h:328:0,
                 from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:
/home/kh/Arduino/libraries/Ethernet/src/Ethernet.h:200:0: warning: "UDP_TX_PACKET_MAX_SIZE" redefined [enabled by default]
 #define UDP_TX_PACKET_MAX_SIZE 24
 ^
In file included from /home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/src/ESP8266mDNS_Legacy.h:47:0,
                 from /home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/src/ESP8266mDNS.h:45,
                 from sketch/defines.h:229,
                 from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:
/home/kh/Downloads/Arduino/arduino-1.8.13/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src/WiFiUdp.h:28:0: note: this is the location of the previous definition
 #define UDP_TX_PACKET_MAX_SIZE 8192
 ^
In file included from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:0:
sketch/defines.h:329:8: warning: #warning Using Ethernet lib [-Wcpp]
       #warning Using Ethernet lib
        ^
In file included from sketch/defines.h:344:0,
                 from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:
/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/EthernetWebServer.h:94:4: warning: #warning Using Ethernet library from EthernetWebServer [-Wcpp]
   #warning Using Ethernet library from EthernetWebServer
    ^
In file included from sketch/defines.h:344:0,
                 from /home/kh/Arduino/khoih-prog_working/Testing_Libs/EthernetWebServer-serveStatic_dev2/examples/AdvancedWebServer/AdvancedWebServer.ino:44:
/home/kh/Arduino/libraries/EthernetWebServer-serveStatic_dev2/src/EthernetWebServer.h:109:4: warning: #warning SENDCONTENT_P_BUFFER_SZ using default 4 Kbytes [-Wcpp]
   #warning SENDCONTENT_P_BUFFER_SZ using default 4 Kbytes
    ^
Executable segment sizes:
IROM   : 266064          - code in flash         (default or ICACHE_FLASH_ATTR) 
IRAM   : 26904   / 32768 - code in IRAM          (ICACHE_RAM_ATTR, ISRs...) 
DATA   : 1664  )         - initialized variables (global, static) in RAM/HEAP 
RODATA : 2856  ) / 81920 - constants             (global, static) in RAM/HEAP 
BSS    : 25216 )         - zeroed variables      (global, static) in RAM/HEAP 
Sketch uses 297488 bytes (28%) of program storage space. Maximum is 1044464 bytes.
Global variables use 29736 bytes (36%) of dynamic memory, leaving 52184 bytes for local variables. Maximum is 81920 bytes.

khoih-prog · 2021-01-27T23:25:49Z

Ah ! Remember now. You have to rename the file Ethernet.h to Ethernet_8266.h in ./hardware/esp8266com/esp8266/libraries/Ethernet directory.

I think I forget to mention in the README about this to avoid conflict.

AC8L · 2021-01-27T23:28:47Z

You mentioned it in defines.h for example sketches but it is in the middle of define statements and looks like something optional :) I think mentioning it in readme will be better.

khoih-prog · 2021-01-27T23:31:35Z

OK, here it is. Will include in README later.

// Note: To rename ESP628266 Ethernet lib files to Ethernet_ESP8266.h and Ethernet_ESP8266.cpp
// In order to USE_ETHERNET_ESP8266

AC8L · 2021-01-27T23:35:28Z

Updated encode.c and cdecode.c files.

And thank you for the great example, it looks gorgeous!

khoih-prog · 2021-01-27T23:39:50Z

I think everything is ready to merge the PR now, but I'd still like to have more compiling and testing to be sure no more issue before making final decision to merge.

Can you help compile and test as many as you can, on whatever board you can? With your quick decision and good knowledge, I wish later you can help maintain this library.

I'll post the new example code for you to test as well. Almost done.

AC8L · 2021-01-27T23:41:41Z

No rush, let's test throroughly.

khoih-prog · 2021-01-27T23:56:31Z

I just post the first 2 examples

Can you test, then update your repo to avoid conflict in PR.

### Releases v1.3.0 1. Add WebServer feature to serve from LittleFS/SPIFFS for ESP32/ESP8266 with examples. Check [**streamFile and serveStatic for ESP8266/ESP32 boards #22**](#22) 2. Add examples [**serveStatic**](https://github.com/khoih-prog/EthernetWebServer/tree/master/examples/serveStatic) and [**serveStaticLoadFile**](https://github.com/khoih-prog/EthernetWebServer/tree/master/examples/serveStaticLoadFile) to use new `serveStatic()`feature 3. Add examples [**ESP32_FS_EthernetWebServer**](https://github.com/khoih-prog/EthernetWebServer/tree/master/examples/ESP32_FS_EthernetWebServer) and [**FS_EthernetWebServer**](https://github.com/khoih-prog/EthernetWebServer/tree/master/examples/FS_EthernetWebServer) to use new `serveStatic()`feature

### Major Releases v1.3.0 1. Add WebServer feature to serve from LittleFS/SPIFFS for ESP32/ESP8266 with examples. Check [**streamFile and serveStatic for ESP8266/ESP32 boards #22**](#22) 2. Add examples [**serveStatic**](https://github.com/khoih-prog/EthernetWebServer/tree/master/examples/serveStatic) and [**serveStaticLoadFile**](https://github.com/khoih-prog/EthernetWebServer/tree/master/examples/serveStaticLoadFile) to use new `serveStatic()`feature 3. Add examples [**ESP32_FS_EthernetWebServer**](https://github.com/khoih-prog/EthernetWebServer/tree/master/examples/ESP32_FS_EthernetWebServer) and [**FS_EthernetWebServer**](https://github.com/khoih-prog/EthernetWebServer/tree/master/examples/FS_EthernetWebServer) to use new `serveStatic()`feature

khoih-prog · 2021-01-28T04:49:44Z

Congrats. The new EthernetWebServer v1.3.0 has just been published, thanks to your PR and contribution.

Major Releases v1.3.0

Add WebServer feature to serve from LittleFS/SPIFFS for ESP32/ESP8266 with examples. Check streamFile and serveStatic for ESP8266/ESP32 boards #22
Add examples serveStatic and serveStaticLoadFile to use new serveStatic()feature
Add examples ESP32_FS_EthernetWebServer and FS_EthernetWebServer to use new serveStatic()feature

Your contribution has been noted in Contributions and Thanks.

Please have more tests, report issues and make more PRs.

Best Regards,

Sardar Azari added 4 commits January 25, 2021 02:36

Fixed file transfer speed of streamFile

bdbec53

Used bufferization.

mimetable files from ESP8266 implementation

58e6a0b

modified namespace to mime_esp to avoid conflicts with existing mimetable.h file

Implemented RequestHandlers for ESP hardware

272a48d

Added check to RequestHandlersImpl.h to include the code only on ESP8266 and ESP32 platforms.

Implemented streamFile and serveStatic methods for ESP8266 and ESP32 …

7dc6569

…platforms On other platforms relevant code breaks, so it will be included only if ESP platforms is detected.

Added example sketch of using serveStatic method

6292c4c

Fixed compiler errors for non-ESP boards

88333b0

Further fixing ESP compile issues

f8ee7fa

Merge branch 'master' into serveStatic_dev2

7d4d3f4

khoih-prog merged commit c8bf8fe into khoih-prog:master Jan 28, 2021

AC8L deleted the serveStatic_dev2 branch January 28, 2021 02:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

streamFile and serveStatic for ESP8266/ESP32 boards #22

streamFile and serveStatic for ESP8266/ESP32 boards #22

AC8L commented Jan 26, 2021

khoih-prog commented Jan 26, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021 •

edited

Loading

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 28, 2021 •

edited

Loading

streamFile and serveStatic for ESP8266/ESP32 boards #22

streamFile and serveStatic for ESP8266/ESP32 boards #22

Conversation

AC8L commented Jan 26, 2021

khoih-prog commented Jan 26, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

1. For boards different from ESP8288/ESP32, this is simple to fix by not using mimetable.cpp (using #if)

2.Even compiler errors for ESP8288/ESP32, this is also simple to fix by using only ESP libb64, and not using libb64 of this library. Could you check and fix

khoih-prog commented Jan 27, 2021 • edited Loading

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

AC8L commented Jan 27, 2021

khoih-prog commented Jan 27, 2021

khoih-prog commented Jan 28, 2021 • edited Loading

Major Releases v1.3.0

1. For boards different from ESP8288/ESP32, this is simple to fix by not using `mimetable.cpp` (using #if)

khoih-prog commented Jan 27, 2021 •

edited

Loading

khoih-prog commented Jan 28, 2021 •

edited

Loading