Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BenchMark]Web upload / Web Download on different FS and platform #576

Closed
luc-github opened this issue Feb 3, 2021 · 11 comments
Closed

Comments

@luc-github
Copy link
Owner

luc-github commented Feb 3, 2021

I think it would be very informative to bench the speed of upload / download a file using FTP / WebDav / Webserver
to LocalFS / SD on ESP32 and ESP8266
using MCVE to be sure the comparaison is correct

Target Speed KB/S Note
ESP32 + WEBSERVER + FAT 56.01
ESP32 + WEBSERVER + LITTLEFS 59.60
ESP8266 + WEBSERVER + LITTLEFS
ESP32 + WEBSERVER + SD native 75.06
ESP8266 + WEBSERVER + SD native
ESP32 + WEBSERVER + SD Fat 101.9 ~122.4 SDFAT 2.1.2
ESP8266 + WEBSERVER + SD Fat
ESP32 + WEBSERVER + SDIO
ESP8266 + WEBSERVER + SDIO
ESP32 + WEBDAV+ LITTLEFS
ESP8266 + WEBDAV+ LITTLEFS
ESP32 + WEBDAV+ SD native
ESP8266 + WEBDAV+ SD native
ESP32 + WEBDAV+ SD Fat
ESP8266 + WEBDAV+ SD Fat
ESP32 + WEBDAV+ SDIO
ESP8266 + WEBDAV+ SDIO
ESP32 + FTP+ LITTLEFS
ESP8266 + FTP+ LITTLEFS
ESP32 + FTP+ SD native
ESP8266 + FTP+ SD native
ESP32 + FTP+ SD Fat
ESP8266 + FTP+ SD Fat
ESP32 + FTP+ SDIO
ESP8266 + FTP+ SDIO

Others FW:

FW Target Speed KB/S Note
grblHAL ESP32 + FatFS + SDCard + WebDAV 300~350
grblHAL iMXRT1062 (Teensy4) + FatFS + SDCard + WebDAV 1127 Ethernet
grblHAL STM32F7xx + FatFS + SDCard + WebDAV 350 Ethernet

Adding with PSRAM could also be a thing to check on ESP32

Once done need to compare with speed in ESP3D / ESP3DLib and GRBL_ESP32 to see what must be improved and how
This is based on feedback ESP32 WebServer upload is very slow compare to ESP8266

Using any asyncwebserver / external webserver is not the current target so it is out of scope - the goal is to do a status of current situation to plan improvements / changes

@luc-github
Copy link
Owner Author

this PR may solve the speed issue espressif/arduino-esp32#4787 (not tested)

@luc-github
Copy link
Owner Author

luc-github commented Jan 9, 2022

Just listing files with Native ESP32 SD is slow in core 1.X, but it 10 times slower or more with core 2.X when using SPI, so it is a must to use the SDFAT2 on ESP32 with SPI
SD Native Core 1.x
SDListing-1 x

SD Native Core 2.x
SDListing-2 x

SDFat2
FAT2

@luc-github
Copy link
Owner Author

luc-github commented Mar 20, 2022

I have added a simple bench check in FS file upload abd SD File Upload in ESP3DLib 3.0 - and tested ESP32

image

same test should be done on ESP3D to see if using core 0 instead of core 1 affect the perfomance

I will test other configurations when I have time

@liqijian101
Copy link

hi luc,I test esp32s2 with sdfat2.0 in spi mode .I find that the sector size of tf card can influence the speed of upload files.

@luc-github
Copy link
Owner Author

what size did you set? what speed did you get?

@liqijian101
Copy link

A low speed card, 8g capacity, 2048ksec, Max speed 270ksec
a high speed card, 16G capacity, 64ksec, Max speed 190KSEC,
and I found that SDFAT2 does not support 32G cards

@liqijian101
Copy link

I also found that using esp32s2sdfat2.0 is faster than using 1.0

@luc-github
Copy link
Owner Author

I did configuration with compatibility with 3Dprinter FW in mind, so 4G/8G is the target, 16G may be also supported, be sur you use https://www.sdcard.org/downloads/formatter/

and use Fat32 format
I did tested 32G/64G/128G in past but long time I did not recheck the support, and definitly not with SDFat 2.x

@terjeio
Copy link

terjeio commented Aug 28, 2022

Some grblHAL speeds with FatFS + SDCard + WebDAV, using WinSCP for testing:

  • ESP32: 300-350KB/s up- and download speeds, this with my heavily modified lwIP http daemon (I have ditched the ESP32 http daemon).
  • iMXRT1062 (Teensy4) 1.1MB/s upload, 3-3.5MB/s download (with ethernet).
  • STM32F7xx: 300-350KB/s up- and download speeds. This is with a broken and slow low-level ethernet driver, there is a new available from ST and I will switch to this soon.

RP2040 (Pi Pico W) support for WebUI is on my todo list, will be interesting to see how this fares...

@luc-github
Copy link
Owner Author

@terjeio thank you I have added your figures on top

@luc-github
Copy link
Owner Author

For information @pipi61 reported #491 (comment) that esp32 + webdav + SDIO can reach 500KB/s

Additionnaly Reprap porting team, that added esp32 support to duetwifi, claim that they reach 1.2MB/s, but in case of Duet wifi, esp32 is only used as data bridge, the webserver is hosted by the main MCU, and I think the SD card is SDIO one, but communitation between ESP32 and MCU are SPI only - TBC as I never took time to dig in

But it shows that there is area of improvement 🐱

Repository owner locked and limited conversation to collaborators Nov 25, 2024
@luc-github luc-github converted this issue into discussion #1063 Nov 25, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Projects
None yet
Development

No branches or pull requests

3 participants