Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build with 2.5.0-dev is about 200k larger compared to 2.4.2 #5476

Closed
3 of 6 tasks
TD-er opened this issue Dec 11, 2018 · 62 comments
Closed
3 of 6 tasks

Build with 2.5.0-dev is about 200k larger compared to 2.4.2 #5476

TD-er opened this issue Dec 11, 2018 · 62 comments
Assignees
Milestone

Comments

@TD-er
Copy link
Contributor

TD-er commented Dec 11, 2018

Basic Infos

  • This issue complies with the issue POLICY doc.
  • I have read the documentation at readthedocs and the issue is not addressed there.
  • I have tested that the issue is present in current master branch (aka latest git).
  • I have searched the issue tracker for a similar issue.
  • If there is a stack dump, I have decoded it.
  • I have filled out all fields below.

Platform

  • Hardware: [ESP-12|ESP8285 device|other]
  • Core Version: [2.5.0-beta1]
  • Development Env: [Platformio] (test with own PIO platform )
  • Operating System: [Windows]

Settings in IDE

  • Module: [Generic ESP8266 Module|other]
  • Flash Mode: [dio]
  • Flash Size: [4MB/1MB]
  • lwip Variant: [v2 Lower Memory]

Problem Description

I was testing to see if I could build ESPeasy with core 2.5.0-beta1 using PlatformIO.

Almost all versions I tested of our project were either unable to build due to exceeding max_size limit, or have become so large they will be useless for their intended purpose.

As an example the minimal_ESP8266_1024_OTA env in our project:

ATA:    [=====     ]  54.8% (used 44852 bytes from 81920 bytes)
PROGRAM: [========= ]  92.8% (used 828304 bytes from 892912 bytes)

The last Travis build of the same environment using core 2.4.2 was:

DATA:    [=====     ]  54.7% (used 44836 bytes from 81920 bytes)
PROGRAM: [=======   ]  69.0% (used 616192 bytes from 892912 bytes)

Am I missing something very obvious?
Like including some library twice (maybe moved into the core) ?

Is it something I am doing wrong with creating a PlatformIO environment?
I just made a .tar.gz from the 2.5.0-beta1 tag and uploaded it to my server and mentioned it in the manifest.json.

@vascojdb
Copy link

@TD-er , as for now you can go to your Arduino settings and change the extra board manager to:
http://arduino.esp8266.com/versions/2.4.2/package_esp8266com_index.json

This will force you to use the previous one, while the new one is not stable yet (it is still a development branch)

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

@vascojdb It was my intention to make a test build using 2.5.0, just to see what issues would arise.
So changing it back is totally not what I wanted to test here.
It is not intended to be used in production environments, it is a test build.
Also, I am not using the Arduino environment, but PlatformIO.

One of the issues I see is the rather big jump in binary size.

@devyte
Copy link
Collaborator

devyte commented Dec 11, 2018

@TD-er what happens if you disable exceptions?

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

@devyte How do I disable them?
Some define is needed then?

@devyte
Copy link
Collaborator

devyte commented Dec 11, 2018

( '.menu.exception.disabled', 'Disabled' ),

Notice the -lstdc++-nox as well, as opposed to -lstdc++ in the enabled case.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

Then I get a lot of these (and still too big, will also test the "buildable" minimal version too)

Linking .pioenvs\test_ESP8266_4096_VCC\firmware.elf
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: .pioenvs\test_ESP8266_4096_VCC\firmware.elf section `.irom0.text' will not fit in region `irom0_0_seg'
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: .pioenvs\test_ESP8266_4096_VCC\firmware.elf section `.text' will not fit in region `iram1_0_seg'
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(class_type_info.o): In function `std::type_info::operator==(std::type_info const&) const':
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv117__class_type_info11__do_upcastEPKS0_PKvRNS0_15__upcast_resultE+0x22): dangerous relocation: call0: call target ou
t of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv117__class_type_info10__do_catchEPKSt9type_infoPPvj+0x29): dangerous relocation: call0: call target out of range: st

rcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv117__class_type_info12__do_dyncastEiNS0_10__sub_kindEPKS0_PKvS3_S5_RNS0_16__dyncast_resultE+0x23): dangerous relocat
ion: call0: call target out of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv117__class_type_info12__do_dyncastEiNS0_10__sub_kindEPKS0_PKvS3_S5_RNS0_16__dyncast_resultE+0x3d): dangerous relocat
ion: call0: call target out of range: strcmp
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(si_class_type_info.o): In function `std::type_info::operator==(std::type_info const&) const':
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv120__si_class_type_info12__do_dyncastEiNS_17__class_type_info10__sub_kindEPKS1_PKvS4_S6_RNS1_16__dyncast_resultE+0x2
d): dangerous relocation: call0: call target out of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv120__si_class_type_info12__do_dyncastEiNS_17__class_type_info10__sub_kindEPKS1_PKvS4_S6_RNS1_16__dyncast_resultE+0x7
e): dangerous relocation: call0: call target out of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv120__si_class_type_info20__do_find_public_srcEiPKvPKNS_17__class_type_infoES2_+0x25): dangerous relocation: call0: c
all target out of range: strcmp
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(tinfo.o): In function `std::type_info::operator==(std::type_info const&) const':
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNKSt9type_info10__do_catchEPKS_PPvj+0x19): dangerous relocation: call0: call target out of range: strcmp
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(eh_personality.o): In function `read_encoded_value_with_base':
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:257:(.text._ZL28read_encoded_value_with_basehjPKhPj+0xe1): dangerous relocation: call0: call target out of range: abort
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(eh_personality.o): In function `size_of_encoded_value':
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:88:(.text._ZL15get_ttype_entryP16lsda_header_infom+0x42): dangerous relocation: call0: call target out of range: abort
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(eh_personality.o): In function `base_of_encoded_value':
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:114:(.text._ZL21base_of_encoded_valuehP15_Unwind_Context+0x33): dangerous relocation: call0: call target out of range: _Unwind_GetTextRelBase
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:116:(.text._ZL21base_of_encoded_valuehP15_Unwind_Context+0x3b): dangerous relocation: call0: call target out of range: _Unwind_GetDataRelBase
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:118:(.text._ZL21base_of_encoded_valuehP15_Unwind_Context+0x43): dangerous relocation: call0: call target out of range: _Unwind_GetRegionStart
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:120:(.text._ZL21base_of_encoded_valuehP15_Unwind_Context+0x49): dangerous relocation: call0: call target out of range: abort
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(eh_personality.o): In function `parse_lsda_header':
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/eh_personality.cc:55:(.text._ZL17parse_lsda_headerP15_Unwind_ContextPKhP16lsda_header_info+0x1b): dangerous relocation: call0: call target out of range
: _Unwind_GetRegionStart

Linking .pioenvs\test_ESP8266_4096_VCC\firmware.elf
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: .pioenvs\test_ESP8266_4096_VCC\firmware.elf section `.irom0.text' will not fit in region `irom0_0_seg'
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: .pioenvs\test_ESP8266_4096_VCC\firmware.elf section `.text' will not fit in region `iram1_0_seg'
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(class_type_info.o): In function `std::type_info::operator==(std::type_info const&) const':
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv117__class_type_info11__do_upcastEPKS0_PKvRNS0_15__upcast_resultE+0x22): dangerous relocation: call0: call target ou
t of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv117__class_type_info10__do_catchEPKSt9type_infoPPvj+0x29): dangerous relocation: call0: call target out of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv117__class_type_info12__do_dyncastEiNS0_10__sub_kindEPKS0_PKvS3_S5_RNS0_16__dyncast_resultE+0x23): dangerous relocat
ion: call0: call target out of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv117__class_type_info12__do_dyncastEiNS0_10__sub_kindEPKS0_PKvS3_S5_RNS0_16__dyncast_resultE+0x3d): dangerous relocat
ion: call0: call target out of range: strcmp
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(si_class_type_info.o): In function `std::type_info::operator==(std::type_info const&) const':
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv120__si_class_type_info12__do_dyncastEiNS_17__class_type_info10__sub_kindEPKS1_PKvS4_S6_RNS1_16__dyncast_resultE+0x2
d): dangerous relocation: call0: call target out of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv120__si_class_type_info12__do_dyncastEiNS_17__class_type_info10__sub_kindEPKS1_PKvS4_S6_RNS1_16__dyncast_resultE+0x7
e): dangerous relocation: call0: call target out of range: strcmp
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNK10__cxxabiv120__si_class_type_info20__do_find_public_srcEiPKvPKNS_17__class_type_infoES2_+0x25): dangerous relocation: call0: c
all target out of range: strcmp
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(tinfo.o): In function `std::type_info::operator==(std::type_info const&) const':
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/typeinfo:124:(.text._ZNKSt9type_info10__do_catchEPKS_PPvj+0x19): dangerous relocation: call0: call target out of range: strcmp
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(eh_personality.o): In function `read_encoded_value_with_base':
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:257:(.text._ZL28read_encoded_value_with_basehjPKhPj+0xe1): dangerous relocation: call0: call target out of range: abort
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(eh_personality.o): In function `size_of_encoded_value':
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:88:(.text._ZL15get_ttype_entryP16lsda_header_infom+0x42): dangerous relocation: call0: call target out of range: abort
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(eh_personality.o): In function `base_of_encoded_value':
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:114:(.text._ZL21base_of_encoded_valuehP15_Unwind_Context+0x33): dangerous relocation: call0: call target out of range: _Unwind_GetTextRelBase
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:116:(.text._ZL21base_of_encoded_valuehP15_Unwind_Context+0x3b): dangerous relocation: call0: call target out of range: _Unwind_GetDataRelBase
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:118:(.text._ZL21base_of_encoded_valuehP15_Unwind_Context+0x43): dangerous relocation: call0: call target out of range: _Unwind_GetRegionStart
d:/ivan/projects/arduinoesp/toolchain/dl/gcc-xtensa/libstdc++-v3/../libgcc/unwind-pe.h:120:(.text._ZL21base_of_encoded_valuehP15_Unwind_Context+0x49): dangerous relocation: call0: call target out of range: abort
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/lib\libsupc++.a(eh_personality.o): In function `parse_lsda_header':
d:\ivan\projects\arduinoesp\toolchain\dl\gcc-xtensa\build-2\xtensa-lx106-elf\libstdc++-v3\libsupc++/../../../../libstdc++-v3/libsupc++/eh_personality.cc:55:(.text._ZL17parse_lsda_headerP15_Unwind_ContextPKhP16lsda_header_info+0x1b): dangerous relocation: call0: call target out of range
: _Unwind_GetRegionStart

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

I guess this one is related: #5423

And according to the description post here, I should rebuild the toolchain?

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 11, 2018

You need to update it with tools/get.py.

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 11, 2018

You can disable exceptions, and also select the lwIP 'no features' variant.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

You need to update it with tools/get.py.

Done and still these errors (ran a clean build too)

Linking .pioenvs\minimal_ESP8266_1024_OTA\firmware.elf
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: .pioenvs\minimal_ESP8266_1024_OTA\firmware.elf section `.text' will not fit in region `iram1_0_seg'

The files in the toolchain-xtensa/bin are still dating from 2015, so I guess that's still the issue.

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 11, 2018

Is your local repository even with current master ?

@aaron-neal
Copy link

@TD-er

I personally removed the entire platform folder then used

platform = https://github.com/platformio/platform-espressif8266.git#feature/stage 

and

build_flags = -DNDEBUG -fno-exceptions -lstdc++-nox -DPIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY_LOW_FLASH

In my platformio.ini file. Everything compiles fine and runs fine.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

@porkyneal With those settings, you're building with core 2.4.2.
I try to build core 2.5.0 beta1.

@d-a-v This repository you mean, I assume?
I downloaded from this tag: https://github.com/esp8266/Arduino/tree/2.5.0-beta1

After diner, I will also test to build in Linux to see if it makes a difference.

@aaron-neal
Copy link

aaron-neal commented Dec 11, 2018

@porkyneal With those settings, you're building with core 2.4.2.
I try to build core 2.5.0 beta1.

@TD-er Did you try? Note the #feature/stage after the URL, this makes a difference

With those settings I have the very latest GIT being compiled, admittedly not the exact 2.5.0.beta.1.
It is pulling the latest master of this repository. Which as it stands is roughly 2.5.0 beta 1.

See Using Arduino Framework with Staging version

@devyte
Copy link
Collaborator

devyte commented Dec 11, 2018

Don't rely on the tag as it currently stands. I expect to have to delete and remake it as soon as the currently ongoing work is finished.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

@porkyneal I just started a build with the feature/stage on a (slow) linux machine, so I can see it after diner.
If that doesn't result in a 2.5.0 dev build, I will try with a new fetch like devyte suggested.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

I have tried building using the staging version, with the same results.
So at least my initial test (using a self defined platform + download) was done "right" so it seems.
And the stage is indeed a core 2.5.0-dev build.
But the size is still way too big.
After that I tried building with the exception off like suggested before and also with running get.py. (update download took forever)
With exception code excluded, it is not possible to build, with the same errors as posted before.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

Deleting stuff, getting it again, etc. and using the following flags:

build_flags               = -D BUILD_GIT='"${sysenv.TRAVIS_TAG}"'
                            -DNDEBUG
                            -DVTABLES_IN_FLASH
                            -fno-exceptions
                            -lstdc++-nox
                            -DPIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY
                            -DCORE_2_5_0

The last one to handle some parts in the code that have to change between core 2.4.2 and 2.5.0

The minimal build I started with, is now:

DATA:    [=====     ]  54.5% (used 44664 bytes from 81920 bytes)
PROGRAM: [=======   ]  71.5% (used 638640 bytes from 892912 bytes)

It is still about 17k larger compared to core 2.4.2.
And still too big for our intended purpose, but that's another issue :)

But when building any other env. in our project I get this:

Linking .pioenvs\normal_ESP8266_4096\firmware.elf
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: .pioenvs\normal_ESP8266_4096\firmware.elf section `.text' will not fit in region `iram1_0_seg'
collect2.exe: error: ld returned 1 exit status
*** [.pioenvs\normal_ESP8266_4096\firmware.elf] Error 1

Running objdump -t firmware.elf I see almost any function name present.
objdump -h firmware.elf from the minimal build (that one finished building):

firmware.elf:     file format elf32-little

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .data         0000074c  3ffe8000  3ffe8000  000000e0  2**4
                  CONTENTS, ALLOC, LOAD, DATA
  1 .noinit       00000004  3ffe874c  3ffe874c  0000082c  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  2 .irom0.text   00092194  40201010  40201010  00009e00  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  3 .text         00007d6c  40100000  40100000  00002094  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  4 .rodata       00001864  3ffe8750  3ffe8750  00000830  2**4
                  CONTENTS, ALLOC, LOAD, DATA
  5 .bss          00008ec8  3ffe9fb8  3ffe9fb8  00002098  2**4
                  ALLOC
  6 .comment      00001c28  00000000  00000000  0009bf94  2**0
                  CONTENTS, READONLY
  7 .xtensa.info  00000038  00000000  00000000  0009dbbc  2**0
                  CONTENTS, READONLY
....

The text part being very close to 32k (0x7D6C = 32108 = 660 bytes left) makes it likely we will cross that one in no-time.

What else can I do to make the iram usage smaller?

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 11, 2018

try with -DPIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY_LOW_FLASH

@TD-er
Copy link
Contributor Author

TD-er commented Dec 11, 2018

-DPIO_FRAMEWORK_ARDUINO_LWIP2_LOW_MEMORY_LOW_FLASH

Same result:

Linking .pioenvs\normal_ESP8266_4096\firmware.elf
c:/users/gijs/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld.exe: .pioenvs\normal_ESP8266_4096\firmware.elf section `.text' will not fit in region `iram1_0_seg'
collect2.exe: error: ld returned 1 exit status
*** [.pioenvs\normal_ESP8266_4096\firmware.elf] Error 1
 [ERROR] Took 61.85 seconds
Environment normal_ESP8266_4096                 [ERROR]
 [ERROR] Took 61.86 seconds

Edit:
Tested on the minimal build too and still the same size for .text

@devyte devyte added this to the 2.5.0 milestone Dec 12, 2018
@devyte devyte assigned earlephilhower and d-a-v and unassigned earlephilhower Dec 12, 2018
@earlephilhower
Copy link
Collaborator

The flash size increasing by 17K, what is that compared to? Flash size is definitely not a design point, so it doesn't surprise me that there is some increase to go with the added functionality. The exceptions stuff you seem to have sorted, so I don't believe it's related to that (but might be things like the floating point support in printf, etc. which weren't in older releases).

As for the IRAM(interrupt) issue, that should be looked at in both your code and the core. There's not much space to begin with and even the ESP SDK itself continues to increase in size there.

I've given it an honest try, but as I'm not an Atom user, PlatformIO user, or a ESPEasy user, I can't for the life of me figure out how to build what you're building using the tools you're using and the updated bits (obviously in flux) that have to swap out the standard stuff. Following the Windows guide online, I got as far as getting Atom on my Linux box, getting PIO, and installing the 8266 branch from a .ZIP of the GH branch you referenced, but in addition to a espressif8266@src-eeabd... directory I see it snarfed a couple other 8266 and even ESP32 dirs in ~/.platformio. And it's been building something like 10 different versions of the ELF in the background as I've been typing.

What you can do is increase the space allocated in the ld* file to the IRAM region (to 0x9000 or so from 0x8000). That ELF won't run of course, but you will get a successful link that you will be able to look at. At that point, you can do an objdump of the .text section to get function names and sizes and can compare to your prior builds to see if there's some standout code section that grew or appeared. I can think of ~500bytes of low hanging fruit that can be moved to HEAP from IRAM, but more detailed analysis would be needed for more effective results.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 12, 2018

Thanks, I will do that this morning, to get an idea what is eating up all the iRAM.

As Windows + PlatformIO is not in your normal Workflow, then you're far away from your normal comfort zone, I assume :)
And up for a very steep learning curve if you're planning to get to know all their quirks.
I do both, Windows and Linux, with the same end-results. So at least Windows can be left out of the equation ;)

I will play around with the objdump, which looks to me like a very powerful tool, but I am yet unfamiliar with it.
Do you have some link to a description/example where objdump was used to track down these iRAM squatters? And what is causing code to be placed in iRAM by the compiler/linker?
I know about the IRAM_ATTR flag which is meant to place executable code into iRAM and there is ICACHE_FLASH_ATTR which is meant to do the opposite. (am I right on that?)

As far as I know, we are using ESP8266_NONOS_SDK, so I guess I need to add ICACHE_FLASH_ATTR to a lot of functions?
In ESP8266_RTOS_SDK the use of this ICACHE_FLASH_ATTR seems to be the default, so isn't needed there. Can we make it default too in the NonOS SDK?

So I guess my first thing to do is to find the biggest users of the iRAM and then attribute them with ICACHE_FLASH_ATTR (as long as the code is not being used on time-critical parts like interrupt handlers. (which our code hardly does)

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 12, 2018

I compiled vanilla / git head ESPEasy with arduino IDE.
It indeed doesn't fit with 2.5.0-beta1 (no exception, lwip-1.4 or lwip-2 (no features/low flash)).
With 2.4.2 I have

Sketch uses 791036 bytes (75%) of program storage space. Maximum is 1044464 bytes.
Global variables use 40804 bytes (49%) of dynamic memory,
leaving 41116 bytes for local variables. Maximum is 81920 bytes.

I have both 2.4.2 and 2.5.0-beta1 map files.
We need an (opensource) parser to compare these map files.
I found this freeware/closed source map file browser which works very well.
Here are the 2.4.2 and 2.5.0 parsed output.

2.4.2-map-sorted.txt.gz
2.5.0-beta1-map-sorted.txt.gz

@TD-er
Copy link
Contributor Author

TD-er commented Dec 12, 2018

I've given it an honest try, but as I'm not an Atom user, PlatformIO user, or a ESPEasy user, I can't for the life of me figure out how to build what you're building using the tools you're using and the updated bits (obviously in flux) that have to swap out the standard stuff. Following the Windows guide online, I got as far as getting Atom on my Linux box, getting PIO, and installing the 8266 branch from a .ZIP of the GH branch you referenced, but in addition to a espressif8266@src-eeabd... directory I see it snarfed a couple other 8266 and even ESP32 dirs in ~/.platformio. And it's been building something like 10 different versions of the ELF in the background as I've been typing.

About building it, you may want to have a look at the build script I made in the tools directory of ESPeasy.
In comments at the top is a list of dependencies, which have to be met (Ubuntu/Debian).
It will create a Python virtualenv, so it will not mess up your system.
I guess building with that script may be quite easy to do and the .elf files will be in the .pioenv dir in the ESPeasy branch.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 12, 2018

I just added the 2.4.2 built files.

N.B. there is also a 'core 2.4.1' version included, since we have that also in the nightly builds due to PWM issues with 2.4.2

Edit:
N.B. the "dev" builds should be the biggest ones, since they have the most plugins included.

@earlephilhower
Copy link
Collaborator

Got 'em, thanks.

@earlephilhower
Copy link
Collaborator

The first big standouts I'm seeing are the 2 functions: onSdaChange and onSclChange.

Combined they are 1.3KB of code space, and they are only present in 2.5.0-b1.

There was some I2C slave support added in 2.5.0-dev, but I was not following it much. @d-a-v or @devyte , any ideas? These are big functions now, where there were none before.

@earlephilhower
Copy link
Collaborator

Also in the same vein as this, there are twi_* functions of ~ 768bytes. Also included with the I2C changes, it seems.

@earlephilhower
Copy link
Collaborator

< = 2.4.2 code, > = 2.5.0-b1 code
Here's my first pass command line-only compare of the function sizes and names. Minor differences aren't a big deal, and it doesn't line up perfectly, but it's a good way to get the feel:

# objdump -t -j .text old.elf  | sort -k2 > a
# objdump -t -j .text new.elf  | sort -k2 > b
# diff -y a b

firmware.elf:     file format elf32-little                      firmware.elf:     file format elf32-little
00000018 .hidden __ashrdi3                                      00000018 .hidden __ashrdi3
00000000 .hidden __divsf3                                       00000000 .hidden __divsf3
00000000 .hidden __eqdf2                                        00000000 .hidden __eqdf2
00000000 .hidden __eqsf2                                        00000000 .hidden __eqsf2
00000000 .hidden __fixsfsi                                      00000000 .hidden __fixsfsi
00000000 .hidden __floatdisf                                    00000000 .hidden __floatdisf
00000000 .hidden __floatundisf                                  00000000 .hidden __floatundisf
00000000 .hidden __gedf2                                        00000000 .hidden __gedf2
00000000 .hidden __gesf2                                        00000000 .hidden __gesf2
00000000 .hidden __gtdf2                                        00000000 .hidden __gtdf2
00000000 .hidden __gtsf2                                        00000000 .hidden __gtsf2
00000000 .hidden __ledf2                                        00000000 .hidden __ledf2
00000000 .hidden __lesf2                                        00000000 .hidden __lesf2
00000000 .hidden __ltdf2                                        00000000 .hidden __ltdf2
00000000 .hidden __ltsf2                                        00000000 .hidden __ltsf2
00000049 .hidden __modsi3                                       00000049 .hidden __modsi3
00000000 .hidden __mulsf3                                     <
00000000 .hidden __nedf2                                        00000000 .hidden __nedf2
00000000 .hidden __nesf2                                        00000000 .hidden __nesf2
00000000 .hidden __unorddf2                                     00000000 .hidden __unorddf2
00000000 .hidden __unordsf2                                     00000000 .hidden __unordsf2
00000000 .text                                                  00000000 .text
0000003b Cache_Read_Disable_2                                   0000003b Cache_Read_Disable_2
0000002a Cache_Read_Enable_2                                    0000002a Cache_Read_Enable_2
00000032 Cache_Read_Enable_New                                  00000032 Cache_Read_Enable_New
0000000b GetAccess                                              0000000b GetAccess
00000052 NMI_Handler                                            00000052 NMI_Handler
00000020 NmiTimSetFunc                                          00000020 NmiTimSetFunc
0000000b RC_GetAckRate                                          0000000b RC_GetAckRate
00000014 RC_GetAckTime                                          00000014 RC_GetAckTime
0000003a RC_GetBlockAckTime                                     0000003a RC_GetBlockAckTime
00000036 RC_GetCtsTime                                          00000036 RC_GetCtsTime
0000000b RC_GetRtsRate                                          0000000b RC_GetRtsRate
SYMBOL TABLE:                                                   SYMBOL TABLE:
00000042 Tx_Copy2Queue                                          00000042 Tx_Copy2Queue
00000000 _DebugExceptionVector                                  00000000 _DebugExceptionVector
00000000 _DoubleExceptionVector                                 00000000 _DoubleExceptionVector
00000000 _KernelExceptionVector                                 00000000 _KernelExceptionVector
00000000 _NMIExceptionVector                                    00000000 _NMIExceptionVector
00000000 _NMILevelVector                                        00000000 _NMILevelVector
00000000 _UserExceptionVector                                   00000000 _UserExceptionVector
00000000 _UserExceptionVector_1                                 00000000 _UserExceptionVector_1
00000038 _Z21Plugin_008_interrupt1v                           | 00000017 _Z9sws_isr_0v
0000002d _Z21Plugin_008_interrupt2v                           | 00000017 _Z9sws_isr_1v
00000014 _Z27Plugin_003_pulse_interrupt1v                     | 00000017 _Z9sws_isr_2v
00000014 _Z27Plugin_003_pulse_interrupt2v                     <
00000014 _Z27Plugin_003_pulse_interrupt3v                     <
00000014 _Z27Plugin_003_pulse_interrupt4v                     <
00000014 _Z9sws_isr_0v                                        <
00000014 _Z9sws_isr_1v                                        <
00000014 _Z9sws_isr_2v                                        <
000000b6 _ZN21ESPeasySoftwareSerial6rxReadEv                    000000b6 _ZN21ESPeasySoftwareSerial6rxReadEv
00000017 __attachInterrupt                                    | 0000001a _ZNSt8__detail15_List_node_base11_M_transferEPS0_S1_
                                                              > 0000004b _ZNSt8__detail15_List_node_base4swapERS0_S1_
                                                              > 0000000e _ZNSt8__detail15_List_node_base7_M_hookEPS0_
                                                              > 0000000a _ZNSt8__detail15_List_node_base9_M_unhookEv
                                                              > 0000000b _ZSt13set_terminatePFvvE
                                                              > 0000000c _ZSt17__throw_bad_allocv
                                                              > 0000000f _ZSt18_Rb_tree_decrementPSt18_Rb_tree_node_base
                                                              > 00000028 _ZSt18_Rb_tree_incrementPKSt18_Rb_tree_node_base
                                                              > 00000027 _ZSt18_Rb_tree_incrementPSt18_Rb_tree_node_base
                                                              > 0000000c _ZSt20__throw_length_errorPKc
                                                              > 0000000c _ZSt25__throw_bad_function_callv
                                                              > 000001e6 _ZSt28_Rb_tree_rebalance_for_erasePSt18_Rb_tree_node
                                                              > 000000e9 _ZSt29_Rb_tree_insert_and_rebalancebPSt18_Rb_tree_no
                                                              > 0000002f _ZStL23local_Rb_tree_decrementPSt18_Rb_tree_node_bas
                                                              > 0000002c _ZStL25local_Rb_tree_rotate_leftPSt18_Rb_tree_node_b
                                                              > 0000002c _ZStL26local_Rb_tree_rotate_rightPSt18_Rb_tree_node_
                                                              > 0000000f _ZdaPv
                                                              > 00000011 _ZdlPv
                                                              > 0000000f _Znaj
                                                              > 00000034 _Znwj
                                                              > 00000019 __attachInterrupt
000000ad __attachInterruptArg                                   000000ad __attachInterruptArg
0000008f __detachInterrupt                                      0000008f __detachInterrupt
00000034 __digitalRead                                          00000034 __digitalRead
00000060 __digitalWrite                                       | 0000005c __digitalWrite
00000000 __divsf3_aux                                           00000000 __divsf3_aux
00000004 __gdb_no_op                                            00000004 __gdb_no_op
00000000 __mulsf3_aux                                         <
0000003c __wrap_spi_flash_read                                  0000003c __wrap_spi_flash_read
00000004 _close_r                                               00000004 _close_r
00000000 _etext                                                 00000000 _etext
00000009 _fstat_r                                               00000009 _fstat_r
00000004 _lseek_r                                               00000004 _lseek_r
00000004 _open_r                                              <
00000021 _putc_r                                              <
00000004 _read_r                                                00000004 _read_r
00000000 _stext                                                 00000000 _stext
000000c3 _umm_free                                              000000c3 _umm_free
0000013b _umm_malloc                                            0000013b _umm_malloc
00000030 _write_r                                             | 00000036 _write_r
00000020 app_entry                                            | 0000000f app_entry
00000017 attachInterrupt                                      | 00000017 app_entry_redefinable
                                                              > 00000019 attachInterrupt
00000000 call_user_start                                        00000000 call_user_start
00000015 call_user_start_local                                  00000015 call_user_start_local
00000043 calloc                                                 00000043 calloc
00000008 chm_get_current_channel                                00000008 chm_get_current_channel
0000002b clockgate_watchdog                                     0000002b clockgate_watchdog
0000001b cont_can_yield                                         0000001b cont_can_yield
0000001d cont_check                                             0000001d cont_check
00000000 cont_continue                                          00000000 cont_continue
0000001a cont_get_free_stack                                    0000001a cont_get_free_stack
00000000 cont_norm                                              00000000 cont_norm
00000000 cont_resume                                            00000000 cont_resume
00000000 cont_ret                                               00000000 cont_ret
00000043 cont_run                                               00000043 cont_run
0000000b cont_wrapper                                           0000000b cont_wrapper
0000002e cont_yield                                             0000002e cont_yield
00000013 delayMicroseconds                                      00000013 delayMicroseconds
0000008f detachInterrupt                                        0000008f detachInterrupt
00000034 digitalRead                                            00000034 digitalRead
00000060 digitalWrite                                         | 0000005c digitalWrite
00000048 esf_rx_buf_alloc                                       00000048 esf_rx_buf_alloc
0000007e espShow                                                0000007e espShow
00000037 esp_crc8                                               00000037 esp_crc8
0000016c ets_timer_arm_new                                      0000016c ets_timer_arm_new
00000054 ets_timer_disarm                                       00000054 ets_timer_disarm
00000010 ets_timer_setfn                                        00000010 ets_timer_setfn
00000185 flash_gd25q32c_read_status                             00000185 flash_gd25q32c_read_status
00000188 flash_gd25q32c_write_status                            00000188 flash_gd25q32c_write_status
0000000f free                                                   0000000f free
00000004 gdb_do_break                                           00000004 gdb_do_break
00000004 gdb_init                                               00000004 gdb_init
00000004 gdb_present                                            00000004 gdb_present
000000e0 interrupt_handler                                    | 000000e4 interrupt_handler
00000008 istr$4166                                            | 00000008 istr$4205
00000007 istr$4175                                            | 00000007 istr$4214
0000000e istr$4181                                            | 0000000e istr$4220
0000000a istr$4190                                            | 0000000a istr$4229
0000000f istr$4196                                            | 0000001a istr$4235
00000052 istr$4202                                            | 0000000f istr$4241
00000011 istr$4208                                            | 00000052 istr$4247
0000000d istr$4218                                            | 00000011 istr$4253
0000000c istr$4224                                            | 0000000e istr$4263
00000021 istr$4230                                            | 00000010 istr$4269
0000000e istr$4242                                            | 00000021 istr$4275
0000001f istr$4251                                            | 0000000d istr$4281
0000000d istr$4260                                            | 0000000c istr$4287
                                                              > 00000021 istr$4293
                                                              > 0000000d istr$4299
                                                              > 00000023 istr$4305
                                                              > 0000001f istr$4320
00000013 lmacDiscardAgedMSDU                                    00000013 lmacDiscardAgedMSDU
0000000f lmacIsActive                                           0000000f lmacIsActive
00000015 lmacIsIdle                                             00000015 lmacIsIdle
00000026 lmacMSDUAged                                           00000026 lmacMSDUAged
00000057 lmacProcessAckTimeout                                  00000057 lmacProcessAckTimeout
00000014 lmacProcessAllTxTimeout                                00000014 lmacProcessAllTxTimeout
0000006f lmacProcessCollision                                   0000006f lmacProcessCollision
0000005d lmacProcessCollisions                                  0000005d lmacProcessCollisions
00000036 lmacProcessCtsTimeout                                  00000036 lmacProcessCtsTimeout
00000008 lmacProcessRtsStart                                    00000008 lmacProcessRtsStart
000000ab lmacProcessTXStartData                                 000000ab lmacProcessTXStartData
00000085 lmacProcessTxError                                     00000085 lmacProcessTxError
0000009c lmacProcessTxRtsError                                  0000009c lmacProcessTxRtsError
000000e4 lmacProcessTxSuccess                                   000000e4 lmacProcessTxSuccess
00000067 lmacProcessTxTimeout                                   00000067 lmacProcessTxTimeout
00000022 lmacRecycleMPDU                                        00000022 lmacRecycleMPDU
00000016 lmacRxDone                                             00000016 lmacRxDone
00000053 lmacSetAcParam                                         00000053 lmacSetAcParam
00000127 lmacTxFrame                                            00000127 lmacTxFrame
                                                              > 000000c8 load_non_32_wide_handler
00000029 malloc                                                 00000029 malloc
0000000f micros                                                 0000000f micros
00000022 micros64                                               00000022 micros64
00000085 millis                                                 00000085 millis
00000125 os_printf_plus                                       | 000002e1 onSclChange
                                                              > 000001d9 onSdaChange
                                                              > 00000039 onTimer
                                                              > 00000123 os_printf_plus
00000246 phy_adc_read_fast                                      00000246 phy_adc_read_fast
00000018 phy_change_channel                                     00000018 phy_change_channel
0000005f phy_get_bb_evm                                         0000005f phy_get_bb_evm
00000067 phy_get_bb_freqoffset                                  00000067 phy_get_bb_freqoffset
0000000b phy_get_mactime                                        0000000b phy_get_mactime
0000014e pm_goto_sleep                                          0000014e pm_goto_sleep
000000b5 pm_rtc_clock_cali                                      000000b5 pm_rtc_clock_cali
0000006d ppCalFrameTimes                                        0000006d ppCalFrameTimes
0000007a ppCalTxop                                              0000007a ppCalTxop
00000026 ppDequeueTxQ                                           00000026 ppDequeueTxQ
0000001f ppDiscardMPDU                                          0000001f ppDiscardMPDU
00000016 ppEnqueueRxq                                           00000016 ppEnqueueRxq
00000034 ppEnqueueTxDone                                        00000034 ppEnqueueTxDone
0000003e ppFetchTxQFirstAvail                                   0000003e ppFetchTxQFirstAvail
00000073 ppProcessTxQ                                           00000073 ppProcessTxQ
0000000f ppRecordBarRRC                                         0000000f ppRecordBarRRC
00000023 ppRollBackTxQ                                          00000023 ppRollBackTxQ
0000001b ppTxqUpdateBitmap                                      0000001b ppTxqUpdateBitmap
0000008a pp_post                                                0000008a pp_post
00000022 pp_post2                                               00000022 pp_post2
00000092 pp_soft_wdt_feed_local                                 00000092 pp_soft_wdt_feed_local
0000001b putchar                                              | 0000007b print_stack
0000002d puts                                                 <
0000000f pvPortCalloc                                         <
0000000f pvPortMalloc                                           0000000f pvPortMalloc
0000000f pvPortRealloc                                        <
00000013 pvPortZalloc                                           00000013 pvPortZalloc
00000099 rcGetRate                                              00000099 rcGetRate
000000e3 rcGetSched                                             000000e3 rcGetSched
00000015 rcReachRetryLimit                                      00000015 rcReachRetryLimit
00000002 rcUpdateDataRxDone                                     00000002 rcUpdateDataRxDone
00000059 rcUpdateRxDone                                         00000059 rcUpdateRxDone
00000093 rcUpdateTxDone                                         00000093 rcUpdateTxDone
00000183 realloc                                                00000183 realloc
00000007 register_get_phy_addr                                  00000007 register_get_phy_addr
0000001d register_phy_ops                                       0000001d register_phy_ops
00000007 sntp_get_current_timestamp                           <
0000000b sntp_time_inc                                          0000000b sntp_time_inc
00000092 spi_flash_check_wr_protect                             00000092 spi_flash_check_wr_protect
00000044 spi_flash_erase_sector                                 00000044 spi_flash_erase_sector
00000055 spi_flash_get_id                                       00000055 spi_flash_get_id
00000057 spi_flash_read                                         00000057 spi_flash_read
00000027 spi_flash_read_status                                  00000027 spi_flash_read_status
00000052 spi_flash_write                                        00000052 spi_flash_write
0000005a spi_flash_write_status                                 0000005a spi_flash_write_status
                                                              > 000000bb stopWaveform
000000aa system_deep_sleep_local_2                              000000aa system_deep_sleep_local_2
00000012 system_get_time                                        00000012 system_get_time
0000002f system_os_post                                         0000002f system_os_post
00000037 system_restart_core                                    00000037 system_restart_core
00000056 system_rtc_mem_read                                    00000056 system_rtc_mem_read
00000056 system_rtc_mem_write                                   00000056 system_rtc_mem_write
0000001f timer0_detachInterrupt                               | 00000179 timer1Interrupt
0000002b timer0_isr_handler                                   <
00000175 timer1Interrupt                                      <
00000031 timer1_detachInterrupt                               <
00000014 timer1_disable                                         00000014 timer1_disable
00000057 timer1_isr_handler                                     00000057 timer1_isr_handler
00000019 timer1_isr_init                                        00000019 timer1_isr_init
0000002e timer1_write                                           0000002e timer1_write
000000b5 trc_NeedRTS                                            000000b5 trc_NeedRTS
                                                              > 00000014 twi_delay
                                                              > 000001fa twi_onTwipEvent
                                                              > 0000003d twi_releaseBus
                                                              > 00000036 twi_reply
                                                              > 00000021 twi_reply$part$1
                                                              > 00000057 twi_stop
00000151 tx_pwr_backoff_old                                     00000151 tx_pwr_backoff_old
00000107 uart_isr                                             | 000000ff uart_isr
00000055 umm_assimilate_down                                    00000055 umm_assimilate_down
00000082 umm_assimilate_up                                      00000082 umm_assimilate_up
0000005e umm_disconnect_from_free_list                          0000005e umm_disconnect_from_free_list
0000007d umm_init                                               0000007d umm_init
0000005a umm_make_new_block$constprop$0                         0000005a umm_make_new_block$constprop$0
0000002c unsign_to_sign                                         0000002c unsign_to_sign
00000002 user_spi_flash_dio_to_qio_pre_init                     00000002 user_spi_flash_dio_to_qio_pre_init
0000000f vPortFree                                              0000000f vPortFree
0000001f wDevDisableRx                                          0000001f wDevDisableRx
000000d2 wDev_AppendRxAmpduLensBlocks                           000000d2 wDev_AppendRxAmpduLensBlocks
00000142 wDev_AppendRxBlocks                                    00000142 wDev_AppendRxBlocks
00000017 wDev_ClearTxqCollisions                                00000017 wDev_ClearTxqCollisions
0000001e wDev_ClearWaitingQueue                                 0000001e wDev_ClearWaitingQueue
0000001e wDev_DisableTransmit                                   0000001e wDev_DisableTransmit
00000036 wDev_EnableTransmit                                    00000036 wDev_EnableTransmit
0000002d wDev_GetBAInfo                                         0000002d wDev_GetBAInfo
0000000e wDev_GetTxqCollisions                                  0000000e wDev_GetTxqCollisions
0000006f wDev_Get_KeyEntry                                      0000006f wDev_Get_KeyEntry
00000061 wDev_MacTim1Arm                                        00000061 wDev_MacTim1Arm
0000001d wDev_MacTim1SetFunc                                    0000001d wDev_MacTim1SetFunc
00000023 wDev_ProcessCollision                                  00000023 wDev_ProcessCollision
00000344 wDev_ProcessFiq                                        00000344 wDev_ProcessFiq
00000058 wDev_SetFrameAckType                                   00000058 wDev_SetFrameAckType
00000047 wDev_SetWaitingQueue                                   00000047 wDev_SetWaitingQueue
00000063 wdt_feed                                               00000063 wdt_feed
00000009 xPortWantedSizeAlign                                   00000009 xPortWantedSizeAlign
00000000 xthal_set_intclear                                   <

@TD-er
Copy link
Contributor Author

TD-er commented Dec 12, 2018

I forgot to mention, in the platformIO config I used are the following disabled in 2.5.0 builds:

  • P003
  • P008
  • Servo.

@earlephilhower
Copy link
Collaborator

@d-a-v your dump was the key. Turns out the linker script was placing the C++ std libs into IRAM by default since the name was changed to enable non-exception-mode. With the 1-line PR IRAM is at 0x7c34 out of 0x8000 in the main ESPEasy build that I'm able to do, which is w/in limits.

@earlephilhower
Copy link
Collaborator

There's probably still more IRAM space savings that we can probably squeeze out, but the main issue is now cleared up. For now, @TD-er , once a new -beta2 release is done, your disabled exception builds should complete.

@PurpleAir
Copy link
Contributor

Do you know 2.5 now uses BearSSL by default? That uses quite a bit more flash.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 13, 2018

I will include some 2.5.0-beta builds with our next nightly build, so some more people will test it. (let's hope they take the "beta" label and the warnings I add serious so we will not see reports like "I have now 2 dozen units I have to manually flash back"...)

My test units running the 2.5.0 code feel a lot more stable with WiFi.
Only the loop-count is a lot lower. (was 6k - 20k, now 1k - 2k2 runs of loop() per second)
Anything added in background processes that may take more time now?
Or is more code executed from flash now, which is slower?

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 13, 2018

My test units running the 2.5.0 code feel a lot more stable with WiFi.

glad to hear!

Only the loop-count is a lot lower. (was 6k - 20k, now 1k - 2k2 runs of loop() per second)

Did you instrument only number of loop/sec ?

How much time spent in loop()'s sub tasks would be interesting to analyse too

@TD-er
Copy link
Contributor Author

TD-er commented Dec 13, 2018

@d-a-v
We have a quite elaborate timing stats page in ESPeasy ;)

For example:

Description Function #calls call/sec min (ms) Avg (ms) max (ms)
P_1_Switch input - Switch ONCE_A_SECOND 11374 2.00 0.046 0.060 0.098
P_1_Switch input - Switch TEN_PER_SECOND 113528 19.96 0.114 0.189 71.485
P_1_Switch input - Switch WRITE 27 0.00 2.252 2.316 2.426
P_1_Switch input - Switch EVENT_OUT 1718 0.30 0.046 0.062 0.094
P_1_Switch input - Switch FIFTY_PER_SECOND 562554 98.88 0.046 0.054 0.094
P_4_Environment - DHT11/12/22 SONOFF2301/7021 READ 956 0.17 2.414 7.361 8.460
P_4_Environment - DHT11/12/22 SONOFF2301/7021 ONCE_A_SECOND 5687 1.00 0.050 0.051 0.064
P_4_Environment - DHT11/12/22 SONOFF2301/7021 TEN_PER_SECOND 56764 9.98 0.041 0.047 0.068
P_4_Environment - DHT11/12/22 SONOFF2301/7021 EVENT_OUT 859 0.15 0.054 0.054 0.063
P_4_Environment - DHT11/12/22 SONOFF2301/7021 FIFTY_PER_SECOND 281277 49.44 0.046 0.048 0.068
Load File   588 0.10 1.730 2.133 2.710
Save File   7 0.00 210.251 328.267 543.279
Loop   9147325 1607.87 0.442 0.648 1717.942
Plugin call 50 p/s   281277 49.44 2.711 2.885 3.288
Plugin call 10 p/s   56764 9.98 2.965 3.121 74.395
Plugin call 10 p/s U   56764 9.98 1.025 1.166 1.323
Plugin call 1 p/s   5687 1.00 3.515 3.968 97.650
SensorSendTask()   956 0.17 5.003 50.574 68.044
sendData()   859 0.15 44.564 47.023 69.655
Compute formula   832 0.15 0.019 0.019 0.024
setNewTimerAt()   401824 70.63 0.136 0.168 0.279
hostByName()   1 0.00 34.693 34.693 34.693
WiFi.isConnected()   27111395 4765.51 0.025 0.028 0.134

Or the graphical view with color highlighting the ones > 100 msec:
image

There is also JSON output of it ;)

Oh and the extravagant amount of checks for wifi connection status is a left-over which can be lowered a lot if WiFi is stable again :)

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 13, 2018

What core version are you comparing with ?
Are you able to identify one particular call which is slower than before ?

@TD-er
Copy link
Contributor Author

TD-er commented Dec 13, 2018

I have not yet looked at such a detailed level.
The screenshot I took was using core 2.5.0
I just noticed the loop counter was much lower than before. (1607/s in the screenshot)
Another one running an older version (based on core 2.4.2):
image
By the way, loop counter in the stats probably had a 32-bit overflow, since avg is lower than min :)

I will flash a 2.4.2 build to the same Sonoff TH16 here on my desk for comparison (using same settings)

@TD-er
Copy link
Contributor Author

TD-er commented Dec 13, 2018

And a core 2.4.2 build of the same codebase, same settings, same module as the Sonoff TH16 before
image

Only running for 160 seconds now, so the loop count could increase a bit more when running longer (lot is done right after start)
It is

  • 1607/sec for core 2.5.0
  • 4807/sec for core 2.4.2

@earlephilhower
Copy link
Collaborator

Maybe it's nothing, but the calls/sec # for some of these are double the indicated amount, no?

@TD-er
Copy link
Contributor Author

TD-er commented Dec 13, 2018

Maybe it's nothing, but the calls/sec # for some of these are double the indicated amount, no?

That's correct.
If there are 2 plugins of the same plugin-ID active, it will count twice as hard.
So the 50/sec call to a plugin may then report 100/sec after a while.

@devyte
Copy link
Collaborator

devyte commented Dec 15, 2018

@earlephilhower the original issue here was bin size, and then iram. I think both have been handled. Can this be closed?
About timing and loop freq, I suggest a new issue for that.

@TD-er
Copy link
Contributor Author

TD-er commented Dec 15, 2018

I agree the size issues are indeed handled.

@devyte
Copy link
Collaborator

devyte commented Dec 15, 2018

Closing per previous comment.

@mikekgr
Copy link

mikekgr commented Jan 8, 2019

Dear Sirs, although this is a closed issue, today I compiled one of my Arduino sketch, using the master branch of the Arduino Core and I have a remarkable binary file increasing near if not even more of 200 KB. That I think is very much... Please see what can cause it.
Thanks and Best Regards,
Mike Kranidis

@PurpleAir
Copy link
Contributor

For me, setting the Arduino menu: "Tools -> Exceptions -> Disabled" solved it.

@penfold42
Copy link

Exceptions off should be the default after #5538

@devyte
Copy link
Collaborator

devyte commented Jan 9, 2019

Correct, there is more work needed before they can be default-enabled.

@mikekgr
Copy link

mikekgr commented Jan 10, 2019

Dear Sirs, yes, you are absolute right,
The Exceptions off, did the trick !
See the two exactly the same compilations with Exceptions on and Exceptions off :
Exceptions on:
Sketch uses 520116 bytes (49%) of program storage space. Maximum is 1044464 bytes.
Global variables use 37692 bytes (46%) of dynamic memory, leaving 44228 bytes for local variables. Maximum is 81920 bytes.

Exceptions off:
Sketch uses 405372 bytes (38%) of program storage space. Maximum is 1044464 bytes.
Global variables use 37524 bytes (45%) of dynamic memory, leaving 44396 bytes for local variables. Maximum is 81920 bytes.

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants