Add an example of going to deep sleep from wake stub (IDFGH-6564)

[igrr]

Is your feature request related to a problem? Please describe.

Deep sleep wake stubs allow small pieces of code to be executed quickly after deep sleep. For example, deep sleep wake stub may check GPIOs or read some sensor. The deep sleep stub code may need to put the chip back into deep sleep if some condition isn't met.

Describe the solution you'd like

• An API which can be called from deep sleep wake stub to go back to deep sleep
• Documentation or an example for this API

Describe alternatives you've considered

• Let the application boot and then use regular esp_deep_sleep_start function — too slow, affects power consumption
• Use ULP coprocessor instead — more difficult to program, limited set of peripherals and GPIOs can be used

Additional context

Originally the feature for the ESP32 was requested in https://www.esp32.com/viewtopic.php?t=2369, and an ESP32 specific example is available at https://gist.github.com/igrr/54f7fbe0513ac14e1aea3fd7fbecfeab. In comments to that gist, users have requested this feature to be provided for ESP32-C3

[helmut64]

I will be ready for testing and providing feedback, my stub works for the first wakeup, on the second wakeup the C3 boots and does not call the sub entry.

[tsVestor]

Same here

[baamiis]

My stub also is not executed second time. I had to revert to older version for this to work. I hope for a solution soon otherwise I am stuck with older version of IDF

[tsVestor]

@baamiis Can you name the lastest esp-idf version where it still works?

[baamiis]

4.22 release it works
4.30 it does not work

[tsVestor]

I did some further testing. In short, it boils down to the fact that the WDT resets the chip after deep sleep has been re-initiated by the wake up stub.

[helmut64]

In my testing, the wakeup is done without a reset, I believe that the WDT does a reset @tsVestor

[andrew-elder]

ESP32S3 has code that looks like

void esp_set_deep_sleep_wake_stub(esp_deep_sleep_wake_stub_fn_t new_stub)
{
#if SOC_PM_SUPPORT_DEEPSLEEP_VERIFY_STUB_ONLY
    wake_stub_fn_handler = new_stub;
#else
    REG_WRITE(RTC_ENTRY_ADDR_REG, (uint32_t)new_stub);
#endif
}

with SOC_PM_SUPPORT_DEEPSLEEP_VERIFY_STUB_ONLY set to 1, so I'm not sure that writing to RTC_ENTRY_ADDR_REG has the desired effect.

[gm-jiang]

Hi @helmut64 @tsVestor @baamiis , about the issue that ESP32-C3 wake stub works for the first wakeup, on second wakeup the C3 boots and does not call the sub entry. Please call ROM function set_rtc_memory_crc() after set wake_stub address to RTC_ENTRY_ADDR_REG. The wake stub can be executed after every deep sleep wakeup.

    // Set the pointer of the wake stub function.                                                                                                                                                          
    REG_WRITE(RTC_ENTRY_ADDR_REG, (uint32_t)&wake_stub);
    // Set rtc memory crc to RTC_MEMORY_CRC.
    set_rtc_memory_crc();
    // Go to sleep.
    CLEAR_PERI_REG_MASK(RTC_CNTL_STATE0_REG, RTC_CNTL_SLEEP_EN);
    SET_PERI_REG_MASK(RTC_CNTL_STATE0_REG, RTC_CNTL_SLEEP_EN);
    // A few CPU cycles may be necessary for the sleep to start...
    while (true) {
        ;     
    }       
    // never reaches here.

[helmut64]

@gm-jiang yes the missing "set_rtc_memory_crc ()" was the problem, it has already been discussed and solved here:
#8208

[andrew-elder]

Does anyone have equivalent instructions for an ESP32S3?

[helmut64]

My understanding is that S2/S2 also needs the set_rtc_memory_crc()
The stub code is in common with all ESP32 MCUs

[gm-jiang]

@andrew-elder @helmut64， ESP32-S3 support SOC_PM_SUPPORT_DEEPSLEEP_VERIFY_STUB_ONLY that only needs check the CRC value of stub code instead of all of the RTC memory. The stub code entry and stub code CRC could be set by S3 ROM function esp_rom_set_rtc_wake_addr((esp_rom_wake_func_t)esp_wake_stub_entry, stub_length) .

[gm-jiang]

Some codes like

#if SOC_PM_SUPPORT_DEEPSLEEP_CHECK_STUB_ONLY
    extern char _rtc_text_start[];
#if CONFIG_ESP32S3_RTCDATA_IN_FAST_MEM
    extern char _rtc_noinit_end[];
    size_t rtc_fast_length = (size_t)_rtc_noinit_end - (size_t)_rtc_text_start;
#else
    extern char _rtc_force_fast_end[];
    size_t rtc_fast_length = (size_t)_rtc_force_fast_end - (size_t)_rtc_text_start;
#endif // CONFIG_ESP32S3_RTCDATA_IN_FAST_MEM
    // Please note, the entry address of stub code is a fixed address at `_rtc_text_start`.
    esp_rom_set_rtc_wake_addr((esp_rom_wake_func_t)_rtc_text_start, rtc_fast_length);
#else
    // Set the pointer of the wake stub function.
    REG_WRITE(RTC_ENTRY_ADDR_REG, (uint32_t)&wake_stub);
    set_rtc_memory_crc();
#endif // SOC_PM_SUPPORT_DEEPSLEEP_CHECK_STUB_MEM

[helmut64]

Thank you for the advise.

[andrew-elder]

@gm-jiang - thank you! That works on my ESP32S3.

I also see that ESP32S3 uses

uint64_t rtc_time_get(void)
{
    SET_PERI_REG_MASK(RTC_CNTL_TIME_UPDATE_REG, RTC_CNTL_TIME_UPDATE);
    uint64_t t = READ_PERI_REG(RTC_CNTL_TIME0_REG);
    t |= ((uint64_t) READ_PERI_REG(RTC_CNTL_TIME1_REG)) << 32;
    return t;
}

so no need to spin/wait for the RTC reading to be populated in the registers.

[gm-jiang]

Hi @andrew-elder, is there any problem found in function rtc_time_get()?

[andrew-elder]

@gm-jiang - no problem at all that I am aware of. The code is just a little different than that used by an ESP32 in other related postings. It took me a while to figure out that ESP32S3 required a different sequence, so I posted here for others to use.

[gm-jiang]

@andrew-elder Thanks for your question, I think it's an improvement in hardware design, but needs to be double-checked.

[gm-jiang]

Hi @andrew-elder, It's been confirmed that this is a hardware improvement.

[jasaw]

I'm new to the ESP32-C3. All the above examples are fantastic for me to get started, but what's the easiest way to transmit and receive on the UART in the rtc wake stub? Any example code or pointers will be greatly appreciated. Thanks.

[gm-jiang]

Hi @jasaw , before enter deep-sleep wake stub, BootROM will do some initialization work on uart0. The
transmit and receive functions provided by ROM can work in stub. You can find these functions in components/esp_rom/include/esp_rom_uart.h.

[jasaw]

Thank you for the response and pointing me to the right direction.
I've been looking for example of how to memory map the SPI flash. Looks like there's a spi_flash_mmap function in ROM, but it appears to be overridden by the SDK implementation that lives in RAM.
Is it safe for me to call the ROM version spi_flash_mmap directly from my RTC wake-up stub or is there a recommended way of doing SPI flash mmap?

[gm-jiang]

Hi @jasaw, Yeah，we do provide an option to choose whether to use the spi_flash_mmap API in ROM or the API in IRAM，the purpose is to save RAM for some applications.
In fact, flash mmap still has many system-related dependencies, such as the system-level function malloc/free. These system-level functions may not be able to run in wake stub.
RTC wake up stub should be as simple as possible, e.g. read some sensor data, pulse count，etc.
Could you tell me why you need call flash mmap API in wake stub?

[jasaw]

@gm-jiang My RTC wake-up stub needs to receive sensor data, write to flash, and go back to deep sleep. It will only bring up wifi if it has collected enough samples.

What's the best way to read and write data from/to SPI flash?
Can you please provide some example code?

[helmut64]

@jasaw the wakeup stub should be used only for very little code blocks, it has tons of restrictions and every line code must be done with care, no C runtime code can be called (e.g. 64bit multiply, etc. because this code is in the flash)
I recommend to collect your sensor data into the RTC memory and wakeup completely after certain amount of records. This allows to access the flash using the full tuning OS.
Regards Helmut

[gm-jiang]

I suggest that wake-up stub receive sensor data and save this data to RTC_RAM, then write the data to flash when boot chip.
For example: you can make the chip boot up every one hour, and write data to flash in the main program. But please note that the RTC RAM is small and can not store too much data.

[gm-jiang]

@jasaw For read and write data from/to SPI flash, I think you can refer to https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/storage/spi_flash.html#spi-flash-access-api

[jasaw]

Yes, the easy way out is to use the left over RTC memory as a buffer for my sensor samples, but this buffer is way too small.

@gm-jiang I still would like to explore writing to flash from my RTC wake-up stub. Are the SPI flash functions that you linked above suitable to be called from RTC wake-up stub?
I found the esp_rom_spiflash_* functions in spi-flash.h header file that should allow me to directly access the SPI flash. However, I still can't read my SPI flash at address 0x300000, so I probably did something wrong here. Curiously, the read is successful when I read low address like 0x000000 or 0x000004.

Here's my code that reads the SPI flash:

// initialize the spi flash
esp_rom_spiflash_attach(esp_rom_efuse_get_flash_gpio_info(), false);
// read data
uint32_t data[10];
esp_rom_spiflash_result_t result = esp_rom_spiflash_read(0x300000, &data, sizeof(data));
// result is ESP_ROM_SPIFLASH_RESULT_ERR but reading address 0x000004 works fine

Why would the spi flash read fail at 0x300000 address?

[gm-jiang]

@jasaw, I think you need call ROM function esp_rom_spiflash_config_param to set chip_size after esp_rom_spiflash_attach, IIRC the default chip_size was initialized as 2MB.
The spi flash read check the address + length > chip_size, then return ESP_ROM_SPIFLASH_RESULT_ERR

[jasaw]

@gm-jiang Thanks. Calling esp_rom_spiflash_config_param to set the chip size fixed the read issue for me.

esp_rom_spiflash_erase_area appears to be missing from the ESP32-C3 linker file. Should I copy the ESP32 implementation or can I add that function in the linker file?

[gm-jiang]

@jasaw You can find this PROVIDE( SPIEraseArea = 0x400001e4 ) it in components/esp_rom/esp32c3/ld/esp32c3.rom.ld, and you can rename it PROVIDE ( esp_rom_spiflash_erase_area = SPIEraseArea ); if needed.

[peturdainn]

I'm having troubles going back to sleep in the stub, either it continues to boot or it bootloops.

target = ESP32c3

To start deepsleep:

esp_deep_sleep_enable_gpio_wakeup(PIN_BIT_MASK(GPIO_NUM_3) , ESP_GPIO_WAKEUP_GPIO_HIGH);
esp_sleep_enable_timer_wakeup(5*1000*1000);
esp_set_deep_sleep_wake_stub(controller_wakeup_stub);
esp_deep_sleep_start();

Then the stub is simply (as a test):

static void RTC_IRAM_ATTR controller_wakeup_stub()
{
    REG_WRITE(RTC_ENTRY_ADDR_REG, (uint32_t)&controller_wakeup_stub);
    set_rtc_memory_crc();
    CLEAR_PERI_REG_MASK(RTC_CNTL_STATE0_REG, RTC_CNTL_SLEEP_EN);
    SET_PERI_REG_MASK(RTC_CNTL_STATE0_REG, RTC_CNTL_SLEEP_EN);
    while (true) {
        ;
}

in sdkconfig I have

CONFIG_BOOTLOADER_RESERVE_RTC_SIZE=0x10
CONFIG_BOOTLOADER_CUSTOM_RESERVE_RTC=y
CONFIG_BOOTLOADER_CUSTOM_RESERVE_RTC_SIZE=0

(not much documentation in the two different sizes, I already tried increasing both but the outcome is the same)

The above does a bootloop like this:

Build:Fe�ESP-ROM:esp32c3-api1-20210207
Build:Fe�ESP-ROM:esp32c3-api1-20210207
Build:Fe�ESP-ROM:esp32c3-api1-20210207
Build:Fe�ESP-ROM:esp32c3-api1-20210207
Build:Fe�ESP-ROM:esp32c3-api1-20210207
Build:Fe�ESP-ROM:esp32c3-api1-20210207
Build:Fe�ESP-ROM:esp32c3-api1-20210207

If I leave out the RTC_CNTL_STATE0_REG lines it proceeds to the bootloader after deepsleep wakeup

Adding in case it matters: this project is using a secure bootloader

Added if I only try to wake up with GPIO 3 the stub isn't run, rst:0x1, with the timer it did run the stub (could even increment a variable)

[gm-jiang]

@peturdainn Could you test your wake up stub code without secure boot?

[peturdainn]

@peturdainn Could you test your wake up stub code without secure boot?

This does not seem to change anything

I created a stripped down project to run on a devboard, first test is wake from timer.
Would like to know what is wrong with it: either does a boot loop or quits deep sleep.
Does it need reprogramming the timer perhaps?

Next test is wake from GPIO

sdkconfig.defaults.txt
main.c.txt
(using IDF 4.4)

UPDATE:
deep sleep wake_stub with GPIO works OK for non-secure bootloader, still not working OK for secure!
I can get the same type of boot loop if I don't wait for the GPIO level to go back idle, so my theory is that the wakeup timer keeps going off.

So: do I need to set a new timeout (and how), or somehow clear something? None of the examples of other users makes this clear.
Feels like this is almost working

UPDATE 2

There's something wrong with RTC_CNTL_WAKEUP_STATE_REG definition in rtc_cntl_reg.h:

#define RTC_CNTL_WAKEUP_STATE_REG          (DR_REG_RTCCNTL_BASE + 0x003C)
/* RTC_CNTL_WAKEUP_ENA : R/W ;bitpos:[31:15] ;default: 17'b1100 ; */
/*description: wakeup enable bitmap*/
#define RTC_CNTL_WAKEUP_ENA  0x0001FFFF
#define RTC_CNTL_WAKEUP_ENA_M  ((RTC_CNTL_WAKEUP_ENA_V)<<(RTC_CNTL_WAKEUP_ENA_S))
#define RTC_CNTL_WAKEUP_ENA_V  0x1FFFF
#define RTC_CNTL_WAKEUP_ENA_S  15

Datasheet says lower 16 bits but as you see above the whole mask is shifted 15, so upper bits! When using the REG_SET_FIELD() macro this certainly sets and clears wrong bits?
MISREAD datasheet, significant bits are upper 16

[peturdainn]

First conclusion for GPIO deep sleep wakeup:
wake_stub with secure bootloader doesn't work, I get rst:0x1 POWERON

Ran same code with or without secure bootloader (two different boards, yes)

IMPORTANT question: is this by design?
Because if this isn't supported there are some big implications for this project.

[gm-jiang]

1. Hi @peturdainn, could you provide more detials about the "Datasheet says lower 16 bits", like in TRM(Technical Reference Manual) v0.5 in x pages. I do not see anythings about this in the Datasheet or TRM, thanks!
2. If you want read some sensor periodically in the stub, it is indeed necessary to configure the wake-up time in the stub, then set stub entry and CRC, then going to deep sleep again.
3. And I will try your example, we need to confirm further the issue about "wake_stub with secure bootloader doesn't work".

[peturdainn]

1. Looks like I misread the datasheet, significant bits are indeed 15-31. Sorry about that. Debug sessions getting too long ;)
2. Have figured out as much, now trying to get this working properly
3. thanks, I'll put my testcode on github later today for reference

[peturdainn]

Everything OK now, managed to create some code that uses wake stub for GPIO and Timer wakeup.
I also got another devkit to test with secure bootloader, and this works too.
Presumably the issue with secure bootloader is related to the device I tested on.

The bootloop was fixed by clearing RTC_CNTL_INT_CLR_REG (well, clearing the INTs by writing 0xFFFF)

Have created a test/demo app for ESP32-c3:
https://github.com/peturdainn/ESP32c3_deepsleep_wakestub_test

There is both a secure and non-secure sdkconfig.defaults included, bring your own signing key ;)

Remarks welcome.

[gm-jiang]

@peturdainn I also tested that the RTC wakeup stub works with secure boot (using flash encryption).
About the issue The bootloop was fixed by clearing RTC_CNTL_INT_CLR_REG (well, clearing the INTs by writing 0xFFFF)， we'll provide an api for the user's stub code that can reconfigure the wakeup timer, maybe something like this

static uint64_t RTC_IRAM_ATTR wake_stub_rtc_time_get(void)
{
    SET_PERI_REG_MASK(RTC_CNTL_TIME_UPDATE_REG, RTC_CNTL_TIME_UPDATE);
#ifdef CONFIG_IDF_TARGET_ESP32
    while (GET_PERI_REG_MASK(RTC_CNTL_TIME_UPDATE_REG, RTC_CNTL_TIME_VALID) == 0) {
        esp_rom_delay_us(1);
    }
    SET_PERI_REG_MASK(RTC_CNTL_INT_CLR_REG, RTC_CNTL_TIME_VALID_INT_CLR);
#endif
    uint64_t t = READ_PERI_REG(RTC_CNTL_TIME0_REG);
    t |= ((uint64_t) READ_PERI_REG(RTC_CNTL_TIME1_REG)) << 32;
    return t;
}

void RTC_IRAM_ATTR esp_wake_stub_set_wakeup_time(uint64_t time_in_us)
{
    uint32_t slow_clk_value = REG_READ(RTC_SLOW_CLK_CAL_REG);
    uint64_t rtc_count_delta = ((time_in_us * (1 << RTC_CLK_CAL_FRACT)) / slow_clk_value);
    uint64_t rtc_curr_count = wake_stub_rtc_time_get();

    rtc_hal_set_wakeup_timer(rtc_curr_count + rtc_count_delta);
}

[peturdainn]

@gm-jiang a yes, rtc_hal_set_wakeup_timer() is what I need(ed), it takes care of setting RTC_CNTL_MAIN_TIMER_ALARM_EN

looking back, these were probably the two issues that I had:

• RTC_CNTL_MAIN_TIMER_ALARM_EN was getting cleared by setting the new trigger
• RTC_CNTL_INT_CLR_REG is needed for timer (worked without it for GPIO)

Any API is better than having to scrape bits and pieces together ;)

[peturdainn]

Turns out that when starting deep sleep, the function esp_deep_sleep_wakeup_prepare() will modify some GPIO settings, something I would have expected would be done in esp_deep_sleep_enable_gpio_wakeup()

This includes switching pull-up/down based on configured level and HOLD.

IMHO this functionality could be split better, and pin config could be left to the caller (or print an error if configured incorrect)?

EDIT: just discovered this is actually the root cause for some issue on our hardware :(

[Alvin1Zhang]

Thanks for reporting and sharing the updates, fix is available at a6f7035, feel free to reopen. Thanks.

[peturdainn]

@Alvin1Zhang any comment on my last remark about the GPIO settings changed in esp_deep_sleep_wakeup_prepare()?

As I wrote, it would make more sense to have this in esp_deep_sleep_enable_gpio_wakeup() so for specific hardware setups it is still possible to override the PU/PD config

Or do I need to create a new issue for this since this remark is not the topic of this issue?

[esp-wzh]

Hi, @peturdainn In esp_deep_sleep_wakeup_prepare, the IO that is set to wake up at high level will be configured as pull-down, and the IO that is set to wake up at low level will be configured as pull-up, which I think is reasonable. If you have more questions, it's better to create a new issue and describe your application scenario and expected behavior in detail.

[jasaw]

Hi,
How do I do 802.15.1 transmit and receive in the wake stub? I send my raw 802.15.1 packets, so I don't need the bluetooth stack.

[esp-wzh]

@jasaw This is not feasible, in wake_stub, radio is not initialized yet, and the size of RTC_RAM is not enough to store the initialization code

[jasaw]

@esp-wzh Is the radio initialization code in ROM? I can map the ROM radio init code and call it from RTC_RAM?

[esp-wzh]

No, the initialization process is very complicated and it is placed in Flash, and the radio will rely on some pll clock and power initialization in the bootloader, so if you needs to use the radio, it is worth running the bootloader