Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nvs: Check if an item is modified before writing out an identical copy (IDFGH-977) #3310

Merged

Conversation

tim-nordell-nimbelink
Copy link
Contributor

This prevents wear and tear on the flash, and it also is faster in some
cases since the read-out of flash is a cheaper operation than the erasure
of flash. Some library modules (such as the esp_wifi) write out to NVS
upon every initialization without checking first that the existing value
is the same, and this speeds up initialization of modules that make
these design choices and moves it into a centralized place.

The comparison functions are based on the read-out functions of the same
name, and changes out the memcpy(...) operations for memcmp(...)
operations.

Signed-off-by: Tim Nordell [email protected]

@github-actions github-actions bot changed the title nvs: Check if an item is modified before writing out an identical copy nvs: Check if an item is modified before writing out an identical copy (IDFGH-977) Apr 15, 2019
@tim-nordell-nimbelink tim-nordell-nimbelink force-pushed the feature/nvs_comparison_check branch 2 times, most recently from f333719 to 2164e91 Compare April 15, 2019 20:12
@tim-nordell-nimbelink
Copy link
Contributor Author

I updated the pull request to fix a couple of things with the changeset:

  • Remove potential erasures that would happen regardless in the callee functions. The comparison functions don't need to clean up after the error case.
  • Replaced an assertion that the read size mismatched with a return condition that the contents were different for cmpMultiPageBlob(...).

Copy link
Member

@igrr igrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tim-nordell-nimbelink Thank you for the PR, it looks very useful. I have left a few mostly cosmetic comments.

Since the logic added in cmpItem and cmpMultiPageBlob is not exactly trivial, may I ask you to add a couple of unit tests for these functions in https://github.com/espressif/esp-idf/blob/master/components/nvs_flash/test_nvs_host/test_nvs.cpp, and a test that writes are not happening when item content is the same (SpiFlashEmulator::getWriteOps can be used for this).
If not, I can add the tests on top of your commit.

components/nvs_flash/include/nvs.h Outdated Show resolved Hide resolved
components/nvs_flash/src/nvs_page.cpp Outdated Show resolved Hide resolved
components/nvs_flash/src/nvs_page.cpp Show resolved Hide resolved
components/nvs_flash/src/nvs_storage.cpp Outdated Show resolved Hide resolved
@tim-nordell-nimbelink
Copy link
Contributor Author

@tim-nordell-nimbelink Thank you for the PR, it looks very useful.

No problem! It helps save wear and tear on the flash on our board (and start-up time out of deep sleep due to extra erases of flash).

I have left a few mostly cosmetic comments.

Sorry, it's easy for me to fall back to the coding style I usually use. I'll go back and fix-up through to match espressif's coding style.

Since the logic added in cmpItem and cmpMultiPageBlob is not exactly trivial, may I ask you to add a couple of unit tests for these functions

It shouldn't be too bad. The other thought I had was that really Page::cmp... and Page::read... are virtually identical. They could utilize a visitor pattern, but that would have a slightly deeper stack depth than the two separate routines. The benefit would be code maintenance, but it would touch more code within the NVS routine.

If not, I can add the tests on top of your commit.

I take it you prefer the tests to be added within the same commit?

-- Tim

@tim-nordell-nimbelink tim-nordell-nimbelink force-pushed the feature/nvs_comparison_check branch 2 times, most recently from 59d3bb2 to 3bfdee6 Compare April 30, 2019 16:36
This prevents wear and tear on the flash, and it also is faster in some
cases since the read-out of flash is a cheaper operation than the erasure
of flash.  Some library modules (such as the esp_wifi) write out to NVS
upon every initialization without checking first that the existing value
is the same, and this speeds up initialization of modules that make
these design choices and moves it into a centralized place.

The comparison functions are based on the read-out functions of the same
name, and changes out the memcpy(...) operations for memcmp(...)
operations.

Signed-off-by: Tim Nordell <[email protected]>
@tim-nordell-nimbelink tim-nordell-nimbelink force-pushed the feature/nvs_comparison_check branch from 3bfdee6 to c3fa249 Compare April 30, 2019 16:41
@tim-nordell-nimbelink
Copy link
Contributor Author

@igrr -

I've gone ahead and switched the logic around to return ESP_OK instead when the comparison is the same. I had to touch a couple of the tests for duplicates that assumed a new write occurred each time a value was written even if it was the same value. (I didn't run the test suite previously, but I had now that I added in the tests for this change set.) Hopefully I got all of the cosmetic changes, too.

-- Tim

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants