Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Key-only iteration #1498

Closed
webmaster128 opened this issue Nov 21, 2022 · 1 comment · Fixed by CosmWasm/wasmvm#450 or #1834
Closed

Key-only iteration #1498

webmaster128 opened this issue Nov 21, 2022 · 1 comment · Fixed by CosmWasm/wasmvm#450 or #1834

Comments

@webmaster128
Copy link
Member

In many places we only need the key when iterating over storage. However, we currently load both key and value into the Wasm space. This performs the following unnecessary operations:

  1. v := iter.Value(), i.e. the actual read on the database
  2. Copying the data from Go to Rust
  3. Copy key and value into a single buffer to write it to Wasm (encode_sections)
  4. Decompose key and value into separate vectors in Wasm (decode_sections2)

The following use cases would benefit from a key-only iterator by consuming less ressources:

  • Deletations such as Map::clear
  • All usages of Prefix::keys and Prefix::keys_raw
  • Iterators that only aim to return the last existing key for incrementing

An implementation can be broken down as follows:

  1. Provide db_next_key and db_next_value that operate on the exact same iterator ID as db_next but only return one value. They can be implemented via Storage::next for now.
  2. Add Storage::next_key and Storage::next_value to the backend storage trait and implement in GoStorage by utilizing GoStorage::next
  3. Add GoIter::next_key and GoIter::next_value and corresponsding FFI calls
  4. Make import db_next_key and db_next_value usable in contracts

next/next_key/next_value all inrement the iterator exactly once. For each call you can decide which return value you want to use. This way they can all share the same iterator ID system.

@webmaster128
Copy link
Member Author

webmaster128 commented Aug 17, 2023

The list above is not necessarily ordered. Since we have #1748 already in 1.3, it makes sense to move from contract to chain, one step at a time. We can always fall back on the next implementation that returns both key and value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant