Replace old storage API with the new one explained in the `StorageKey…

…` RFC (#4464)

## Description
Implement FuelLabs/sway-rfcs#23

Closes #3419 (technically via the
RFC)
Closes #3202
Closes #3043
Closes #2639
Closes #2465
Closes #4304

This is a big one but mostly removes stuff. It's hard to review but the
summary of changes below should be sufficient. Most of the changes here
are things that I just had to do to make CI pass.

- Removed the `--experimental-storage` flag and made its
[behavior](#4297) the default. Also
removed everything that was required for the previous storage APIs in
favour of the new ones.
- Break down the `std::storage` into multiple submodules:
 - `storage_key` implements the API for `std::core::StorageKey`
- `storage_api` implements the free functions `read` (previously `get`),
`write` (previously `store`), and `clear`.
- 4 more modules for the dynamic storage types which now use the new
`StorageKey` API.
- `#[storage(write)]` now allows reading from storage as well. This is
needed because the way we pack structs in storage now requires that we
sometimes read from storage first if we were to write a portion of a
slot.
- Removed the "storage only types" checks and the corresponding tests.
- Removed the `__get_storage_key` intrinsic and the `get_storage_key` IR
instruction and all corresponding tests. Also removed the `state_index`
metadata as it is no longer required.
- Added tests and example to showcase nested storage maps and nested
storage vectors.

## Checklist

- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.

Co-authored-by: Joshua Batty <[email protected]>

docs/book/src/blockchain-development/purity.md

@@ -16,6 +16,8 @@ fn increment_amount(increment: u64) -> u64 {
 }
 ```
 
+> **Note**: the `#[storage(write)]` attribute also permits a function to read from storage. This is due to the fact that partially writing a storage slot requires first reading the slot.
+
 Impure functions which call other impure functions must have at least the same storage privileges or a superset of those for the function called. For example, to call a function with write access a caller must also have write access, or both read and write access. To call a function with read and write access the caller must also have both privileges.
 
 The `storage` attribute may also be applied to [methods and associated functions](../basics/methods_and_associated_functions.md), [trait](../advanced/traits.md) and [ABI](../sway-program-types/smart_contracts.md#the-abi-declaration) declarations.

docs/book/src/blockchain-development/storage.md

@@ -32,7 +32,7 @@ Generic storage maps are available in the standard library as `StorageMap<K, V>`
 
 ## Manual Storage Management
 
-It is possible to leverage FuelVM storage operations directly using the `std::storage::store` and `std::storage::get` functions provided in the standard library. With this approach you will have to manually assign the internal key used for storage. An example is as follows:
+It is possible to leverage FuelVM storage operations directly using the `std::storage::storage_api::write` and `std::storage::storage_api::read` functions provided in the standard library. With this approach you will have to manually assign the internal key used for storage. An example is as follows:
 
 ```sway
 {{#include ../../../../examples/storage_example/src/main.sw}}

docs/book/src/common-collections/storage_map.md

@@ -47,28 +47,24 @@ We can get a value out of the storage map by providing its `key` to the `get` me
 
 Here, `value1` will have the value that's associated with the first address, and the result will be `42`. The `get` method returns an `Option<V>`; if there’s no value for that key in the storage map, `get` will return `Option::None`. This program handles the `Option` by calling `unwrap_or` to set `value1` to zero if `map` doesn't have an entry for the key.
 
-## Storage maps with multiple keys
+## Storage Maps with Multiple Keys
 
-You might find yourself needing a `StorageMap<K1, V1>` where the type `V1` is itself another `StorageMap<K2, V2>`. This is not allowed in Sway. The right approach is to use a single `StorageMap<K, V>` where `K` is a tuple `(K1, K2)`. For example:
+Maps with multiple keys can be implemented using tuples as keys. For example:
 
 ```sway
 {{#include ../../../../examples/storage_map/src/main.sw:storage_map_tuple_key}}
 ```
 
-## Limitations
+## Nested Storage Naps
 
-It is not currently allowed to have a `StorageMap<K, V>` as a component of a complex type such as a struct or an enum. For example, the code below is not legal:
+It is possible to nest storage maps as follows:
 
 ```sway
-Struct Wrapper {
-    map1: StorageMap<u64, u64>,
-    map2: StorageMap<u64, u64>,
-}
+{{#include ../../../../examples/storage_map/src/main.sw:storage_map_nested}}
+```
 
-storage {
-    w: Wrapper
-}
-...
+The nested map can then be accessed as follows:
 
-storage.w.map1.insert(..);
+```sway
+{{#include ../../../../examples/storage_map/src/main.sw:storage_map_nested_access}}
 ```

docs/book/src/common-collections/storage_vec.md

@@ -47,7 +47,7 @@ To read a value stored in a vector at a particular index, you can use the `get`
 {{#include ../../../../examples/storage_vec/src/main.sw:storage_vec_get}}
 ```
 
-Note three details here. First, we use the index value of `2` to get the third element because vectors are indexed by number, starting at zero. Second, we get the third element by using the `get` method with the index passed as an argument, which gives us an `Option<T>`. Third, the ABI function calling `get` only requires the annotation `#[storage(read)]` as one might expect because `get` does not write to storage.
+Note three details here. First, we use the index value of `2` to get the third element because vectors are indexed by number, starting at zero. Second, we get the third element by using the `get` method with the index passed as an argument, which gives us an `Option<StorageKey<T>>`. Third, the ABI function calling `get` only requires the annotation `#[storage(read)]` as one might expect because `get` does not write to storage.
 
 When the `get` method is passed an index that is outside the vector, it returns `None` without panicking. This is particularly useful if accessing an element beyond the range of the vector may happen occasionally under normal circumstances. Your code will then have logic to handle having either `Some(element)` or `None`. For example, the index could be coming as a contract method argument. If the argument passed is too large, the method `get` will return a `None` value, and the contract method may then decide to revert when that happens or return a meaningful error that tells the user how many items are in the current vector and give them another chance to pass a valid value.
 
@@ -59,7 +59,7 @@ To access each element in a vector in turn, we would iterate through all of the
 {{#include ../../../../examples/storage_vec/src/main.sw:storage_vec_iterate}}
 ```
 
-Again, this is quite similar to iterating over the elements of a `Vec<T>` where we use the method `len` to return the length of the vector. We also call the method `unwrap` to extract the `Option` returned by `get`. We know that `unwrap` will not fail (i.e. will not cause a revert) because each index `i` passed to `get` is known to be smaller than the length of the vector.
+Again, this is quite similar to iterating over the elements of a `Vec<T>` where we use the method `len` to return the length of the vector. We also call the method `unwrap` to extract the `Option` returned by `get` followed by a call to `read()` to actually read the stored value. We know that `unwrap` will not fail (i.e. will not cause a revert) because each index `i` passed to `get` is known to be smaller than the length of the vector.
 
 ## Using an Enum to store Multiple Types
 
@@ -82,3 +82,17 @@ We can now push different enum variants to the storage vector as follows:
 ```
 
 Now that we’ve discussed some of the most common ways to use storage vectors, be sure to review the API documentation for all the many useful methods defined on `StorageVec<T>` by the standard library. For now, these can be found in the [source code for `StorageVec<T>`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/storage.sw). For example, in addition to `push`, a `pop` method removes and returns the last element, a `remove` method removes and returns the element at some chosen index within the vector, an `insert` method inserts an element at some chosen index within the vector, etc.
+
+## Nested Storage Vecs
+
+It is possible to nest storage vectors as follows:
+
+```sway
+{{#include ../../../../examples/storage_vec/src/main.sw:storage_vec_nested}}
+```
+
+The nested vector can then be accessed as follows:
+
+```sway
+{{#include ../../../../examples/storage_vec/src/main.sw:access_nested_vec}}
+```

docs/book/src/introduction/standard_library.md

@@ -13,7 +13,7 @@ The standard library is made implicitly available to all Forc projects created u
 Importing items from the standard library can be done using the `use` keyword, just as importing items from any Sway project. For example:
 
 ```sway
-use std::storage::StorageVec;
+use std::storage::storage_vec::*;
 ```
 
 This imports the `StorageVec` type into the current namespace.
@@ -30,7 +30,8 @@ The current version of the prelude lives in [`std::prelude`](https://github.com/
 - [`std::contract_id::ContractId`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/contract_id.sw), a wrapper around the `b256` type representing the ID of a contract.
 - [`std::identity::Identity`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/identity.sw), an enum with two possible variants: `Address: Address` and `ContractId: ContractId`.
 - [`std::vec::Vec`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/vec.sw), a growable, heap-allocated vector.
-- [`std::storage::StorageMap`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/storage.sw), a key-value mapping in contract storage.
+- [`std::storage::storage_key::*`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/storage/storage_key.sw), contains the API for accessing a `core::storage::StorageKey` which describes a location in storage.
+- [`std::storage::storage_map::*`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/storage/storage_map.sw), a key-value mapping in contract storage.
 - [`std::option::Option`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/option.sw), an enum which expresses the presence or absence of a value.
 - [`std::result::Result`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/result.sw), an enum for functions that may succeed or fail.
 - [`std::assert::assert`](https://github.com/FuelLabs/sway/blob/master/sway-lib-std/src/assert.sw), a function that reverts the VM if the condition provided to it is `false`.

docs/book/src/reference/known_issues_and_workarounds.md

@@ -8,8 +8,6 @@
 
 * [#1182](https://github.com/FuelLabs/sway/issues/1182) Arrays in a `storage` block are not yet supported. See the [Manual Storage Management](../blockchain-development/storage.md#manual-storage-management) section for details on how to use `store` and `get` from the standard library to manage storage slots directly. Note, however, that `StorageMap<K, V>` _does_ support arbitrary types for `K` and `V` without any limitations.
 
-* [#1796](https://github.com/FuelLabs/sway/issues/2465): It is not yet allowed to use `StorageMap<K, V>` as a component of a complex type such as a struct or an enum.
-
 ## General
 
 * No compiler optimization passes have been implemented yet, therefore bytecode will be more expensive and larger than it would be in production. Note that eventually the optimizer will support zero-cost abstractions, avoiding the need for developers to go down to inline assembly to produce optimal code.

examples/Forc.lock

@@ -122,6 +122,7 @@ dependencies = ['std']
 [[package]]
 name = 'storage_variables'
 source = 'member'
+dependencies = ['std']
 
 [[package]]
 name = 'storage_vec'

examples/cei_analysis/src/main.sw

@@ -19,7 +19,7 @@ impl MyContract for Contract {
     #[storage(read, write)]
     fn withdraw(external_contract_id: ContractId) {
         let sender = msg_sender().unwrap();
-        let bal = storage.balances.get(sender).unwrap_or(0);
+        let bal = storage.balances.get(sender).try_read().unwrap_or(0);
 
         assert(bal > 0);
 

examples/counter/src/main.sw

@@ -15,14 +15,14 @@ storage {
 impl TestContract for Contract {
     #[storage(write)]
     fn initialize_counter(value: u64) -> u64 {
-        storage.counter = value;
+        storage.counter.write(value);
         value
     }
 
     #[storage(read, write)]
     fn increment_counter(amount: u64) -> u64 {
-        let incremented = storage.counter + amount;
-        storage.counter = incremented;
+        let incremented = storage.counter.read() + amount;
+        storage.counter.write(incremented);
         incremented
     }
 }

examples/identity/src/main.sw

@@ -55,7 +55,7 @@ impl IdentityExample for Contract {
     fn access_control_with_identity() {
         // ANCHOR: access_control_with_identity
         let sender = msg_sender().unwrap();
-        require(sender == storage.owner, MyError::UnauthorizedUser(sender));
+        require(sender == storage.owner.read(), MyError::UnauthorizedUser(sender));
         // ANCHOR_END: access_control_with_identity
     }
 }

examples/ownership/src/main.sw

@@ -17,17 +17,17 @@ impl OwnershipExample for Contract {
     // ANCHOR: revoke_owner_example
     #[storage(write)]
     fn revoke_ownership() {
-        storage.owner = Option::None;
+        storage.owner.write(Option::None);
     }
     // ANCHOR_END: revoke_owner_example
     // ANCHOR: set_owner_example
     #[storage(write)]
     fn set_owner(identity: Identity) {
-        storage.owner = Option::Some(identity);
+        storage.owner.write(Option::Some(identity));
     }
     // ANCHOR_END: set_owner_example
     #[storage(read)]
     fn owner() -> Option<Identity> {
-        storage.owner
+        storage.owner.read()
     }
 }

examples/storage_example/src/main.sw

@@ -1,6 +1,6 @@
 contract;
 
-use std::storage::{get, store};
+use std::storage::storage_api::{read, write};
 
 abi StorageExample {
     #[storage(write)]
@@ -15,12 +15,12 @@ const STORAGE_KEY: b256 = 0x0000000000000000000000000000000000000000000000000000
 impl StorageExample for Contract {
     #[storage(write)]
     fn store_something(amount: u64) {
-        store(STORAGE_KEY, amount);
+        write(STORAGE_KEY, 0, amount);
     }
 
     #[storage(read)]
     fn get_something() -> u64 {
-        let value: Option<u64> = get::<u64>(STORAGE_KEY);
+        let value: Option<u64> = read::<u64>(STORAGE_KEY, 0);
         value.unwrap_or(0)
     }
 }

examples/storage_map/src/main.sw

@@ -7,6 +7,9 @@ storage {
     // ANCHOR: storage_map_tuple_key
     map_two_keys: StorageMap<(b256, bool), b256> = StorageMap {},
     // ANCHOR_END: storage_map_tuple_key
+    // ANCHOR: storage_map_nested
+    nested_map: StorageMap<u64, StorageMap<u64, u64>> = StorageMap {},
+    // ANCHOR_END: storage_map_nested
 }
 
 abi StorageMapExample {
@@ -15,6 +18,9 @@ abi StorageMapExample {
 
     #[storage(read, write)]
     fn get_from_storage_map();
+
+    #[storage(read, write)]
+    fn access_nested_map();
 }
 
 impl StorageMapExample for Contract {
@@ -37,7 +43,20 @@ impl StorageMapExample for Contract {
         storage.map.insert(addr1, 42);
         storage.map.insert(addr2, 77);
 
-        let value1 = storage.map.get(addr1).unwrap_or(0);
+        let value1 = storage.map.get(addr1).try_read().unwrap_or(0);
     }
     // ANCHOR_END: storage_map_get
+
+    // ANCHOR: storage_map_nested_access
+    #[storage(read, write)]
+    fn access_nested_map() {
+        storage.nested_map.get(0).insert(1, 42);
+        storage.nested_map.get(2).insert(3, 24);
+
+        assert(storage.nested_map.get(0).get(1).read() == 42);
+        assert(storage.nested_map.get(0).get(0).try_read().is_none()); // Nothing inserted here
+        assert(storage.nested_map.get(2).get(3).read() == 24);
+        assert(storage.nested_map.get(2).get(2).try_read().is_none()); // Nothing inserted here
+    }
+    // ANCHOR_END: storage_map_nested_access
 }

examples/storage_variables/Forc.toml

@@ -1,8 +1,8 @@
 [project]
 authors = ["Fuel Labs <[email protected]>"]
 entry = "main.sw"
-implicit-std = false
 license = "Apache-2.0"
 name = "storage_variables"
 
 [dependencies]
+std = { path = "../../sway-lib-std" }

examples/storage_variables/src/main.sw

@@ -32,20 +32,20 @@ impl StorageExample for Contract {
     // ANCHOR: storage_write
     #[storage(write)]
     fn store_something() {
-        storage.var1.x = 42;
-        storage.var1.y = 77;
-        storage.var2.w = 0x1111111111111111111111111111111111111111111111111111111111111111;
-        storage.var2.z = true;
+        storage.var1.x.write(42);
+        storage.var1.y.write(77);
+        storage.var2.w.write(0x1111111111111111111111111111111111111111111111111111111111111111);
+        storage.var2.z.write(true);
     }
     // ANCHOR_END: storage_write
     // ANCHOR: storage_read
     #[storage(read)]
     fn get_something() -> (u64, u64, b256, bool) {
         (
-            storage.var1.x,
-            storage.var1.y,
-            storage.var2.w,
-            storage.var2.z,
+            storage.var1.x.read(),
+            storage.var1.y.read(),
+            storage.var2.w.read(),
+            storage.var2.z.read(),
         )
     }
     // ANCHOR_END: storage_read

examples/storage_vec/src/main.sw

@@ -1,7 +1,7 @@
 contract;
 
 // ANCHOR: storage_vec_import
-use std::storage::StorageVec;
+use std::storage::storage_vec::*;
 // ANCHOR_END: storage_vec_import
 // ANCHOR: storage_vec_multiple_types_enum
 enum TableCell {
@@ -17,6 +17,9 @@ storage {
     // ANCHOR: storage_vec_multiple_types_decl
     row: StorageVec<TableCell> = StorageVec {},
     // ANCHOR_END: storage_vec_multiple_types_decl
+    // ANCHOR: storage_vec_nested
+    nested_vec: StorageVec<StorageVec<u64>> = StorageVec {},
+    // ANCHOR_END: storage_vec_nested
 }
 
 abi StorageVecContract {
@@ -31,6 +34,9 @@ abi StorageVecContract {
 
     #[storage(read, write)]
     fn push_to_multiple_types_storage_vec();
+
+    #[storage(read, write)]
+    fn access_nested_vec();
 }
 
 impl StorageVecContract for Contract {
@@ -48,7 +54,7 @@ impl StorageVecContract for Contract {
     fn read_from_storage_vec() {
         let third = storage.v.get(2);
         match third {
-            Option::Some(third) => log(third),
+            Option::Some(third) => log(third.read()),
             Option::None => revert(42),
         }
     }
@@ -58,7 +64,7 @@ impl StorageVecContract for Contract {
     fn iterate_over_a_storage_vec() {
         let mut i = 0;
         while i < storage.v.len() {
-            log(storage.v.get(i).unwrap());
+            log(storage.v.get(i).unwrap().read());
             i += 1;
         }
     }
@@ -71,4 +77,33 @@ impl StorageVecContract for Contract {
         storage.row.push(TableCell::Boolean(true));
     }
     // ANCHOR_END: storage_vec_multiple_types_fn
+
+    // ANCHOR: access_nested_vec 
+    #[storage(read, write)]
+    fn access_nested_vec() {
+        storage.nested_vec.push(StorageVec {});
+        storage.nested_vec.push(StorageVec {});
+
+        let mut inner_vec0 = storage.nested_vec.get(0).unwrap();
+        let mut inner_vec1 = storage.nested_vec.get(1).unwrap();
+
+        inner_vec0.push(0);
+        inner_vec0.push(1);
+
+        inner_vec1.push(2);
+        inner_vec1.push(3);
+        inner_vec1.push(4);
+
+        assert(inner_vec0.len() == 2);
+        assert(inner_vec0.get(0).unwrap().read() == 0);
+        assert(inner_vec0.get(1).unwrap().read() == 1);
+        assert(inner_vec0.get(2).is_none());
+
+        assert(inner_vec1.len() == 3);
+        assert(inner_vec1.get(0).unwrap().read() == 2);
+        assert(inner_vec1.get(1).unwrap().read() == 3);
+        assert(inner_vec1.get(2).unwrap().read() == 4);
+        assert(inner_vec1.get(3).is_none());
+    }
+    // ANCHOR_END: access_nested_vec 
 }

examples/subcurrency/src/main.sw

@@ -64,7 +64,7 @@ impl Token for Contract {
         };
 
         // Increase the balance of receiver
-        storage.balances.insert(receiver, storage.balances.get(receiver).unwrap_or(0) + amount);
+        storage.balances.insert(receiver, storage.balances.get(receiver).try_read().unwrap_or(0) + amount);
     }
 
     #[storage(read, write)]
@@ -76,12 +76,12 @@ impl Token for Contract {
         };
 
         // Reduce the balance of sender
-        let sender_amount = storage.balances.get(sender).unwrap_or(0);
+        let sender_amount = storage.balances.get(sender).try_read().unwrap_or(0);
         assert(sender_amount > amount);
         storage.balances.insert(sender, sender_amount - amount);
 
         // Increase the balance of receiver
-        storage.balances.insert(receiver, storage.balances.get(receiver).unwrap_or(0) + amount);
+        storage.balances.insert(receiver, storage.balances.get(receiver).try_read().unwrap_or(0) + amount);
 
         log(Sent {
             from: sender,