Generic price data provider running. Looking to depeg next

.env.example

@@ -1,4 +1,12 @@
+# Testing for RedStone uses these global transactions for ./script/DeployRedstoneVST.sh
+RPC_URL=
+PRIVATE_KEY=
+SEQUENCER_ADDRESS=
+FACTORY_ADDRESS=
+CONTRACT_PATH=
+
+# Globals
 ETHERSCAN_API_KEY=
 ARBITRUM_GOERLI_RPC_URL=
 PRIVATE_KEY=
-ARBITRUM_RPC_URL=
+ARBITRUM_RPC_URL=

foundry.toml

@@ -5,13 +5,15 @@ libs = ['lib']
 test = 'test/V2'
 
 
-remappings = ["@openzeppelin/=lib/openzeppelin-contracts/","@chainlink=lib/chainlink/contracts/src/v0.8","@solmate=lib/solmate/src"]
+remappings = ["@openzeppelin/=lib/openzeppelin-contracts/","@chainlink=lib/chainlink/contracts/src/v0.8","@solmate=lib/solmate/src","@redstone-finance=lib/redstone-finance"]
 fs_permissions = [{ access = "read", path = "./"}]
 
 # See more config options https://github.com/foundry-rs/foundry/tree/master/config
 
 [etherscan]
-arbitrum = { key = "${ETHERSCAN_API_KEY}" }
+#arbitrum = { key = "${ETHERSCAN_API_KEY}" }
+#goerli = { key = "${ETHERSCAN_API_KEY}" }
 
 [rpc_endpoints]
-arbitrum = "${ARBITRUM_RPC_URL}"
+arbitrum = "${ARBITRUM_RPC_URL}"
+goerli = "${GOERLI_RPC_URL}"

lib/redstone-finance/evm-connector/README.md

@@ -0,0 +1,267 @@
+# 🔗 @redstone-finance/evm-connector
+
+[![License](https://img.shields.io/badge/license-MIT-green)](https://choosealicense.com/licenses/mit/)
+[![Discord](https://img.shields.io/discord/786251205008949258?logo=discord)](https://discord.gg/2CT6hN6C)
+[![NPM](https://img.shields.io/npm/v/@redstone-finance/evm-connector)](https://www.npmjs.com/package/@redstone-finance/evm-connector)
+[![Twitter](https://img.shields.io/twitter/follow/redstone_defi?style=flat&logo=twitter)](https://twitter.com/intent/follow?screen_name=limestone_defi)
+
+The @redstone-finance/evm-connector module implements an alternative design of providing oracle data to smart contracts. Instead of constantly persisting data on EVM storage (by data providers), the information is brought on-chain only when needed (by end users). Until that moment data remains in the decentralised cache layer, which is powered by RedStone light cache gateways and streamr data broadcasting protocol. Data is transferred to the EVM by end users, who should attach signed data packages to their transaction calldata. The information integrity is verified on-chain through signature checking.
+
+- [🚀 Working demo](#---working-demo)
+- [📦 Installation](#---installation)
+- [🔥 Getting started](#---getting-started)
+  - [1. Modifying your contracts](#1-modifying-your-contracts)
+  - [2. Updating the interface](#2-updating-the-interface)
+    - [Contract object wrapping](#contract-object-wrapping)
+- [💡 How it works](#---how-it-works)
+  - [Storage-less approach](#storage-less-approach)
+  - [Data packing (off-chain data encoding)](#data-packing--off-chain-data-encoding-)
+  - [Data unpacking (on-chain data verification)](#data-unpacking--on-chain-data-verification-)
+  - [On-chain aggregation](#on-chain-aggregation)
+  - [Types of values](#types-of-values)
+  - [Security considerations](#security-considerations)
+  - [Recommendations](#recommendations)
+  - [Benchmarks](#benchmarks)
+    - [Gas report for 1 unique signer:](#gas-report-for-1-unique-signer-)
+    - [Gas report for 10 unique signers:](#gas-report-for-10-unique-signers-)
+- [👨‍💻 Development and contributions](#------development-and-contributions)
+  - [Installing the dependencies](#installing-the-dependencies)
+  - [Compiling and running the tests](#compiling-and-running-the-tests)
+- [📄 License](#---license)
+<!-- The table of contents above was generated by https://ecotrust-canada.github.io/markdown-toc/ -->
+
+## 🚀 Working demo
+
+- See a bunch of smart contract examples that use the evm-connector in our [dedicated repo with examples](https://github.com/redstone-finance/redstone-evm-examples)
+
+## 📦 Installation
+
+Install [@redstone-finance/evm-connector](https://www.npmjs.com/package/@redstone-finance/evm-connector) from NPM registry
+
+```bash
+# Using yarn
+yarn add @redstone-finance/evm-connector
+
+# Using NPM
+npm install @redstone-finance/evm-connector
+```
+
+## 🔥 Getting started
+
+### 1. Modifying your contracts
+
+You need to apply a minimum change to the source code to enable smart contract to access data. Your contract needs to extend one of our custom base contracts, which are located in the [contracts/data-services](./contracts/data-services) folder.
+
+We strongly recommend you to have some upgradability mechanism for your contracts (it can be based on multisig, DAO, or anything else). This way, you can quickly switch to the latest trusted data providers in case of changes or problems with the current providers.
+
+```js
+import "@redstone-finance/evm-connector/dist/contracts/data-services/AvalancheDataServiceConsumerBase.sol";
+
+contract YourContractName is AvalancheDataServiceConsumerBase {
+  ...
+}
+```
+
+💡 Note: You can also override the following functions (do it on your own risk):
+
+- `validateTimestamp(uint256 receivedTimestampMilliseconds)` - to enable custom logic of timestamp validation
+- `aggregateValues(uint256[] memory values) returns (uint256)` - to enable custom logic of aggregating values from different providers (by default this function takes the median value)
+- `getAuthorisedSignerIndex(address _signerAddress) returns (uint8)` function and `getUniqueSignersThreshold() returns (uint8)` function - to enable custom logic of signers authorisation
+
+After applying the mentioned change you will be able to access the data calling the local `getOracleNumericValueFromTxMsg` function. You should pass the data feed id converted to `bytes32`.
+
+```js
+// Getting a single value
+uint256 ethPrice = getOracleNumericValueFromTxMsg(bytes32("ETH"));
+
+// Getting several values
+bytes32[] memory dataFeedIds = new bytes32[](2);
+dataFeedIds[0] = bytes32("ETH");
+dataFeedIds[1] = bytes32("BTC");
+uint256[] memory values = getOracleNumericValuesFromTxMsg(dataFeedIds);
+uint256 ethPrice = values[0];
+uint256 btcPrice = values[1];
+```
+
+You can see all available data feeds [in our web app.](https://app.redstone.finance)
+
+### 2. Updating the interface
+
+You should also update the code responsible for submitting transactions. If you're using [ethers.js](https://github.com/ethers-io/ethers.js/), we've prepared a dedicated library to make the transition seamless.
+
+#### Contract object wrapping
+
+First, you need to import the wrapper code to your project
+
+```ts
+// Typescript
+import { WrapperBuilder } from "@redstone-finance/evm-connector";
+
+// Javascript
+const { WrapperBuilder } = require("@redstone-finance/evm-connector");
+```
+
+Then you can wrap your ethers contract pointing to the selected [redstone data service](https://app.redstone.finance) and requested data feeds.
+
+```js
+const yourEthersContract = new ethers.Contract(address, abi, provider);
+
+// Connecting all provider's prices (consumes more GAS)
+const wrappedContract = WrapperBuilder
+                          .wrap(yourEthersContract)
+                          .usingDataService({
+                            dataServiceId: "avalanche-main-data-service"
+                            uniqueSignersCount: 10,
+                            dataFeeds: ["ETH", "AVAX", "BTC"]
+                          });
+
+```
+
+Now you can access any of the contract's methods in exactly the same way as interacting with the ethers-js code:
+
+```js
+wrappedContract.executeYourMethod(arg1, arg2);
+```
+
+It's also possible to request pure bytes data. Take a look at [bytes-many-data-feeds.test.ts](./test/mock-wrapper/bytes-many-data-feeds.test.ts) to learn more.
+
+## 💡 How it works
+
+### Storage-less approach
+
+Putting data directly into storage is the easiest to make information accessible to smart contracts. However, the convenience comes at a high price, as the storage access is the most costly operation in [EVM](https://ethereum.github.io/yellowpaper/paper.pdf) (20k gas for 256bit word ~ $160k for 1Mb checked 30/08/2021) making it prohibitively expensive to use.
+
+That's why, Redstone proposes a completely new storage-less approach.
+
+At a top level, transferring data to an EVM environment requires packing an extra payload to a user's transaction and processing the message on-chain.
+
+[![image.png](https://i.postimg.cc/5NZSqtFT/image.png)](https://postimg.cc/xc3m9n53)
+
+#### Data packing (off-chain data encoding)
+
+1. Relevant data needs to be fetched from the decentralized cache layer, powered by the [streamr network](https://streamr.network/) and the RedStone light cache nodes
+2. Data is packed into a message according to the following structure
+
+![redstone-tx-payload-improved-2](https://user-images.githubusercontent.com/48165439/196044365-8cb3e020-56f4-46cd-b058-105772aca3a5.png)
+
+<!---
+- `TX_PAYLOAD` = `[DATA_PACKAGES][UNSIGNED_METADATA]`
+  - `UNSIGNED_METADATA` = `[ANY_MESSAGE][MESSAGE_BYTE_SIZE:3b][REDSTONE_MARKER:9b]`
+    - `REDSTONE_MARKER` = `0x000002ed57011e0000`
+  - `DATA_PACKAGES` = `[DATA_PACKAGE[0]]..[DATA_PACKAGE[N]][NUMBER_OF_DATA_PACKAGES:2b]`
+    - `DATA_PACKAGE` = `[DATA_POINTS][TIMESTAMP:6b][DATA_POINT_VALUE_BYTE_SIZE:4b][DATA_POINTS_COUNT:3b][SIGNATURE:65b]`
+      - `DATA_POINT` = `[DATA_POINT_VALUE][DATA_FEED_ID:32b]`
+-->
+
+3. The package is appended to the original transaction message, signed and submitted to the network
+
+_All of the steps are executed automatically by the ContractWrapper and transparent to the end-user_
+
+#### Data unpacking (on-chain data verification)
+
+1. The appended data packages are extracted from the `msg.data`
+2. For each data package we:
+   - Verify if the signature was created by trusted provider
+   - Validate the timestamp, checking if the information is not obsolete
+3. Then, for each requested data feed we:
+   - Calculate the number of received unique signers
+   - Extract value for each unique signer
+   - Calculate the aggregated value (median by default)
+
+_This logic is executed in the on-chain environment and we optimised the execution using a low-level assembly code to reduce gas consumption to the absolute minimum_
+
+### On-chain aggregation
+
+To increase the security of the Redstone oracle system, we've created the on-chain aggregation mechanism. This mechanism adds an additional requirement of passing at least X signatures from different authorised data providers for a given data feed. The values of different providers are then aggregated before returning to a consumer contract (by default, we use median value calculation for aggregation). This way, even if some small subset of providers corrupt (e.g. 2 of 10), it should not significantly affect the aggregated value.
+
+There are the following on-chain aggregation params in Redstone consumer base contract:
+
+- `getUniqueSignersThreshold` function
+- `getAuthorisedSignerIndex` function
+- `aggregateValues` function (for numeric values)
+- `aggregateByteValues` function (for bytes arrays)
+
+### Types of values
+
+We support 2 types of data to be received in contract:
+
+1. Numeric 256-bit values (used by default)
+2. Bytes arrays with dynamic size
+
+### Security considerations
+
+- Do not modify the `getUniqueSignersThreshold` function, unless you 100% sure about it
+- Pay attention to the timestamp validation logic. For some use-cases (e.g. synthetic DEX), you would need to cache the latest values in your contract storage to avoid arbitrage attacks
+- Enable secure upgradability mechanism for your contract (ideally based on multi-sig or DAO)
+- Monitor the Redstone data services registry and quickly modify signer authorisation logic in your contracts in case of changes (we will also notify you if you are a paying client)
+
+### Recommendations
+
+- Try to design your contracts in a way where you don't need to request many data feeds in the same transaction
+- Use ~10 required unique signers for a good balance between security and gas cost efficiency
+
+### Benchmarks
+
+You can check the benchmarks script and reports in the [benchmarks](./benchmarks) folder.
+
+#### Gas report for 1 unique signer:
+
+```js
+{
+  "1 data feed": {
+    "attaching to calldata": 1840,
+    "data extraction and validation": 10782
+  },
+  "2 data feeds": {
+    "attaching to calldata": 3380,
+    "data extraction and validation": 18657
+  },
+  "10 data feeds": {
+    "attaching to calldata": 15832,
+    "data extraction and validation": 95539
+  },
+}
+```
+
+#### Gas report for 10 unique signers:
+
+```js
+{
+  "1 data feed": {
+    "attaching to calldata": 15796,
+    "data extraction and validation": 72828
+  },
+  "2 data feeds": {
+    "attaching to calldata": 31256,
+    "data extraction and validation": 146223
+  },
+  "10 data feeds": {
+    "attaching to calldata": 156148,
+    "data extraction and validation": 872336
+  },
+  "20 data feeds": {
+    "attaching to calldata": 313340,
+    "data extraction and validation": 2127313
+  }
+}
+```
+
+## 👨‍💻 Development and contributions
+
+The codebase consists of a wrapper written in typescript which is responsible for packing the data and solidity smart contracts that extract the information. We encourage anyone to build and test the code and we welcome any issues with suggestions and pull requests.
+
+### Installing the dependencies
+
+```bash
+yarn install
+```
+
+### Compiling and running the tests
+
+```bash
+yarn test
+```
+
+## 📄 License
+
+Redstone EVM connector is an open-source and free software released under the BUSL-1.1 License.