Add documentation around FRAME Offchain workers

docs/sdk/Cargo.toml

@@ -17,9 +17,13 @@ workspace = true
 # Needed for all FRAME-based code
 parity-scale-codec = { version = "3.0.0", default-features = false }
 scale-info = { version = "2.6.0", default-features = false }
-frame = { path = "../../substrate/frame", features = ["experimental", "runtime"] }
+frame = { path = "../../substrate/frame", features = [
+	"experimental",
+	"runtime",
+] }
 pallet-examples = { path = "../../substrate/frame/examples" }
 pallet-default-config-example = { path = "../../substrate/frame/examples/default-config" }
+pallet-example-offchain-worker = { path = "../../substrate/frame/examples/offchain-worker" }
 
 # How we build docs in rust-docs
 simple-mermaid = "0.1.1"
@@ -31,7 +35,7 @@ kitchensink-runtime = { path = "../../substrate/bin/node/runtime" }
 chain-spec-builder = { package = "staging-chain-spec-builder", path = "../../substrate/bin/utils/chain-spec-builder" }
 subkey = { path = "../../substrate/bin/utils/subkey" }
 
-# Substrate
+# Substrate Client
 sc-network = { path = "../../substrate/client/network" }
 sc-rpc-api = { path = "../../substrate/client/rpc-api" }
 sc-rpc = { path = "../../substrate/client/rpc" }
@@ -43,6 +47,7 @@ sc-consensus-grandpa = { path = "../../substrate/client/consensus/grandpa" }
 sc-consensus-beefy = { path = "../../substrate/client/consensus/beefy" }
 sc-consensus-manual-seal = { path = "../../substrate/client/consensus/manual-seal" }
 sc-consensus-pow = { path = "../../substrate/client/consensus/pow" }
+
 substrate-wasm-builder = { path = "../../substrate/utils/wasm-builder" }
 
 # Cumulus
@@ -60,6 +65,7 @@ sp-api = { path = "../../substrate/primitives/api" }
 sp-core = { path = "../../substrate/primitives/core" }
 sp-keyring = { path = "../../substrate/primitives/keyring" }
 sp-runtime = { path = "../../substrate/primitives/runtime" }
+sp-offchain = { path = "../../substrate/primitives/offchain" }
 
 # XCM
 xcm = { package = "staging-xcm", path = "../../polkadot/xcm" }

docs/sdk/src/reference_docs/frame_offchain_workers.rs

@@ -0,0 +1,115 @@
+//! # Offchain Workers
+//!
+//! This reference document explains how offchain workers work in Substrate and FRAME. The main
+//! focus is upon FRAME's implementation of this functionality. Nonetheless, offchain workers are a
+//! Substrate-provided feature and can be used with possible alternatives to [`frame`] as well.
+//!
+//! Offchain workers are a commonly misunderstood topic, therefore we explain them bottom-up,
+//! starting at the fundamentals and then describing the developer interface.
+//!
+//! ## Context
+//!
+//! Recall from [`crate::reference_docs::wasm_meta_protocol`] that the node and the runtime
+//! communicate with one another via host functions and runtime APIs. Many of these interactions
+//! contribute to the actual state transition of the blockchain. For example [`sp_api::Core`] is the
+//! main runtime API that is called to execute new blocks.
+//!
+//! Offchain workers are in principle not different in any way: It is a runtime API exposed by the
+//! wasm blob ([`sp_offchain::OffchainWorkerApi`]), and the node software calls into it when it
+//! deems fit. But, crucially, this API call is different in that:
+//!
+//! 1. It can have no impact on the state ie. it is _OFF (the) CHAIN_. If any state is altered
+//!    during the execution of this API call, it is discarded.
+//! 2. It has access to an extended set of host functions that allow the wasm blob to do more. For
+//!    example, call into HTTP requests.
+//!
+//! > The main way through which an offchain worker can interact with the state is by submitting an
+//! > extrinsic to the chain. This is the ONLY way to alter the state from an offchain worker.
+//! > [`pallet_example_offchain_worker`] provides an example of this.
+//!
+//!
+//! Given the "Off Chain" nature of this API, it is important to remember that calling this API is
+//! entirely optional. Some nodes might call into it, some might not, and it would have no impact on
+//! the execution of your blockchain because no state is altered no matter the execution of the
+//! offchain worker API.
+//!
+//! Substrate's CLI allows some degree of configuration about this, allowing node operators to
+//! specify when they want to run the offchain worker API. See
+//! [`sc_cli::RunCmd::offchain_worker_params`].
+//!
+//! ## Nondeterministic Execution
+//!
+//! Needless to say, given the above description, the code in your offchain worker API can be
+//! nondeterministic, as it is executed at unknown times, by unknown nodes, and has no impact on the
+//! state. This is why an HTTP ([`sp_runtime::offchain::http`]) API is readily provided to the
+//! offchain worker APIs. Because there is no need for determinism in this context.
+//!
+//! > A common mistake here is for novice developers to see this HTTP API, and imagine that
+//! > `polkadot-sdk` somehow magically solved the determinism in blockchains, and now a blockchain
+//! > can make HTTP calls and it will all work. This is absolutely NOT the case. An HTTP call made
+//! > by the offchain worker is non-deterministic by design. Blockchains can't and always won't be
+//! > able to perform non-deterministic operations such as making HTTP calls to a foreign server.
+//!
+//! ## FRAME's API
+//!
+//! [`frame`] provides a simple API through which pallets can define offchain worker functions. This
+//! is part of [`frame::traits::Hooks`], which is implemented as a part of
+//! [`frame::pallet_macros::hooks`].
+//!
+//! ```
+//! use frame::prelude::*;
+//!
+//! #[frame::pallet]
+//! pub mod pallet {
+//! 	use super::*;
+//!
+//! 	#[pallet::config]
+//! 	pub trait Config: frame_system::Config {}
+//!
+//! 	#[pallet::pallet]
+//! 	pub struct Pallet<T>(_);
+//!
+//! 	#[pallet::hooks]
+//! 	impl<T: Config> Hooks for Pallet<T> {
+//! 		fn offchain_worker(block_number: T::BlockNumber) {
+//! 			// ...
+//! 		}
+//! 	}
+//! }
+//! ```
+//!
+//! Additionally, [`sp_runtime::offchain`] provides a set of utilities that can be used to moderate
+//! the execution of offchain workers.
+//!
+//! ## Think Twice: Why Use Substrate's Offchain Workers?
+//!
+//! Consider the fact that in principle, an offchain worker code written using the above API is no
+//! different than an equivalent written with an _actual offchain interaction library_, such as
+//! [Polkadot-JS](https://polkadot.js.org/docs/), or any of the other ones listed [here](https://github.com/substrate-developer-hub/awesome-substrate?tab=readme-ov-file#client-libraries).
+//!
+//! They can both read from the state, and have no means of updating the state, other than the route
+//! of submitting an extrinsic to the chain. Therefore, it is worth thinking twice before embedding
+//! a logic as a part of Substrate's offchain worker API. Does it have to be there? can it not be a
+//! simple, actual offchain application that lives outside of the chain's WASM blob?
+//!
+//! Some of the reasons why you might want to do the opposite, and actually embed an offchain worker
+//! API into the WASM blob are:
+//!
+//! * Accessing the state is easier within the `offchain_worker` function, as it is already a part
+//!   of the runtime, and [`frame::pallet_macros::storage`] provides all the tools needed to read
+//!   the state. Other client libraries might provide varying degrees of capability here.
+//! * It will be updated in synchrony with the runtime. A Substrate's offchain application is part
+//!   of the same WASM blob, and is therefore guaranteed to be up to date.
+//!
+//! For example, imagine you have modified a storage item to have a new type. This will possibly
+//! require a [`crate::reference_docs::frame_runtime_migration`], and any offchain code, such as a
+//! Polkadot-JS application, will have to be updated to reflect this change. Whereas the WASM
+//! offchain worker code is guaranteed to already be updated, or else the runtime code will not even
+//! compile.
+//!
+//!
+//! ## Further References
+//!
+//! - <https://forum.polkadot.network/t/offchain-workers-design-assumptions-vulnerabilities/2548>
+//! - <https://substrate.stackexchange.com/questions/11058/how-can-i-create-ocw-that-wont-activates-every-block-but-will-activates-only-w/11060#11060>
+//! - [Offchain worker example](https://github.com/paritytech/polkadot-sdk/tree/master/substrate/frame/examples/offchain-worker)

docs/sdk/src/reference_docs/mod.rs

@@ -100,3 +100,7 @@ pub mod frame_runtime_migration;
 /// light-node-first out of the box.
 // TODO: @jsdw @josepot https://github.com/paritytech/polkadot-sdk-docs/issues/68
 pub mod light_nodes;
+
+/// Learn about the offchain workers, how they function, and how to use them, as provided by the
+/// [`frame`] APIs.
+pub mod frame_offchain_workers;

substrate/frame/support/src/traits/hooks.rs

@@ -434,7 +434,7 @@ pub trait Hooks<BlockNumber> {
 	}
 
 	/// Implementing this function on a pallet allows you to perform long-running tasks that are
-	/// dispatched as separate threads, and entirely independent of the main wasm runtime.
+	/// dispatched as separate threads, and entirely independent of the main blockchain execution.
 	///
 	/// This function can freely read from the state, but any change it makes to the state is
 	/// meaningless. Writes can be pushed back to the chain by submitting extrinsics from the