Move Learn and Under-the-Hood to Guides (#39)

docs/guides/_category_.json

@@ -0,0 +1,4 @@
+{
+  "position": 5,
+  "label": "Guides"
+}

docs/guides/contract-lifecycle.mdx

@@ -0,0 +1,38 @@
+---
+sidebar_position: 3
+title: Contract Lifecycle
+---
+
+## Development
+Contract development can be done on a local computer with as little as 3 necessary components: an IDE, a copy of the Rust toolchain, and a copy of the Soroban SDK.
+
+The SDK contains a full working copy of the host environment, as well as a "mock" version of the ledger for persistent storage of contract data. It is therefore possible (and encouraged) to edit, compile, test and debug contracts directly against this "local" copy of the host, entirely offline and without even accessing a test network.
+
+To make the local development process even more convenient and fast, the contract being developed can (and should) be compiled as native code and linked directly to the local host, rather than compiled to WASM and run in a local VM sandbox. Both configurations are possible, but the native configuration is fastest and provides the richest testing and debugging experience.
+
+The SDK-provided local contract host also contains a local web server that serves the necessary HTTP API endpoint used for client applications to interact with a contract. This can be used for local development of applications, again without needing to deploy contracts to any test or live network.
+## Deployment
+Once a contract has been tested and debugged locally, it can be deployed. To do this it must be compiled to WASM code, and then included by value in a transaction sent to the intended deployment network.
+
+The SDK provides a command-line utility that invokes the Rust compiler with the correct settings for producing a WASM bundle for deployment, but developers can also build this themselves.
+
+Before submitting to the network, developers should inspect the resulting WASM binary emitted by the Rust compiler to ensure that it contains only the intended code and data, and is as small as possible. The SDK command-line utility contains diagnostic commands to assist with this process.
+
+The SDK command-line utility can also build and submit the transaction deploying a WASM contract to the network. Deployment requires sufficient network credentials to sign a transaction performing the deployment and pay its fees. Contracts should be deployed to test networks and thoroughly tested there being deployed to the live network. 
+## Execution
+Deployed contracts live on chain in a CONTRACT_DATA ledger entry. They are executed within a VM sandbox managed by a host environment inside stellar-core. Each transaction that leads to a contract execution is run in a separate host environment, and each contract called by such a transaction (either directly or indirectly from another contract) is executed in a separate guest WASM VM contained within the transaction’s host environment. 
+
+Execution is initiated by a host function called "call". The "call" host function can itself be invoked two different ways: either by someone submitting a transaction to the network that invokes "call" directly, or indirectly by some other contract invoking "call".
+
+In either case, the "call" host function is provided with the ID of a contract to invoke, the name of a function in the contract, and a vector of argument values to pass. The "call" host function then sets up a VM sandbox for the called contract, loads and instantiates its WASM bytecode, and invokes the named function, passing the provided arguments.
+
+Each contract execution continues until the contract either completes successfully or traps with an error condition. If execution completes successfully, all ledger entries modified during execution will be written back to the ledger atomically. If execution traps, all modified ledger entries will be discarded and the contract will have no effect on the ledger.
+
+A variety of conditions in either the guest or host environments can cause a contract to trap. If a host function is called with invalid arguments, for example, the host will trap. Similarly if the contract performs an erroneous WASM bytecode such as a division by zero or access to memory out of bounds, the WASM VM will trap. Also if the contract uses more resources than its enclosing transaction has paid for, the contract will trap.
+## Monitoring
+Contracts can be monitored in two main ways: by observing events emitted during their execution, and by examining the ledger entries written by them.
+
+TBD: expand this section.
+
+## Upgrading contracts
+TBD: depends on decisions pending in CAPs.

docs/guides/debugging.mdx

@@ -0,0 +1,42 @@
+---
+sidebar_position: 9
+title: Debugging
+---
+
+Soroban contracts are as much as possible regular Rust programs and can be debugged using the same tools you'd usually use.
+
+The debugging facilities available differ significantly depending on whether a contract is compiled natively for local testing, or compiled into WASM for deployment.
+
+Deciding between these two modes and making the most of them while debugging requires a **careful understanding** of which code is compiled-in to deployed contracts and which code is only available for local testing.
+
+## Local-testing mode
+
+It is possible (and encouraged during development) to compile Soroban contracts **natively** (eg. as x86-64 or AArch64 code) and link against the host environment **directly**, such that the contract is not a guest running in a WASM virtual machine at all: it is simply one native library calling another -- its host -- and both host and contract are linked together as a single program, together with a test harness.
+
+This configuration is referred to as **"local-testing mode"** and since it eliminates the WASM virtual machine from the debugging experience it has many advantages:
+
+  - Tests run much faster since there is no VM interpreting them, and will execute in parallel by default on multiple threads.
+  - Tests can use numerous standard testing and debugging techniques:
+    - [The standard Rust `#[test]` harness](https://doc.rust-lang.org/reference/attributes/testing.html), including push-button IDE support for running and re-running single tests.
+    - [Standard IDE-supported debuggers](https://code.visualstudio.com/docs/languages/rust#_debugging), including IDE support for setting breakpoints and inspecting values in both contract and host.
+    - Lightweight debugging via [standard logging](https://docs.rs/log/latest/log/) or [tracing](https://docs.rs/tracing/latest/tracing/).
+    - Systematic testing such as [fuzzing](https://crates.io/crates/cargo-fuzz), [property-testing](https://crates.io/crates/proptest), or even [model checking](https://crates.io/crates/kani-verifier) or [formal verification](https://github.com/xldenis/creusot).
+    - The simplest of all debugging approaches, [printing to standard error](https://doc.rust-lang.org/std/macro.eprintln.html).
+
+Local-testing mode is the **default** configuration when compiling code targeting your local computer's CPU and operating system, which is what cargo will do if you set up a new Rust project and don't specify a target.
+
+## WASM mode
+
+If on the other hand you wish to compile for deployment, you must tell cargo to build for the WASM target.
+
+Building for WASM will _disable_ many of the debugging facilities described above, typically for one of three reasons:
+
+  - The WASM VM simply can't (or the VM we've chosen doesn't) provide them.
+  - The WASM VM _could_ provide them but doing so would violate constraints of the [contract Rust dialect](rust-dialect.mdx).
+  - The WASM VM _could_ provide them but doing so would make the resulting WASM code impractically large.
+
+While we encourage most testing to happen in local-testing mode, some problems will obviously only arise in deployment and some debugging facilities thus remain available even there:
+
+  - A "sandbox" host with a mock-ledger that can read and write `CONTRACT_DATA` ledger entries to the local filesystem.
+  - A general logging system that allows contracts to log values of the [shared host/guest "value" type](environment-concepts.mdx), even in production.
+  - User-extensible `Status` codes that can be returned from any contract call to indicate problems.

docs/guides/environment-concepts.mdx

@@ -0,0 +1,47 @@
+---
+sidebar_position: 2
+title: Environment Concepts
+--- 
+
+The contract environment is an _interface_ that defines the facilities -- objects, functions, data sources, etc. -- available to contracts.
+
+##  Host and Guest
+As an interface, the environment has two sides, which we refer to as the host environment and the guest environment. Code in the host environment _implements_ the environment interface; code in the guest environment _uses_ the environment interface.
+
+The **host environment** is provided by a known set of Rust crates, compiled once into stellar-core (or the SDK). Multiple contracts interact with the same host environment, and the host environment has access to any facilities of its enclosing operating system: files, networking, memory, etc.
+
+In contrast, a new **guest environment** is established for each invocation of each smart contract. Each contract sees a single environment interface, and can only call functions provided by the environment interface. In other words, the guest environment is a sandbox for executing arbitrary code within safe parameters.
+
+## WebAssembly
+The on-chain guest environment is isolated inside a WebAssembly (WASM) virtual machine ("VM"). This means that deployed contract code is compiled to WASM bytecode rather than native machine code. The host environment includes an interpreter for the VM, and a new short-lived VM is instantiated for each call to a contract, running the bytecode for the contract and then exiting.
+
+The use of a VM helps provide security against any potential guest-code misbehavior, to both host and other guest environments, as well as ensuring portability of guest code between hosts running on different types of hardware.
+
+When developing and testing contract code off-chain, it is possible to compile contract code to native machine code rather than WASM bytecode, and to [run tests and debug contracts](debugging-contracts.mdx) against a local copy of the host environment by linking directly to it, rather than executing within a VM. This configuration runs much faster and provides much better debugging information, but is only possible locally, off-chain. On-chain deployed contracts are always WASM.
+
+WebAssembly is a relatively low-level VM, which means that it does not provide a very rich set of standard or "built-in" operations. In contrast to VMs like the JVM, it has no garbage collector (not even a memory allocator), no IO facilities, no standard data structures like lists, arrays, maps or strings, no concepts of objects or types at all besides basic machine types like 32 and 64-bit integers. 
+
+As a result, programs compiled to WASM bytecode often face a dilemma: if they want rich standard functionality, they must often include a copy of all the "support code" for that functionality within themselves. But if they do, they dramatically increase their code size, which incurs costs and limits performance. Moreover, including such support code limits their ability to interoperate with other programs that may include different, incompatible support code.
+
+The way out of this dilemma is for the environment itself to provide support code for rich standard functionality, in the form of host objects and functions that guest code can use by reference. Each contract refers to the same functionality implemented in the host, ensuring much smaller code size, higher performance, and greater interoperability between contracts. This is what Soroban does.
+
+## Host objects and functions
+Shared, standard functionality available to all contract guest code is provided through the environment interface in terms of host objects and host functions.
+
+The environment supports a small number of types of host objects covering data structures like vectors, maps, binary blobs, and arbitrary precision numbers. Host objects are all immutable, are allocated and reside within the host environment, and are only available in the guest environment by reference. Guest code refers to host objects by integer-valued handles.
+
+There is also a slightly larger set of host functions that act on host objects: creating, modifying, inspecting and manipulating them. Some host functions allow copying blocks of binary data into and out of the VM memory of the guest, and some host functions perform cryptographic operations on host objects.
+
+There are also host functions for interacting with select components of the host environment beyond the host object repertoire, such as reading and writing ledger entries, emitting events, calling other contracts, and accessing information about the transaction context in which guest code is executing.
+
+### Serialization
+Host objects can be passed (by handle) directly to storage routines or between collaborating contracts. **No serialization or deserialization code needs to exist in the guest**: the host knows how to serialize and deserialize all of its object types and does so transparently whenever necessary.
+
+## Values and types
+All host functions can accept as arguments and return values from, at most, the limited WASM VM repertoire of machine-level types. To simplify matters, Soroban further limits all host functions to passing and returning values from within a single specialized form of 64-bit integers called "value" or "the value type". Through careful bit-packing, the value type can encode any of several separate types more meaningful to users than just "integers".
+
+Specifically, the value type can directly encode 63-bit unsigned integers (equivalent to positive signed 64-bit numbers), but also boolean true and false, signed or unsigned 32-bit integers, typed host object handles, typed error codes, small symbols (up to 10 latin-alphanumeric characters), small bitsets (up to 60 bits), or a unique void value. Individual bits in a value are allocated to tagging and switching between these cases dynamically, and host functions or objects that require specific cases may reject values of other cases.
+
+Since the value type can contain a handle to a host object, any container object that can contain the value type can in fact hold any object. Therefore the host map and vector types -- container data structures -- are defined merely as containers for the value type, where the specific case of each value may vary from container to container or even among the elements of a container. In this way, the host container types are more like the containers of dynamic-typed languages like JavaScript or Python. The SDK also provides static, uniformly-typed wrappers when this is desired, prohibiting values outside the designated case from being added to the container.
+
+![](/diagrams/environment-concepts.png)

docs/guides/events.mdx

@@ -0,0 +1,11 @@
+---
+sidebar_position: 6
+title: Events
+---
+
+Events are the mechanism that applications off-chain can use to monitor changes
+and events in contracts on-chain.
+
+:::danger
+Soroban events are under active-development and are not yet available.
+:::

docs/guides/gas-and-metering.mdx

@@ -0,0 +1,12 @@
+---
+sidebar_position: 8
+title: Gas and metering
+---
+
+Gas and metering are the mechanisms that will measure the cost of running a
+contract and bill that cost to the account executing the contract.
+
+:::danger
+Soroban gas and metering are under active-development and are not yet available.
+:::
+

docs/guides/high-level-overview.mdx

@@ -0,0 +1,30 @@
+---
+sidebar_position: 1
+title: High level overview
+---
+
+## Rust language
+Contracts are small programs written in the [Rust programming language](https://www.rust-lang.org). In order to write contracts, [install a Rust toolchain](https://www.rust-lang.org/tools/install), configure your [editor to support Rust programs](https://www.rust-lang.org/tools), and [learn at least some basic Rust concepts](https://www.rust-lang.org/learn).
+
+Contracts can be compiled to native code for local (off-chain) testing, but must be compiled as WebAssembly ("WASM") for deployment. The on-chain host environment only allows uploading WASM contracts, and runs them within a WASM "sandbox" virtual machine ("VM").
+
+In practice several special characteristics of the contract execution environment -- resource constraints, security considerations, nonstandard IO and invocation primitives, the requirement for strict determinism -- will mean that contracts can only use a very narrow subset of the full Rust language, and must use specialized libraries for most tasks. See the [rust dialect](rust-dialect.mdx) section for details.
+
+In particular both the Rust standard library and most 3rd party libraries ("crates" in Rust parlance) will not be available for direct off-the-shelf use in contracts, due to the constraints listed above. Some crates will be possible to adapt to use in contracts, others may be incorporated into the host environment as host objects or functions.
+
+In the future it is possible that other source languages may be supported. The execution environment is somewhat language-agnostic, and some very early experiments suggest that other languages may be possible. However, at this time only Rust is supported.
+
+## SDK
+Contracts are developed with the help of a Software Development Kit or SDK. The SDK consists of both a Rust crate and a command-line tool.
+
+The SDK crate acts as a substitute for the Rust standard library -- providing data structures and utility functions for contracts -- as well as providing access to smart-contract-specific functionality from the contract environment, like cryptographic hashing and signature verification, access to on-chain persistent storage, and location and invocation of secondary contracts via stable identifiers.
+
+### Local testing mode
+The SDK command-line tool provides a developer-focused front-end for compiling, testing, inspecting, versioning and deploying contracts.
+
+The SDK also includes a _complete implementation of the contract host environment_ -- identical to the one that runs on-chain -- such that contracts can be **run locally** on a developer's workstation, and can be [tested and debugged](debugging-contracts.mdx) directly with a local debugger within a standard IDE, as well as a native test harness for fast-feedback unit testing and high-speed fuzzing or property testing.
+
+## Host Environment
+The host environment is a set of Rust crates that are compiled into both the SDK command-line tool and stellar-core, the transaction processor at the heart of the stellar network. It comprises a set of host objects and functions, an interface to on-chain storage and contract invocation, a resource-accounting and fee-charging system, and a WASM interpreter.
+
+Most contract developers will not frequently need to interact with the host environment directly -- SDK functions wrap most of its facilities and provide richer and more ergonomic types and functions -- but it is helpful to understand its structure in order to understand the conceptual model the SDK is presenting. It is also likely that some parts of the host environment will be visible when testing or debugging contracts compiled natively on a local workstation.