Clarify onions part 2: a bit deeper rework

04-onion-routing.md

@@ -50,7 +50,8 @@ A node:
   * [Packet Structure](#packet-structure)
     * [Payload Format](#payload-format)
     * [Basic Multi-Part Payments](#basic-multi-part-payments)
-    * [Route Blinding](#route-blinding)
+  * [Route Blinding](#route-blinding)
+    * [Inside encrypted_recipient_data: encrypted_data_tlv](Inside-encrypted_recipient_data-encrypted_data_tlv)
   * [Accepting and Forwarding a Payment](#accepting-and-forwarding-a-payment)
     * [Payload for the Last Node](#payload-for-the-last-node)
     * [Non-strict Forwarding](#non-strict-forwarding)
@@ -203,7 +204,7 @@ This is formatted according to the Type-Length-Value format defined in [BOLT #1]
         * [`tu64`:`total_msat`]
     1. type: 10 (`encrypted_recipient_data`)
     2. data:
-        * [`...*byte`:`encrypted_data`]
+        * [`...*byte`:`encrypted_recipient_data`]
     1. type: 12 (`current_path_key`)
     2. data:
         * [`point`:`path_key`]
@@ -435,124 +436,113 @@ otherwise meets the amount criterion (eg. some other failure, or
 invoice timeout), however if it were to fulfill only some of them,
 intermediary nodes could simply claim the remaining ones.
 
-### Route Blinding
+## Route Blinding
 
-Nodes receiving onion packets may hide their identity from senders by
-"blinding" an arbitrary amount of hops at the end of an onion path.
+1. subtype: `blinded_path`
+2. data:
+   * [`point`:`first_node_id`]
+   * [`point`:`first_path_key`]
+   * [`byte`:`num_hops`]
+   * [`num_hops*blinded_path_hop`:`path`]
 
-When using route blinding, nodes find a route to themselves from a given
-"introduction node" and initial "path key". They then use ECDH with
-each node in that route to create a "blinded" node ID and an encrypted blob
-(`encrypted_data`) for each one of the blinded nodes.
+1. subtype: `blinded_path_hop`
+2. data:
+    * [`point`:`blinded_node_id`]
+    * [`u16`:`enclen`]
+    * [`enclen*byte`:`encrypted_recipient_data`]
 
-They communicate this blinded route and the encrypted blobs to the sender.
-The sender finds a route to the introduction node and extends it with the
-blinded route provided by the recipient. The sender includes the encrypted
-blobs in the corresponding onion payloads: they allow nodes in the blinded
-part of the route to "unblind" the next node and correctly forward the packet.
+A blinded path consists of:
+1. an initial introduction point (`first_node_id`)
+2. an initial key to share a secret with the first node_id (`first_path_key`)
+3. a series of tweaked node ids (`path.blinded_node_id`)
+4. a series of binary blobs encrypted to the nodes (`path.encrypted_recipient_data`)
+   to tell them the next hop.
 
-Note that there are two ways for the sender to reach the introduction
-point: one is to create a normal (unblinded) payment, and place the
-initial blinding point in `current_path_key` along with the
-`encrypted_data` in the onion payload for the introduction point to
-start the blinded path. The second way is to create a blinded path to
-the introduction point, set `next_path_key_override` inside the
-`encrypted_data_tlv` on the hop prior to the introduction point to the
-initial blinding point, and have it sent to the introduction node.
+For example, Dave wants Alice to reach him via public node Bob then
+Carol.  He creates a chain of public keys ("path_keys") for Bob, Carol
+and finally himself, so he can share a secret with each of them.  These
+keys are a simple chain, so each node can derive the next `path_key` without
+having to be told explicitly.
 
-The `encrypted_data` is a TLV stream, encrypted for a given blinded node, that
-may contain the following TLV fields:
+From these shared secrets, Dave creates and encrypts three `encrypted_data_tlv`s:
+1. encrypted_data_bob: For Bob to tell him to forward to Carol
+2. encrypted_data_carol: For Carol to tell her to forward to him
+3. encrypted_data_dave: For himself to indicate the path was used, and any metadata he wants.
 
-1. `tlv_stream`: `encrypted_data_tlv`
-2. types:
-    1. type: 1 (`padding`)
-    2. data:
-        * [`...*byte`:`padding`]
-    1. type: 2 (`short_channel_id`)
-    2. data:
-        * [`short_channel_id`:`short_channel_id`]
-    1. type: 4 (`next_node_id`)
-    2. data:
-        * [`point`:`node_id`]
-    1. type: 6 (`path_id`)
-    2. data:
-        * [`...*byte`:`data`]
-    1. type: 8 (`next_path_key_override`)
-    2. data:
-        * [`point`:`path_key`]
-    1. type: 10 (`payment_relay`)
-    2. data:
-        * [`u16`:`cltv_expiry_delta`]
-        * [`u32`:`fee_proportional_millionths`]
-        * [`tu32`:`fee_base_msat`]
-    1. type: 12 (`payment_constraints`)
-    2. data:
-        * [`u32`:`max_cltv_expiry`]
-        * [`tu64`:`htlc_minimum_msat`]
-    1. type: 14 (`allowed_features`)
-    2. data:
-        * [`...*byte`:`features`]
+To mask the node ids, he also derives three blinding factors from the
+shared secrets, which turn Bob into Bob', Carol into Carol' and Dave
+into Dave'.
 
-#### Requirements
+So this is the `blinded_path` he hands to Alice.
+
+1. `first_node_id`: Bob
+2. `first_path_key`: the first path key for Bob 
+3. `path`: [Bob', encrypted_data_bob], [Carol', encrypted_data_carol], [Dave', encrypted_data_dave]
+
+There are two different ways for Alice to construct an onion which gets to Bob (since he's probably not a direct peer of hers) which are described in the requirements below.
+
+But after Bob the path is always the same: he will send Carol the `path_key` he derived, along with the onion.  She will use the `path_key` to derive the tweak for the onion (which Alice encrypted for Carol' not Carol) so she can decrypt it, and also to derive the key to decrypt `encrypted_data_tlv` which will tell her to forward to Dave (and possibly additional restrictions Dave specified).
+
+### Requirements
 
-A recipient $`N_r`$ creating a blinded route $`N_0 \rightarrow N_1 \rightarrow ... \rightarrow N_r`$ to itself:
+Note that the creator of the blinded path (i.e. the recipient) is creating it for the sender to use to create an onion, and for the intermediate nodes to read the instructions, hence there are two reader sections here.
 
-- MUST create a blinded node ID $`B_i`$ for each node using the following algorithm:
+The writer of a `blinded_path`:
+
+- MUST create a viable path to itself ($`N_r`$) i.e. $`N_0 \rightarrow N_1 \rightarrow ... \rightarrow N_r`$.
+- MUST set `first_node_id` to $`N_0`$
+- MUST create a series of ECDH shared secrets for each node in the route using the following algorithm:
   - $`e_0 \leftarrow \{0;1\}^{256}`$ ($`e_0`$ SHOULD be obtained via CSPRNG)
   - $`E_0 = e_0 \cdot G`$
   - For every node in the route:
     - let $`N_i = k_i * G`$ be the `node_id` ($`k_i`$ is $`N_i`$'s private key)
     - $`ss_i = SHA256(e_i * N_i) = SHA256(k_i * E_i)`$ (ECDH shared secret known only by $`N_r`$ and $`N_i`$)
-    - $`B_i = HMAC256(\text{"blinded\_node\_id"}, ss_i) * N_i`$ (blinded `node_id` for $`N_i`$, private key known only by $`N_i`$)
-    - $`rho_i = HMAC256(\text{"rho"}, ss_i)`$ (key used to encrypt the payload for $`N_i`$ by $`N_r`$)
+    - $`rho_i = HMAC256(\text{"rho"}, ss_i)`$ (key used to encrypt `encrypted_recipient_data` for $`N_i`$ by $`N_r`$)
     - $`e_{i+1} = SHA256(E_i || ss_i) * e_i`$ (ephemeral private path key, only known by $`N_r`$)
     - $`E_{i+1} = SHA256(E_i || ss_i) * E_i`$ (`path_key`. NB: $`N_i`$ MUST NOT learn $`e_i`$)
+- MUST set `first_path_key` to $`E_0`$
+- MUST create a series of blinded node IDs $`B_i`$ for each node using the following algorithm:
+  - $`B_i = HMAC256(\text{"blinded\_node\_id"}, ss_i) * N_i`$ (blinded `node_id` for $`N_i`$, private key known only by $`N_i`$)
+  - MUST set `blinded_node_id` for each `blinded_path_hop` in `path` to $`B_i`$
 - MAY replace $`E_{i+1}`$ with a different value, but if it does:
   - MUST set `encrypted_data_tlv[i].next_path_key_override` to $`E_{i+1}`$
 - MAY store private data in `encrypted_data_tlv[r].path_id` to verify that the route is used in the right context and was created by them
 - SHOULD add padding data to ensure all `encrypted_data_tlv[i]` have the same length
 - MUST encrypt each `encrypted_data_tlv[i]` with ChaCha20-Poly1305 using the corresponding $`rho_i`$ key and an all-zero nonce to produce `encrypted_recipient_data[i]`
-- MUST communicate the blinded node IDs $`B_i`$ and `encrypted_recipient_data[i]` to the sender
-- MUST communicate the real node ID of the introduction point $`N_0`$ to the sender
-- MUST communicate the first `path_key` $`E_0`$ to the sender
+- MAY add additional "dummy" hops at the end of the path (which it will ignore on receipt) to obscure the path length.
+
+The reader of the `blinded_path`:
+- MUST prepend its own onion payloads to reach the `first_node_id`
+- MUST include the corresponding `encrypted_recipient_data` in each onion payload within `path`
+- For the first entry in `path`:
+  - if it is sending a payment:
+    - SHOULD create an unblinded onion payment to `first_node_id`, and include `first_path_key` as `current_path_key`.
+  - otherwise:
+    - MUST encrypt the first blinded path onion to the first `blinded_node_id`.
+    - MUST set `next_path_key_override` in the prior onion payload to `first_path_key`.
+- For each successive entry in `path`:
+  - MUST encrypt the onion to the corresponding `blinded_node_id`.
 
-A reader:
+The reader of the `encrypted_recipient_data`:
 
-- If it receives `path_key` ($`E_i`$) from the prior peer:
-  - MUST use $`b_i`$ instead of its private key $`k_i`$ to decrypt the onion.
-    Note that the node may instead tweak the onion ephemeral key with
-    $`HMAC256(\text{"blinded\_node\_id"}, ss_i)`$ which achieves the same result.
-- Otherwise:
-  - MUST use $`k_i`$ to decrypt the onion, to extract `current_path_key` ($`E_i`$).
 - MUST compute:
   - $`ss_i = SHA256(k_i * E_i)`$ (standard ECDH)
   - $`b_i = HMAC256(\text{"blinded\_node\_id"}, ss_i) * k_i`$
   - $`rho_i = HMAC256(\text{"rho"}, ss_i)`$
-  - $`E_{i+1} = SHA256(E_i || ss_i) * E_i`$
-- MUST decrypt the `encrypted_data` field using $`rho_i`$ and use the
-  decrypted fields to locate the next node
-- If the `encrypted_data` field is missing or cannot be decrypted:
+- MUST decrypt the `encrypted_recipient_data` field using $`rho_i`$ as a key using ChaCha20-Poly1305 and an all-zero nonce key.
+- If the `encrypted_recipient_data` field is missing, cannot be decrypted into an `encrypted_data_tlv` or contains unknown even fields:
   - MUST return an error
-- If `encrypted_data` contains a `next_path_key_override`:
-  - MUST use it as the next `path_key` instead of $`E_{i+1}`$
+- If the `encrypted_data_tlv` contains a `next_path_key_override`:
+  - MUST use it as the next `path_key`.
 - Otherwise:
-  - MUST use $`E_{i+1}`$ as the next `path_key`
+  - MUST use $`E_{i+1} = SHA256(E_i || ss_i) * E_i`$ as the next `path_key`
 - MUST forward the onion and include the next `path_key` in the lightning
   message for the next node
+- If it is the final recipient:
+  - MUST ignore the message if the `path_id` does not match the blinded route it
+    created for this purpose
 
-The final recipient:
-
-- MUST compute:
-  - $`ss_r = SHA256(k_r * E_r)`$ (standard ECDH)
-  - $`b_r = HMAC256(\text{"blinded\_node\_id"}, ss_r) * k_r`$
-  - $`rho_r = HMAC256(\text{"rho"}, ss_r)`$
-- MUST decrypt the `encrypted_data` field using $`rho_r`$
-- If the `encrypted_data` field is missing or cannot be decrypted:
-  - MUST return an error
-- MUST ignore the message if the `path_id` does not match the blinded route it
-  created
-
-#### Rationale
+### Rationale
 
 Route blinding is a lightweight technique to provide recipient anonymity.
 It's more flexible than rendezvous routing because it simply replaces the public
@@ -561,13 +551,17 @@ choose what data they put in the onion for each hop. Blinded routes are also
 reusable in some cases (e.g. onion messages).
 
 Each node in the blinded route needs to receive $`E_i`$ to be able to decrypt
-the onion and the `encrypted_data` payload. Protocols that use route blinding
-must specify how this value is propagated between nodes.
+the onion and the `encrypted_recipient_data` payload.
 
 When concatenating two blinded routes generated by different nodes, the
 last node of the first route needs to know the first `path_key` of the
 second route: the `next_path_key_override` field must be used to transmit this
-information.
+information.  In theory this method could be used for payments (not just
+onion messages), but we recommend using an unblinded path to reach the 
+`first_node_id` and using `current_path_key` there: this means that the
+node can tell it is being used as an introductory point, but also does
+not require blinded path support on the nodes to reach that point, and
+gives meaningful errors on the unblinded part of the payment.
 
 The final recipient must verify that the blinded route is used in the right
 context (e.g. for a specific payment) and was created by them. Otherwise a
@@ -585,7 +579,7 @@ should add a random delay before forwarding the error. Failures are likely to
 be probing attempts and message timing may help the attacker infer its distance
 to the final recipient.
 
-The `padding` field can be used to ensure that all `encrypted_data` have the
+The `padding` field can be used to ensure that all `encrypted_recipient_data` have the
 same length. It's particularly useful when adding dummy hops at the end of a
 blinded route, to prevent the sender from figuring out which node is the final
 recipient.
@@ -599,6 +593,48 @@ set `payment_constraints.max_cltv_expiry` to restrict the lifetime of a blinded
 route and reduce the risk that an intermediate node updates its fees and rejects
 payments (which could be used to unblind nodes inside the route).
 
+### Inside `encrypted_recipient_data`: `encrypted_data_tlv`
+
+The `encrypted_recipient_data` is a TLV stream, encrypted for a given blinded node, that
+may contain the following TLV fields:
+
+1. `tlv_stream`: `encrypted_data_tlv`
+2. types:
+    1. type: 1 (`padding`)
+    2. data:
+        * [`...*byte`:`padding`]
+    1. type: 2 (`short_channel_id`)
+    2. data:
+        * [`short_channel_id`:`short_channel_id`]
+    1. type: 4 (`next_node_id`)
+    2. data:
+        * [`point`:`node_id`]
+    1. type: 6 (`path_id`)
+    2. data:
+        * [`...*byte`:`data`]
+    1. type: 8 (`next_path_key_override`)
+    2. data:
+        * [`point`:`path_key`]
+    1. type: 10 (`payment_relay`)
+    2. data:
+        * [`u16`:`cltv_expiry_delta`]
+        * [`u32`:`fee_proportional_millionths`]
+        * [`tu32`:`fee_base_msat`]
+    1. type: 12 (`payment_constraints`)
+    2. data:
+        * [`u32`:`max_cltv_expiry`]
+        * [`tu64`:`htlc_minimum_msat`]
+    1. type: 14 (`allowed_features`)
+    2. data:
+        * [`...*byte`:`features`]
+
+#### Rationale
+
+Encrypted recipient data is created by the final recipient to give to the
+sender, containing instructions for the node on how to handle the message (it can also be created by the sender themselves: the node forwarding cannot tell).  It's used
+in both payment onions and onion messages onions.  See [Route Blinding](#route-blinding).
+
+
 # Accepting and Forwarding a Payment
 
 Once a node has decoded the payload it either accepts the payment locally, or forwards it to the peer indicated as the next hop in the payload.
@@ -1459,7 +1495,7 @@ For consistency, all onion messages use [Route Blinding](#route-blinding).
    * `filler`
 
 The `onionmsg_tlv` itself is a TLV: an intermediate node expects an
-`encrypted_data` which it can decrypt into an `encrypted_data_tlv`
+`encrypted_recipient_data` which it can decrypt into an `encrypted_data_tlv`
 using the `path_key` which it is handed along with the onion message.
 
 Field numbers 64 and above are reserved for payloads for the final
@@ -1475,19 +1511,6 @@ even, of course!).
     2. data:
         * [`...*byte`:`encrypted_recipient_data`]
 
-1. subtype: `blinded_path`
-2. data:
-   * [`point`:`first_node_id`]
-   * [`point`:`first_path_key`]
-   * [`byte`:`num_hops`]
-   * [`num_hops*onionmsg_hop`:`path`]
-
-1. subtype: `onionmsg_hop`
-2. data:
-    * [`point`:`blinded_node_id`]
-    * [`u16`:`enclen`]
-    * [`enclen*byte`:`encrypted_recipient_data`]
-
 #### Requirements
 
 The creator of `encrypted_recipient_data` (usually, the recipient of the onion):