Update private message delivery and add new call types

yellow-paper/docs/addresses-and-keys/precompiles.md

@@ -55,10 +55,19 @@ encrypt_and_tag<N>({ public_keys: Field[], recipient: AztecAddress, plaintext: F
 
 Same as `encrypt_and_tag`, but batched using the same logic as `encrypt<N>`.
 
+```
+decrypt(public_keys: Field[], owner: AztecAddress, cyphertext: Field[]): Field[]
+```
+
+Decrypts the given cyphertext, encrypted for the provided owner. Instead of receiving the decryption key, this method triggers an oracle call to fetch the private decryption key directly from the local PXE and validates it against the supplied public key, in order to avoid leaking a user secret to untrusted application code. This method is intended for provable decryption use cases.
+
+
 ## Defined precompiles
 
 List of precompiles defined by the protocol and their assigned address.
 
+<!-- TODO: Should we have a precompile for delegation? Or handle that at the registry/app level? Probably registry, since precompiles cannot go back to the registry to re-read? -->
+
 ### Noop
 
 Address `0x01` is defined to always be a noop. Accounts that explicitly signal that they cannot receive encrypted payloads can advertise this precompile. Validation method returns `true` only for an empty list of public keys, and all other methods return empty.

yellow-paper/docs/calls/batched-calls.md

@@ -0,0 +1,26 @@
+---
+sidebar_position: 3
+---
+
+# Batched calls
+
+Calls to private functions can be _batched_ instead of executed [synchronously](./sync-calls.md). When executing a batched call to a private function, the function is not executed on the spot, but enqueued for execution at the end of local execution. Once the private call stack has been emptied, all batched execution requests are grouped by target, defined as recipient and function selector, and executed via a single call to each target.
+
+Batched calls are implemented by pushing a `PrivateCallStackItem` with the flag `is_execution_request` into a `private_batched_queue` in the execution context, and require an oracle call to `batchPrivateFunctionCall` with the same arguments as other oracle function calls.
+
+Batched calls are processed by the private kernel circuit. On each kernel circuit iteration, if the private call stack is not empty, the kernel circuit pops and processes the topmost entry. Otherwise, if the batched queue is not empty, the kernel pops the first item, collects and deletes all other items with the same target, and calls into the target with the concatenation of the arguments for all collected calls. Note that this allows batched calls to trigger synchronous calls.
+
+In pseudocode, the kernel circuit executes the following logic:
+
+```
+loop:
+  if next_call_stack_item = context.private_call_stack.pop():
+    execute(next_call_stack_item.address, next_call_stack_item.function_selector, next_call_stack_item.arguments)
+  else if next_batched_call = context.private_batched_queue.pop():
+    let calls = context.private_batched_queue.get_and_delete(enqueued_call.target == target)
+    execute(target.address, target.function_selector, calls.map(arguments))
+  else:
+    break
+```
+
+The rationale for batched calls is to minimize the number of function calls in private execution, in order to reduce total proving times. Batched calls are mostly intended for usage with note delivery precompiles, since these do not require synchronous execution, and allows for processing all notes to be encrypted and tagged with the same mechanism using the same call. Batched calls can also be used for other common functions that do not require to be executed synchronously and are likely to be invoked multiple times.

yellow-paper/docs/calls/index.md

@@ -4,7 +4,7 @@ title: Calls
 
 # Calls
 
-Functions in the Aztec Network can call other functions. These calls are [synchronous](./sync-calls.md) when they they occur within private functions or within public functions, but are [enqueued](./enqueued-calls.md) when done from a private to a public function. The protocol also supports alternate call methods, such as static calls.
+Functions in the Aztec Network can call other functions. These calls are [synchronous](./sync-calls.md) when they they occur within private functions or within public functions, but are [enqueued](./enqueued-calls.md) when done from a private to a public function, and optionally [batched](./batched-calls.md). The protocol also supports alternate call methods, such as [static](./static-calls.md) and [unconstrained](./unconstrained-calls.md) calls.
 
 In addition to function calls, the protocol allows for communication via message-passing back-and-forth between L1 and L2, as well as from public to private functions.
 

yellow-paper/docs/calls/public_private_messaging.md

@@ -1,5 +1,5 @@
 ---
-sidebar_position: 5
+sidebar_position: 10
 ---
 
 # Inter-Layer Calls

yellow-paper/docs/calls/static-calls.md

@@ -1,5 +1,5 @@
 ---
-sidebar_position: 3
+sidebar_position: 5
 ---
 # Static calls
 

yellow-paper/docs/calls/unconstrained-calls.md

@@ -0,0 +1,15 @@
+---
+sidebar_position: 6
+---
+
+# Unconstrained calls
+
+<!-- TODO: Validate use cases for unconstrained calls. Maybe these are not actually needed? -->
+
+Private function calls can be executed as _unconstrained_. Unconstrained function calls execute the code at the target and return the result, but their execution is not constrained. It is responsibility of the caller to constrain the result, if needed. Unconstrained calls are a generalization of oracle function calls, where the call is not to a PXE function but to another contract. Side effects from unconstrained calls are ignored. Note that all calls executed from an unconstrained call frame will be unconstrained as well.
+
+Unconstrained calls are executed via a `unconstrainedCallPrivateFunction` oracle call, which accepts the same arguments as a regular `callPrivateFunction`, and return the result from the function call. Unconstrained calls are not pushed into the `private_call_stack` and do not incur in an additional kernel iteration.
+
+Rationale for unconstrained calls is to allows apps to consume results from functions that do not need to be provable. An example use case for unconstrained calls is unconstrained encryption and note tagging, which can be used when the sender is incentivized to ensure the recipient receives the data sent. 
+
+Another motivation for unconstrained calls is for retrieving or computing data where the end result can be more efficiently constrained by the caller.

yellow-paper/docs/private-message-delivery/_category_.json

@@ -3,6 +3,6 @@
   "position": 2,
   "link": {
     "type": "generated-index",
-    "description": "Delivering messages privately on the Aztec network..."
+    "description": "Private message delivery encompasses the encryption, tagging, and broadcasting of private messages on the Aztec Network."
   }
 }

yellow-paper/docs/private-message-delivery/encryption-and-decryption.md

@@ -1,6 +1,25 @@
 ---
-sidebar_position: 2
+sidebar_position: 3
 ---
 
 # Encryption and Decryption
 
+Applications should be able to provably encrypt data for a target user, as part of private message delivery. As stated on the Keys section, we define three types of encrypted data, based on the sender and the recipient, from the perspective of a user:
+
+- Incoming data: data created by someone else, encrypted for and sent to the user.
+- Outgoing data: data created by the user to be sent to someone else, encrypted for the user.
+- Internal incoming data: data created by the user, encrypted for and sent to the user.
+
+Encryption mechanisms support these three types of encryption, which may rely on different keys advertised by the user.
+
+## Precompiles and Note Discovery
+
+Even though encryption is a well-solved problem, unlike note discovery, the protocol bundles both operations together for simplicity and efficiency. Most use cases call for encryption and note tagging to be executed together, so note tagging precompile contracts are expected to handle encryption as well. This allows users to choose their preferred encryption method, trading between encryption cost and bits of security.
+
+## Key Abstraction
+
+To support different kinds of encryption mechanisms, the protocol does not make any assumptions on the type of public keys advertised by each user. Validation of their public keys is handled by the precompile contract selected by the user.
+
+## Provable Decryption
+
+While provable encryption is required to guarantee correct private message delivery, provable decryption is required for disclosing activity within an application. This allows auditability and compliance use cases, as well as being able to prove that a user did not execute certain actions. To support this, encryption precompiles also allow for provable decryption.

yellow-paper/docs/private-message-delivery/note-discovery.md

@@ -1,15 +1,23 @@
 ---
-sidebar_position: 3
+sidebar_position: 2
 ---
 
 # Note Discovery
 
-## Requirements
-
 When users interact with contracts they will generate and publish encrypted notes for other network participants. In order for a user to consume notes that belong to them, they need to identify, retrieve and decrypt them. A simple, privacy-preserving approach to this would be to download all of the notes and attempt decryption. However, the total number of encrypted notes published by the network will be substantial, making it infeasible for some users to do this. Those users will want to utilize a note discovery protocol to privately identify their notes.
 
-A number of techniques currently exist to help with this and it is a field into which a lot of research is being conducted. Therefore, our approach is not to dictate or enshrine a specific note discovery mechanism but to put in place the necessary abstractions such that users can freely choose. Additionally, through this approach we allow for integration of new or improved protocols in the future.
+## Precompiles
+
+A number of techniques currently exist to help with note discovery and it is a field into which a lot of research is being conducted, where each technique has its tradeoffs. Therefore, our approach is not to dictate a specific note discovery mechanism, but to implement multiple options via precompiles that users can choose from. These precompiles define both encryption and tagging mechanisms, and allow constraining their correct execution. Additionally, through this approach we allow for integration of new or improved protocols in the future.
+
+> Note: Constraining tag generation is not solely about ensuring that the generated tag is of the correct format. It is also necessary to constrain that tags are generated in the correct sequence. A tag sequence with duplicate or missing tags makes it much more difficult for the recipient to retrieve their notes. This will likely require tags to be nullified once used.
 
 ## Tag Abstraction
 
-When applications produce notes they will need to call a protocol defined function within the account contract of the recipient and request that a tag be generated. From the protocol's perspective, this tag will simply be a stream of bytes relevant only to the recipient's note discovery protocol. It will be up to the account contract to constrain that the correct tag has been generated and from there the protocol circuits along with the rollup contract will ensure that the tag is correctly published along with the note.
+When applications produce notes they will need to call a protocol defined function within the account contract of the recipient and request that a tag be generated. From the protocol's perspective, this tag will simply be a stream of bytes relevant only to the recipient's note discovery protocol. It will be up to the account contract to constrain that the correct tag has been generated and from there the protocol circuits along with the rollup contract will ensure that the tag is correctly published along with the note.
+
+## User Handshaking
+
+Even if Alice correctly encrypts the note she creates for Bob and generates the correct tag to go with it, how does Bob know that Alice has sent him a note? Bob's note discovery protocol may require him to speculatively 'look' for notes with the tags that Alice (and his other counterparties) have generated. If Alice and Bob know each other then they can communicate out-of-protocol. But if they have no way of interacting then the network needs to provide a mechanism by which Bob can be alerted to the need to start searching for a specific sequence of tags.
+
+To facilitate this we will deploy a canonical 'handshake' contract that can be used to create a private note for a recipient containing the sender's information (e.g. public key). It should only be necessary for a single handshake to take place between two users. The notes generated by this contract will be easy to identify enabling users to retrieve these notes, decrypt them and use the contents in any deterministic tag generation used by their chosen note discovery protocol.

yellow-paper/docs/private-message-delivery/private-message-delivery.md

@@ -4,39 +4,35 @@ sidebar_position: 1
 
 # Private Message Delivery
 
-## Requirements
-
 Maintaining the core tenet of privacy within the Aztec Network imposes a number of requirements related to the transfer of notes from one user to another. If Alice executes a function that generates a note for Bob:
 
-1. Alice will need to encrypt that note such that Bob, and only Bob is able to decrypt it.
-2. Alice will need to broadcast the encrypted note so as to make it available for Bob to retrieve.
-3. Alice will need to broadcast a 'tag' alongside the encrypted note. This tag must be identifiable by Bob's chosen [note discovery protocol](./note-discovery.md) but not identifiable by any third party.
+1. Alice will need to **encrypt** that note such that Bob, and only Bob is able to decrypt it.
+2. Alice will need to **broadcast** the encrypted note so as to make it available for Bob to retrieve.
+3. Alice will need to **broadcast a 'tag'** alongside the encrypted note. This tag must be identifiable by Bob's chosen [note discovery protocol](./note-discovery.md) but not identifiable by any third party.
+
+## Requirements
 
-Fulfilling these requirements will enable users to privately identify, retrieve, decrypt and consume their application notes.
+- **Users must be able to choose their note tagging mechanism**. We expect improved note discovery schemes to be designed over time. The protocol should be flexible enough to accommodate them and for users to opt in to using them as they become available. This flexibility should be extensible to encryption mechanisms as well as a soft requirement.
+- **Users must be able to receive notes before interacting with the network**. A user should be able to receive a note just by generating an address. It should not be necessary for them to deploy their account contract in order to receive a note.
+- **Applications must be able to safely send notes to any address**. Sending a note to an account could potentially transfer control of the call to that account, allowing the account to control whether they want to accept the note or not, and potentially bricking an application, since there is no catching exceptions in private function execution.
+- **Addresses must be as small as possible**. Addresses will be stored and broadcasted constantly in applications. Larger addresses means more data usage, which is the main driver for cost. Addresses must fit in at most 256 bits, or ideally a single field element.
+- **Total number of function calls should be minimized**. Every function call requires an additional iteration of the private kernel circuit, which adds several seconds of proving time.
+- **Encryption keys should be rotatable**. Users should be able to rotate their encryption keys in the event their private keys are compromised, so that any further interactions with apps can be private again, without having to migrate to a new account.
 
 ## Constraining Message Delivery
 
-The network will constrain:
+The protocol will allow constraining:
 
 1. The encryption of a user's note.
 2. The generation of the tag for that note.
 3. The publication of that note to the correct data availability layer.
 
-Constraining [note encryption](./encryption-and-decryption.md) and tagging will be done through protocol defined functions within a user's account contract. The advantages of this approach are:
+Each app will define whether to constrain each step in private message delivery. Encryption and tagging will be done through a set of precompiled contracts, each contract offering a different mechanism, and users will advertise their preferred mechanisms in a canonical registry.
+
+The advantages of this approach are:
 
-1. It enables a user to select their preferred [note discovery protocol](./note-discovery.md) and/or encryption scheme.
+1. It enables a user to select their preferred [note discovery protocol](./note-discovery.md) and [encryption scheme](./encryption-and-decryption.md).
 2. It ensures that notes are correctly encrypted with a user's public encryption key.
 3. It ensures that notes are correctly tagged for a user's chosen [note discovery protocol](./note-discovery.md).
 4. It provides scope for upgrading these functions or introducing new schemes as the field progresses.
-5. It protects applications from malicious account contracts providing unprovable functions.
-
-> Note: Constraining tag generation is not solely about ensuring that the generated tag is of the correct format. It is also necessary to constrain that tags are generated in the correct sequence. A tag sequence with duplicate or missing tags makes it much more difficult for the recipient to retrieve their notes. This will likely require tags to be nullified once used.
-
-Constraining publication to the correct data availability layer will be performed via a combination of the protocol circuits and the rollup contract on L1.
-
-## User Handshaking
-
-Even if Alice correctly encrypts the note she creates for Bob and generates the correct tag to go with it, how does Bob know that Alice has sent him a note? Bob's [note discovery protocol](./note-discovery.md) may require him to speculatively 'look' for notes with the tags that Alice (and his other counterparties) have generated. If Alice and Bob know each other then they can communicate out-of-protocol. But if they have no way of interacting then the network needs to provide a mechanism by which Bob can be alerted to the need to start searching for a specific sequence of tags.
-
-To facilitate this we will deploy a 'handshake' contract that can be used to create a private note for a recipient containing the sender's information (e.g. public key). It should only be necessary for a single handshake to take place between two users. The notes generated by this contract will be easy to identify enabling users to retrieve these notes, decrypt them and use the contents in any deterministic tag generation used by their chosen note discovery protocol.
-
+5. It protects applications from malicious unprovable functions.