Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: nullified note retrieval in get_notes and view_notes #4238

Merged
merged 28 commits into from
Jan 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
d90acf4
Add status to NoteFilter, basic KV db support
nventuro Jan 19, 2024
185f525
Add filtering test cases
nventuro Jan 19, 2024
12e71d2
Add filter option, create tests
nventuro Jan 24, 2024
d73827d
Make includeNullified mandatory
nventuro Jan 24, 2024
1a96ef6
Slight param rename
nventuro Jan 25, 2024
f805574
Provide methods for the options
nventuro Jan 25, 2024
70e81b4
Fix typo
nventuro Jan 25, 2024
fb05a44
Switch to camel case
nventuro Jan 25, 2024
62c0256
Link to new issue
nventuro Jan 25, 2024
8b63fe3
Formatting pass
nventuro Jan 25, 2024
c241d38
Remove extraneous comment
nventuro Jan 25, 2024
853ac0b
Remove todo comment
nventuro Jan 25, 2024
6cc5b0e
More formatting edits
nventuro Jan 25, 2024
c05ab06
Switch to enums
nventuro Jan 25, 2024
e2e8d33
Add test for mixed retrievals
nventuro Jan 26, 2024
9b0e1a3
Merge branch 'master' into get-nullified-notes
nventuro Jan 26, 2024
f3378ca
Increase timeouts
nventuro Jan 26, 2024
67f6889
Remove superflous todo
nventuro Jan 26, 2024
98a71cc
Merge branch 'master' into get-nullified-notes
nventuro Jan 26, 2024
174e954
Undo cpp formatting changes
nventuro Jan 26, 2024
1f89524
Insert tests into note_getter tests
nventuro Jan 26, 2024
9efd709
Fix noir formatting
nventuro Jan 26, 2024
97574ff
More formatting fixes
nventuro Jan 26, 2024
1680997
Merge branch 'master' into get-nullified-notes
nventuro Jan 26, 2024
c084d7a
Merge branch 'master' into get-nullified-notes
nventuro Jan 29, 2024
4fe0342
Fix missing merge issue
nventuro Jan 29, 2024
0edd036
Fix todo link
nventuro Jan 30, 2024
483e274
Merge branch 'master' into get-nullified-notes
nventuro Jan 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions yarn-project/acir-simulator/src/acvm/oracle/oracle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,7 @@ export class Oracle {
sortOrder: ACVMField[],
[limit]: ACVMField[],
[offset]: ACVMField[],
[status]: ACVMField[],
[returnSize]: ACVMField[],
): Promise<ACVMField[]> {
const noteDatas = await this.typedOracle.getNotes(
Expand All @@ -174,6 +175,7 @@ export class Oracle {
sortOrder.map(s => +s),
+limit,
+offset,
+status,
);

const noteLength = noteDatas?.[0]?.note.items.length ?? 0;
Expand Down
2 changes: 2 additions & 0 deletions yarn-project/acir-simulator/src/acvm/oracle/typed_oracle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import {
CompleteAddress,
MerkleTreeId,
Note,
NoteStatus,
NullifierMembershipWitness,
PublicDataWitness,
PublicKey,
Expand Down Expand Up @@ -137,6 +138,7 @@ export abstract class TypedOracle {
_sortOrder: number[],
_limit: number,
_offset: number,
_status: NoteStatus,
): Promise<NoteData[]> {
throw new Error('Not available.');
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { AuthWitness, FunctionL2Logs, L1NotePayload, Note, UnencryptedL2Log } from '@aztec/circuit-types';
import { AuthWitness, FunctionL2Logs, L1NotePayload, Note, NoteStatus, UnencryptedL2Log } from '@aztec/circuit-types';
import {
CallContext,
ContractDeploymentData,
Expand Down Expand Up @@ -190,6 +190,7 @@ export class ClientExecutionContext extends ViewDataOracle {
* @param sortOrder - The order of the corresponding index in sortBy. (1: DESC, 2: ASC, 0: Do nothing)
* @param limit - The number of notes to retrieve per query.
* @param offset - The starting index for pagination.
* @param status - The status of notes to fetch.
* @returns Array of note data.
*/
public async getNotes(
Expand All @@ -202,12 +203,13 @@ export class ClientExecutionContext extends ViewDataOracle {
sortOrder: number[],
limit: number,
offset: number,
status: NoteStatus,
): Promise<NoteData[]> {
// Nullified pending notes are already removed from the list.
const pendingNotes = this.noteCache.getNotes(this.contractAddress, storageSlot);

const pendingNullifiers = this.noteCache.getNullifiers(this.contractAddress);
const dbNotes = await this.db.getNotes(this.contractAddress, storageSlot);
const dbNotes = await this.db.getNotes(this.contractAddress, storageSlot, status);
const dbNotesFiltered = dbNotes.filter(n => !pendingNullifiers.has((n.siloedNullifier as Fr).value));

const notes = pickNotes<NoteData>([...dbNotesFiltered, ...pendingNotes], {
Expand Down
5 changes: 3 additions & 2 deletions yarn-project/acir-simulator/src/client/db_oracle.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { L2Block, MerkleTreeId, NullifierMembershipWitness, PublicDataWitness } from '@aztec/circuit-types';
import { L2Block, MerkleTreeId, NoteStatus, NullifierMembershipWitness, PublicDataWitness } from '@aztec/circuit-types';
import { CompleteAddress, Header } from '@aztec/circuits.js';
import { FunctionArtifactWithDebugMetadata, FunctionSelector } from '@aztec/foundation/abi';
import { AztecAddress } from '@aztec/foundation/aztec-address';
Expand Down Expand Up @@ -61,9 +61,10 @@ export interface DBOracle extends CommitmentsDB {
*
* @param contractAddress - The AztecAddress instance representing the contract address.
* @param storageSlot - The Fr instance representing the storage slot of the notes.
* @param status - The status of notes to fetch.
* @returns A Promise that resolves to an array of note data.
*/
getNotes(contractAddress: AztecAddress, storageSlot: Fr): Promise<NoteData[]>;
getNotes(contractAddress: AztecAddress, storageSlot: Fr, status: NoteStatus): Promise<NoteData[]>;

/**
* Retrieve the artifact information of a specific function within a contract.
Expand Down
5 changes: 4 additions & 1 deletion yarn-project/acir-simulator/src/client/view_data_oracle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import {
AztecNode,
CompleteAddress,
MerkleTreeId,
NoteStatus,
NullifierMembershipWitness,
PublicDataWitness,
} from '@aztec/circuit-types';
Expand Down Expand Up @@ -169,6 +170,7 @@ export class ViewDataOracle extends TypedOracle {
* @param sortOrder - The order of the corresponding index in sortBy. (1: DESC, 2: ASC, 0: Do nothing)
* @param limit - The number of notes to retrieve per query.
* @param offset - The starting index for pagination.
* @param status - The status of notes to fetch.
* @returns Array of note data.
*/
public async getNotes(
Expand All @@ -181,8 +183,9 @@ export class ViewDataOracle extends TypedOracle {
sortOrder: number[],
limit: number,
offset: number,
status: NoteStatus,
): Promise<NoteData[]> {
const dbNotes = await this.db.getNotes(this.contractAddress, storageSlot);
const dbNotes = await this.db.getNotes(this.contractAddress, storageSlot, status);
return pickNotes<NoteData>(dbNotes, {
selects: selectBy
.slice(0, numSelects)
Expand Down
5 changes: 4 additions & 1 deletion yarn-project/aztec-nr/aztec/src/note/note_getter.nr
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ use dep::protocol_types::constants::{
};
use crate::context::PrivateContext;
use crate::note::{
note_getter_options::{NoteGetterOptions, Select, Sort, SortOrder, Comparator},
note_getter_options::{NoteGetterOptions, Select, Sort, SortOrder, Comparator, NoteStatus},
note_interface::NoteInterface,
note_viewer_options::NoteViewerOptions,
utils::compute_note_hash_for_read_or_nullify,
Expand Down Expand Up @@ -135,6 +135,7 @@ unconstrained fn get_note_internal<Note, N>(storage_slot: Field, note_interface:
[],
1, // limit
0, // offset
NoteStatus.ACTIVE,
placeholder_note,
placeholder_fields
)[0].unwrap() // Notice: we don't allow dummies to be returned from get_note (singular).
Expand All @@ -159,6 +160,7 @@ unconstrained fn get_notes_internal<Note, N, FILTER_ARGS>(
sort_order,
options.limit,
options.offset,
options.status,
placeholder_opt_notes,
placeholder_fields
);
Expand Down Expand Up @@ -187,6 +189,7 @@ unconstrained pub fn view_notes<Note, N>(
sort_order,
options.limit,
options.offset,
options.status,
placeholder_opt_notes,
placeholder_fields
)
Expand Down
24 changes: 22 additions & 2 deletions yarn-project/aztec-nr/aztec/src/note/note_getter_options.nr
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,17 @@ impl Sort {
}
}

struct NoteStatusEnum {
ACTIVE: u2,
ACTIVE_OR_NULLIFIED: u2,
}

global NoteStatus = NoteStatusEnum {
ACTIVE: 1,
ACTIVE_OR_NULLIFIED: 2,
// TODO 4217: add 'NULLIFIED'
};

fn return_all_notes<Note, N>(
notes: [Option<Note>; MAX_READ_REQUESTS_PER_CALL],
_p: Field
Expand All @@ -68,6 +79,7 @@ struct NoteGetterOptions<Note, N, FILTER_ARGS> {
offset: u32,
filter: fn ([Option<Note>; MAX_READ_REQUESTS_PER_CALL], FILTER_ARGS) -> [Option<Note>; MAX_READ_REQUESTS_PER_CALL],
filter_args: FILTER_ARGS,
status: u2,
}
// docs:end:NoteGetterOptions

Expand All @@ -85,6 +97,7 @@ impl<Note, N, FILTER_ARGS> NoteGetterOptions<Note, N, FILTER_ARGS> {
offset: 0,
filter: return_all_notes,
filter_args: 0,
status: NoteStatus.ACTIVE,
}
}

Expand All @@ -101,6 +114,7 @@ impl<Note, N, FILTER_ARGS> NoteGetterOptions<Note, N, FILTER_ARGS> {
offset: 0,
filter,
filter_args,
status: NoteStatus.ACTIVE,
}
}

Expand All @@ -120,16 +134,22 @@ impl<Note, N, FILTER_ARGS> NoteGetterOptions<Note, N, FILTER_ARGS> {
*self
}

// This method lets you set a limit for the maximum number of notes to be retrieved in a single query result.
// This method lets you set a limit for the maximum number of notes to be retrieved in a single query result.
pub fn set_limit(&mut self, limit: u32) -> Self {
assert(limit <= MAX_READ_REQUESTS_PER_CALL as u32);
self.limit = limit;
*self
}

// This method sets the offset value, which determines where to start retrieving notes in the query results.
// This method sets the offset value, which determines where to start retrieving notes in the query results.
pub fn set_offset(&mut self, offset: u32) -> Self {
self.offset = offset;
*self
}

// This method sets the status value, which determines whether to retrieve active or nullified notes.
pub fn set_status(&mut self, status: u2) -> Self {
self.status = status;
*self
}
}
16 changes: 12 additions & 4 deletions yarn-project/aztec-nr/aztec/src/note/note_viewer_options.nr
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use dep::std::option::Option;
use dep::protocol_types::constants::MAX_NOTES_PER_PAGE;
use crate::note::note_getter_options::{Select, Sort, Comparator};
use crate::note::note_getter_options::{Select, Sort, Comparator, NoteStatus};
use crate::types::vec::BoundedVec;

// docs:start:NoteViewerOptions
Expand All @@ -9,6 +9,7 @@ struct NoteViewerOptions<Note, N> {
sorts: BoundedVec<Option<Sort>, N>,
limit: u32,
offset: u32,
status: u2,
}
// docs:end:NoteViewerOptions

Expand All @@ -19,12 +20,13 @@ impl<Note, N> NoteViewerOptions<Note, N> {
sorts: BoundedVec::new(Option::none()),
limit: MAX_NOTES_PER_PAGE as u32,
offset: 0,
status: NoteStatus.ACTIVE,
}
}

// This method adds a `Select` criterion to the options.
// It takes a field_index indicating which field to select,
// a value representing the specific value to match in that field, and
// It takes a field_index indicating which field to select,
// a value representing the specific value to match in that field, and
// a comparator (For possible values of comparators, please see the Comparator enum from note_getter_options)
pub fn select(&mut self, field_index: u8, value: Field, comparator: Option<u3>) -> Self {
self.selects.push(Option::some(Select::new(field_index, value, comparator.unwrap_or(Comparator.EQ))));
Expand All @@ -46,4 +48,10 @@ impl<Note, N> NoteViewerOptions<Note, N> {
self.offset = offset;
*self
}

// This method sets the status value, which determines whether to retrieve active or nullified notes.
pub fn set_status(&mut self, status: u2) -> Self {
self.status = status;
*self
}
}
5 changes: 5 additions & 0 deletions yarn-project/aztec-nr/aztec/src/oracle/notes.nr
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ fn get_notes_oracle<N, S>(
_sort_order: [u2; N],
_limit: u32,
_offset: u32,
_status: u2,
_return_size: u32,
_placeholder_fields: [Field; S]
) -> [Field; S] {}
Expand All @@ -46,6 +47,7 @@ unconstrained fn get_notes_oracle_wrapper<N, S>(
sort_order: [u2; N],
limit: u32,
offset: u32,
status: u2,
mut placeholder_fields: [Field; S]
) -> [Field; S] {
let return_size = placeholder_fields.len() as u32;
Expand All @@ -59,6 +61,7 @@ unconstrained fn get_notes_oracle_wrapper<N, S>(
sort_order,
limit,
offset,
status,
return_size,
placeholder_fields
)
Expand All @@ -75,6 +78,7 @@ unconstrained pub fn get_notes<Note, N, M, S, NS>(
sort_order: [u2; M],
limit: u32,
offset: u32,
status: u2,
mut placeholder_opt_notes: [Option<Note>; S], // TODO: Remove it and use `limit` to initialize the note array.
placeholder_fields: [Field; NS] // TODO: Remove it and use `limit` to initialize the note array.
) -> [Option<Note>; S] {
Expand All @@ -88,6 +92,7 @@ unconstrained pub fn get_notes<Note, N, M, S, NS>(
sort_order,
limit,
offset,
status,
placeholder_fields
);
let num_notes = fields[0] as u32;
Expand Down
11 changes: 11 additions & 0 deletions yarn-project/circuit-types/src/notes/note_filter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@ import { AztecAddress, Fr } from '@aztec/circuits.js';

import { TxHash } from '../index.js';

/**
* The status of notes to retrieve.
*/
export enum NoteStatus {
ACTIVE = 1,
ACTIVE_OR_NULLIFIED = 2,
// TODO 4217: add 'NULLIFIED'
}

/**
* A filter used to fetch Notes.
* @remarks This filter is applied as an intersection of all it's params.
Expand All @@ -15,6 +24,8 @@ export type NoteFilter = {
storageSlot?: Fr;
/** The owner of the note (whose public key was used to encrypt the note). */
owner?: AztecAddress;
/** The status of the note. Defaults to 'ACTIVE'. */
status?: NoteStatus;
};

/**
Expand Down
Loading
Loading