Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: PXE sync on demand #10613

Merged
merged 33 commits into from
Dec 12, 2024
Merged
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
74420bb
wait for pxe to synch before checking balances
Thunkar Dec 9, 2024
92ba769
actually commit
Thunkar Dec 9, 2024
0337f73
comment
Thunkar Dec 9, 2024
a0483db
fix
Thunkar Dec 9, 2024
d989cad
Merge branch 'master' into gj/deflate_e2e_2_pxes
Thunkar Dec 9, 2024
823921e
Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…
Thunkar Dec 9, 2024
0fde995
merge
Thunkar Dec 9, 2024
0d58475
Merge branch 'gj/deflate_e2e_2_pxes' of github.com:AztecProtocol/azte…
Thunkar Dec 9, 2024
fe40d4f
fmt
Thunkar Dec 10, 2024
3721212
Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…
Thunkar Dec 10, 2024
9540975
Merge branch 'master' into gj/deflate_e2e_2_pxes
Thunkar Dec 10, 2024
782902a
Merge branch 'master' into gj/deflate_e2e_2_pxes
Thunkar Dec 10, 2024
4a180a0
Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…
Thunkar Dec 10, 2024
9499176
Merge branch 'master' into gj/deflate_e2e_2_pxes
Thunkar Dec 10, 2024
f91e77e
Merge branch 'master' into gj/deflate_e2e_2_pxes
Thunkar Dec 11, 2024
11a8b71
Merge branch 'gj/deflate_e2e_2_pxes' of github.com:AztecProtocol/azte…
Thunkar Dec 11, 2024
712f25e
synchronize on demand
Thunkar Dec 11, 2024
64b4996
Merge branch 'master' into gj/deflate_e2e_2_pxes
Thunkar Dec 11, 2024
307b76b
Merge branch 'gj/deflate_e2e_2_pxes' of github.com:AztecProtocol/azte…
Thunkar Dec 11, 2024
a120e99
improvements and comments
Thunkar Dec 11, 2024
5a2a6f7
don't sync on note sync
Thunkar Dec 11, 2024
13ff2a6
fmt
Thunkar Dec 11, 2024
3ddccf1
Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…
Thunkar Dec 11, 2024
2f9a2ee
fix
Thunkar Dec 11, 2024
278c6fe
ffs
Thunkar Dec 11, 2024
da9353f
Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…
Thunkar Dec 12, 2024
725759e
removed jobqueue, stop, etc
Thunkar Dec 12, 2024
f50c8e9
fmt
Thunkar Dec 12, 2024
00fd955
renaming
Thunkar Dec 12, 2024
bc56c9c
fix
Thunkar Dec 12, 2024
ff200de
typo
Thunkar Dec 12, 2024
77f8ddb
fmt
Thunkar Dec 12, 2024
a2385ba
Merge branch 'master' of github.com:AztecProtocol/aztec-packages into…
Thunkar Dec 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
synchronize on demand
Thunkar committed Dec 11, 2024
commit 712f25e25cc6ba838aa0d48aeb2530c3e16e3117
14 changes: 0 additions & 14 deletions yarn-project/aztec.js/src/contract/sent_tx.test.ts
Original file line number Diff line number Diff line change
@@ -23,22 +23,8 @@ describe('SentTx', () => {
pxe.getTxReceipt.mockResolvedValue(txReceipt);
});

it('waits for all notes of the accounts to be available', async () => {
pxe.getSyncStatus.mockResolvedValueOnce({ blocks: 25 }).mockResolvedValueOnce({ blocks: 25 });

const actual = await sentTx.wait({ timeout: 1, interval: 0.4 });
expect(actual).toEqual(txReceipt);
});

it('does not wait for notes sync', async () => {
pxe.getSyncStatus.mockResolvedValue({ blocks: 19 });
const actual = await sentTx.wait({ timeout: 1, interval: 0.4, waitForNotesAvailable: false });
expect(actual).toEqual(txReceipt);
});

it('throws if tx is dropped', async () => {
pxe.getTxReceipt.mockResolvedValue({ ...txReceipt, status: TxStatus.DROPPED } as TxReceipt);
pxe.getSyncStatus.mockResolvedValue({ blocks: 19 });
await expect(sentTx.wait({ timeout: 1, interval: 0.4, ignoreDroppedReceiptsFor: 0 })).rejects.toThrow(/dropped/);
});

11 changes: 1 addition & 10 deletions yarn-project/aztec.js/src/contract/sent_tx.ts
Original file line number Diff line number Diff line change
@@ -124,16 +124,7 @@ export class SentTx {
}
return undefined;
}
// If we don't care about waiting for notes to be synced, return the receipt
const waitForNotesAvailable = opts?.waitForNotesAvailable ?? DefaultWaitOpts.waitForNotesAvailable;
if (!waitForNotesAvailable) {
return txReceipt;
}
// Check if all sync blocks on the PXE Service are greater or equal than the block in which the tx was mined
const { blocks } = await this.pxe.getSyncStatus();
const targetBlock = txReceipt.blockNumber!;
const areNotesAvailable = blocks >= targetBlock;
return areNotesAvailable ? txReceipt : undefined;
return txReceipt;
},
'isMined',
opts?.timeout ?? DefaultWaitOpts.timeout,
1 change: 0 additions & 1 deletion yarn-project/aztec.js/src/index.ts
Original file line number Diff line number Diff line change
@@ -149,7 +149,6 @@ export {
type PXE,
type PartialAddress,
type PublicKey,
type SyncStatus,
} from '@aztec/circuit-types';

// TODO: These kinds of things have no place on our public api.
7 changes: 0 additions & 7 deletions yarn-project/aztec.js/src/wallet/base_wallet.ts
Original file line number Diff line number Diff line change
@@ -10,7 +10,6 @@ import {
type PXEInfo,
type PrivateExecutionResult,
type SiblingPath,
type SyncStatus,
type Tx,
type TxExecutionRequest,
type TxHash,
@@ -170,12 +169,6 @@ export abstract class BaseWallet implements Wallet {
getNodeInfo(): Promise<NodeInfo> {
return this.pxe.getNodeInfo();
}
isGlobalStateSynchronized() {
return this.pxe.isGlobalStateSynchronized();
}
getSyncStatus(): Promise<SyncStatus> {
return this.pxe.getSyncStatus();
}
addAuthWitness(authWitness: AuthWitness) {
return this.pxe.addAuthWitness(authWitness);
}
11 changes: 0 additions & 11 deletions yarn-project/bot/src/factory.ts
Original file line number Diff line number Diff line change
@@ -6,7 +6,6 @@ import {
type DeployOptions,
createLogger,
createPXEClient,
retryUntil,
} from '@aztec/aztec.js';
import { type AztecNode, type FunctionCall, type PXE } from '@aztec/circuit-types';
import { Fr, deriveSigningKey } from '@aztec/circuits.js';
@@ -67,16 +66,6 @@ export class BotFactory {
if (isInit) {
this.log.info(`Account at ${account.getAddress().toString()} already initialized`);
const wallet = await account.register();
const blockNumber = await this.pxe.getBlockNumber();
await retryUntil(
async () => {
const status = await this.pxe.getSyncStatus();
return blockNumber <= status.blocks;
},
'pxe synch',
3600,
1,
);
return wallet;
} else {
this.log.info(`Initializing account at ${account.getAddress().toString()}`);
1 change: 0 additions & 1 deletion yarn-project/circuit-types/src/interfaces/index.ts
Original file line number Diff line number Diff line change
@@ -18,7 +18,6 @@ export * from './proving-job.js';
export * from './pxe.js';
export * from './server_circuit_prover.js';
export * from './service.js';
export * from './sync-status.js';
export * from './world_state.js';
export * from './prover-broker.js';
export * from './p2p.js';
19 changes: 0 additions & 19 deletions yarn-project/circuit-types/src/interfaces/pxe.test.ts
Original file line number Diff line number Diff line change
@@ -43,7 +43,6 @@ import { Tx, TxHash, TxProvingResult, TxReceipt, TxSimulationResult } from '../t
import { TxEffect } from '../tx_effect.js';
import { TxExecutionRequest } from '../tx_execution_request.js';
import { type EventMetadataDefinition, type PXE, type PXEInfo, PXESchema } from './pxe.js';
import { type SyncStatus } from './sync-status.js';

jest.setTimeout(12_000);

@@ -258,16 +257,6 @@ describe('PXESchema', () => {
expect(result).toEqual(await handler.getPXEInfo());
});

it('isGlobalStateSynchronized', async () => {
const result = await context.client.isGlobalStateSynchronized();
expect(result).toBe(true);
});

it('getSyncStatus', async () => {
const result = await context.client.getSyncStatus();
expect(result).toEqual(await handler.getSyncStatus());
});

it('getContractInstance', async () => {
const result = await context.client.getContractInstance(address);
expect(result).toEqual(instance);
@@ -502,14 +491,6 @@ class MockPXE implements PXE {
pxeVersion: '1.0',
});
}
isGlobalStateSynchronized(): Promise<boolean> {
return Promise.resolve(true);
}
getSyncStatus(): Promise<SyncStatus> {
return Promise.resolve({
blocks: 1,
});
}
getContractInstance(address: AztecAddress): Promise<ContractInstanceWithAddress | undefined> {
expect(address).toEqual(this.address);
return Promise.resolve(this.instance);
19 changes: 0 additions & 19 deletions yarn-project/circuit-types/src/interfaces/pxe.ts
Original file line number Diff line number Diff line change
@@ -43,7 +43,6 @@ import { SiblingPath } from '../sibling_path/sibling_path.js';
import { Tx, TxHash, TxProvingResult, TxReceipt, TxSimulationResult } from '../tx/index.js';
import { TxEffect } from '../tx_effect.js';
import { TxExecutionRequest } from '../tx_execution_request.js';
import { type SyncStatus, SyncStatusSchema } from './sync-status.js';

// docs:start:pxe-interface
/**
@@ -351,22 +350,6 @@ export interface PXE {
*/
getPXEInfo(): Promise<PXEInfo>;

/**
* Checks whether all the blocks were processed (tree roots updated, txs updated with block info, etc.).
* @returns True if there are no outstanding blocks to be synched.
* @remarks This indicates that blocks and transactions are synched even if notes are not. Compares local block number with the block number from aztec node.
* @deprecated Use `getSyncStatus` instead.
*/
isGlobalStateSynchronized(): Promise<boolean>;

/**
* Returns the latest block that has been synchronized globally and for each account. The global block number
* indicates whether global state has been updated up to that block, whereas each address indicates up to which
* block the private state has been synced for that account.
* @returns The latest block synchronized for blocks, and the latest block synched for notes for each public key being tracked.
*/
getSyncStatus(): Promise<SyncStatus>;

/**
* Returns a Contract Instance given its address, which includes the contract class identifier,
* initialization hash, deployment salt, and public keys hash.
@@ -540,8 +523,6 @@ export const PXESchema: ApiSchemaFor<PXE> = {
getProvenBlockNumber: z.function().returns(z.number()),
getNodeInfo: z.function().returns(NodeInfoSchema),
getPXEInfo: z.function().returns(PXEInfoSchema),
isGlobalStateSynchronized: z.function().returns(z.boolean()),
getSyncStatus: z.function().returns(SyncStatusSchema),
getContractInstance: z
.function()
.args(schemas.AztecAddress)
13 changes: 0 additions & 13 deletions yarn-project/circuit-types/src/interfaces/sync-status.ts

This file was deleted.

23 changes: 1 addition & 22 deletions yarn-project/cli/src/cmds/pxe/get_block.ts
Original file line number Diff line number Diff line change
@@ -3,29 +3,8 @@ import { type LogFn, type Logger } from '@aztec/foundation/log';

import { inspectBlock } from '../../utils/inspect.js';

export async function getBlock(
rpcUrl: string,
maybeBlockNumber: number | undefined,
follow: boolean,
debugLogger: Logger,
log: LogFn,
) {
export async function getBlock(rpcUrl: string, maybeBlockNumber: number | undefined, debugLogger: Logger, log: LogFn) {
const client = await createCompatibleClient(rpcUrl, debugLogger);
const blockNumber = maybeBlockNumber ?? (await client.getBlockNumber());
await inspectBlock(client, blockNumber, log, { showTxs: true });

if (follow) {
let lastBlock = blockNumber;
setInterval(async () => {
const newBlock = await client.getBlockNumber();
if (newBlock > lastBlock) {
const { blocks } = await client.getSyncStatus();
if (blocks >= newBlock) {
log('');
await inspectBlock(client, newBlock, log, { showTxs: true });
lastBlock = newBlock;
}
}
}, 1000);
}
}
3 changes: 1 addition & 2 deletions yarn-project/cli/src/cmds/pxe/index.ts
Original file line number Diff line number Diff line change
@@ -54,11 +54,10 @@ export function injectCommands(program: Command, log: LogFn, debugLogger: Logger
.command('get-block')
.description('Gets info for a given block or latest.')
.argument('[blockNumber]', 'Block height', parseOptionalInteger)
.option('-f, --follow', 'Keep polling for new blocks')
.addOption(pxeOption)
.action(async (blockNumber, options) => {
const { getBlock } = await import('./get_block.js');
await getBlock(options.rpcUrl, blockNumber, options.follow, debugLogger, log);
await getBlock(options.rpcUrl, blockNumber, debugLogger, log);
});

program
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@ import { type BenchmarkingContract } from '@aztec/noir-contracts.js/Benchmarking
import { type SequencerClient } from '@aztec/sequencer-client';

import { type EndToEndContext } from '../fixtures/utils.js';
import { benchmarkSetup, getFolderSize, makeDataDirectory, sendTxs, waitNewPXESynced } from './utils.js';
import { benchmarkSetup, createNewPXE, getFolderSize, makeDataDirectory, sendTxs } from './utils.js';

const BLOCK_SIZE = BENCHMARK_HISTORY_BLOCK_SIZE;
const CHAIN_LENGTHS = BENCHMARK_HISTORY_CHAIN_LENGTHS;
@@ -69,7 +69,7 @@ describe('benchmarks/process_history', () => {
// Create a new pxe and measure how much time it takes it to sync with failed and successful decryption
// Skip the first two blocks used for setup (create account contract and deploy benchmarking contract)
context.logger.info(`Starting new pxe`);
const pxe = await waitNewPXESynced(node, contract, INITIAL_L2_BLOCK_NUM + setupBlockCount);
const pxe = await createNewPXE(node, contract, INITIAL_L2_BLOCK_NUM + setupBlockCount);

// Register the owner account and wait until it's synced so we measure how much time it took
context.logger.info(`Registering owner account on new pxe`);
nventuro marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { type BenchmarkingContract } from '@aztec/noir-contracts.js/Benchmarking
import { type SequencerClient } from '@aztec/sequencer-client';

import { type EndToEndContext } from '../fixtures/utils.js';
import { benchmarkSetup, sendTxs, waitNewPXESynced } from './utils.js';
import { benchmarkSetup, createNewPXE, sendTxs } from './utils.js';

describe('benchmarks/publish_rollup', () => {
let context: EndToEndContext;
@@ -41,7 +41,7 @@ describe('benchmarks/publish_rollup', () => {

// Spin up a new pxe and sync it, we'll use it to test sync times of new accounts for the last block
context.logger.info(`Starting new pxe`);
const pxe = await waitNewPXESynced(node, contract, blockNumber! - 1);
const pxe = await createNewPXE(node, contract, blockNumber! - 1);

// Register the owner account and wait until it's synced so we measure how much time it took
Comment on lines +44 to 46
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also these

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above

context.logger.info(`Registering owner account on new pxe`);
7 changes: 3 additions & 4 deletions yarn-project/end-to-end/src/benchmarks/utils.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { type AztecNodeConfig, type AztecNodeService } from '@aztec/aztec-node';
import { type AztecNode, BatchCall, INITIAL_L2_BLOCK_NUM, type SentTx, retryUntil, sleep } from '@aztec/aztec.js';
import { type AztecNode, BatchCall, INITIAL_L2_BLOCK_NUM, type SentTx, sleep } from '@aztec/aztec.js';
import { times } from '@aztec/foundation/collection';
import { randomInt } from '@aztec/foundation/crypto';
import { BenchmarkingContract } from '@aztec/noir-contracts.js/Benchmarking';
@@ -90,13 +90,13 @@ export async function sendTxs(
}

/**
* Creates a new PXE and awaits until it's synced with the node.
* Creates a new PXE
* @param node - Node to connect the pxe to.
* @param contract - Benchmark contract to add to the pxe.
* @param startingBlock - First l2 block to process.
* @returns The new PXE.
*/
export async function waitNewPXESynced(
export async function createNewPXE(
node: AztecNode,
contract: BenchmarkingContract,
startingBlock: number = INITIAL_L2_BLOCK_NUM,
@@ -111,6 +111,5 @@ export async function waitNewPXESynced(
} as PXEServiceConfig;
const pxe = await createPXEService(node, pxeConfig);
await pxe.registerContract(contract);
await retryUntil(() => pxe.isGlobalStateSynchronized(), 'pxe-global-sync');
return pxe;
}
12 changes: 0 additions & 12 deletions yarn-project/end-to-end/src/e2e_2_pxes.test.ts
Original file line number Diff line number Diff line change
@@ -8,7 +8,6 @@ import {
type Logger,
type PXE,
type Wallet,
retryUntil,
sleep,
} from '@aztec/aztec.js';
import { ChildContract, TestContract, TokenContract } from '@aztec/noir-contracts.js';
@@ -103,21 +102,12 @@ describe('e2e_2_pxes', () => {
return contract.instance;
};

const awaitServerSynchronized = async (server: PXE) => {
const isServerSynchronized = async () => {
return await server.isGlobalStateSynchronized();
};
await retryUntil(isServerSynchronized, 'server sync', 10);
};

const getChildStoredValue = (child: { address: AztecAddress }, pxe: PXE) =>
pxe.getPublicStorageAt(child.address, new Fr(1));

it('user calls a public function on a contract deployed by a different user using a different PXE', async () => {
const childCompleteAddress = await deployChildContractViaServerA();

await awaitServerSynchronized(pxeA);

// Add Child to PXE B
await pxeB.registerContract({
artifact: ChildContract.artifact,
@@ -129,8 +119,6 @@ describe('e2e_2_pxes', () => {
const childContractWithWalletB = await ChildContract.at(childCompleteAddress.address, walletB);
await childContractWithWalletB.methods.pub_inc_value(newValueToSet).send().wait({ interval: 0.1 });

await awaitServerSynchronized(pxeA);

const storedValueOnB = await getChildStoredValue(childCompleteAddress, pxeB);
expect(storedValueOnB).toEqual(newValueToSet);

8 changes: 0 additions & 8 deletions yarn-project/end-to-end/src/e2e_block_building.test.ts
Original file line number Diff line number Diff line change
@@ -483,14 +483,6 @@ describe('e2e_block_building', () => {
// PXE should have cleared out the 30-note from tx2, but reapplied the 20-note from tx1
expect(await contract.methods.summed_values(ownerAddress).simulate()).toEqual(21n);

// PXE should be synced to the block number on the new chain
await retryUntil(
async () => (await pxe.getSyncStatus()).blocks === newTx1Receipt.blockNumber,
'wait for pxe block header sync',
15,
1,
);

// And we should be able to send a new tx on the new chain
logger.info('Sending new tx on reorgd chain');
const tx3 = await contract.methods.create_note(ownerAddress, ownerAddress, 10).send().wait();
2 changes: 1 addition & 1 deletion yarn-project/pxe/src/bin/index.ts
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ import { createLogger } from '@aztec/foundation/log';

import { getPXEServiceConfig } from '../config/index.js';
import { startPXEHttpServer } from '../pxe_http/index.js';
import { createPXEService } from '../utils/index.js';
import { createPXEService } from '../pxe_service/create_pxe_service.js';

const { PXE_PORT = 8080, AZTEC_NODE_URL = 'http://localhost:8079' } = process.env;

Loading

Unchanged files with check annotations Beta

test:
FROM +build
RUN yarn test

Check failure on line 301 in yarn-project/Earthfile

GitHub Actions / yarn-project-test

Error

The command RUN yarn test did not complete successfully. Exit code 1
run-e2e:
ARG test