feat: Create standalone instances of Aztec Node and RPC Server #2486

PhilWindle · 2023-09-23T19:34:54Z

This PR enables both Aztec Node and RPC Servers to be created as standalone services with JSON PRC endpoints.

Checklist:

Remove the checklist to signal you've completed it. Enable auto-merge if the PR is ready to merge.

If the pull request requires a cryptography review (e.g. cryptographic algorithm implementations) I have added the 'crypto' tag.
I have reviewed my diff in github, line by line and removed unexpected formatting changes, testing logs, or commented-out code.
Every change is related to the PR description.
I have linked this pull request to relevant issues (if any exist).

socket-security · 2023-09-24T12:04:06Z

New dependencies detected. Learn more about Socket for GitHub ↗︎

Packages	Version	New capabilities	Transitives	Size	Publisher
querystring-es3	0.2.1	None	`+0`	16.1 kB	spaintrain
browserify-zlib	0.2.0	None	`+0`	192 kB	dignifiedquire
stream-http	3.2.0	None	`+1`	31.3 kB	jhiesey

spalladino

Now that was quite a PR. I think we should be able to simplify much of it (and remove many of the new dependencies) by going back to having NodeInfo and L1Addresses as just interfaces rather than classes. And I'd also consider moving terraform templates specific to our own infra out of the public monorepo.

spalladino · 2023-09-25T17:54:13Z

yarn-project/aztec-node/Dockerfile

We have been using entrypoint pretty liberally, but I'd push for having just yarn as entrypoint, and start as its command. Not really important though unless we try to make this consistent across the board.

spalladino · 2023-09-25T17:57:01Z

yarn-project/aztec-node/terraform/main.tf

+      },
+      {
+        "name": "BOOTSTRAP_NODES",
+        "value": "/ip4/44.201.46.76/tcp/40400/p2p/12D3KooWGBpbC6qQFkaCYphjNeY6sV99o4SnEWyTeBigoVriDn4D"


Where's this IP coming from? Should we make it a param, or add a comment?

Committed by accident, will revert

spalladino · 2023-09-25T17:58:07Z

yarn-project/aztec-node/terraform/main.tf

This template feels pretty coupled with our (as in Aztec Labs') own infrastructure, and not really usable by anyone outside the org. Maybe we should move it to a separate private repo, and leave here only templates reusable by the community?

spalladino · 2023-09-25T17:58:29Z

yarn-project/aztec-node/terraform/main.tf

+        "name": "SEARCH_START_BLOCK",
+        "value": "15918000"


Why this block in particular?

spalladino · 2023-09-25T18:01:33Z

yarn-project/aztec-sandbox/src/bin/index.ts

+ * Creates the sandbox from provided config and deploys any initial L1 and L2 contracts
+ * @param isP2PSandbox - Flag indicating if this sandbox is to connect to a P2P network
+ */
+async function createAndDeploySandbox(isP2PSandbox: boolean) {


Is the isP2PSandbox flavor used anywhere?

spalladino · 2023-09-25T18:17:26Z

yarn-project/aztec.js/package.json

+    "assert": "^2.1.0",
+    "browserify-zlib": "^0.2.0",
    "buffer": "^6.0.3",
    "crypto-browserify": "^3.12.0",
    "jest": "^29.5.0",
    "jest-mock-extended": "^3.0.3",
    "process": "^0.11.10",
+    "querystring-es3": "^0.2.1",
    "resolve-typescript-plugin": "^2.0.1",
    "stream-browserify": "^3.0.0",
+    "stream-http": "^3.2.0",


From what you mentioned earlier, I understand these are the new deps needed to please the webpack god. It seems these are required because we now have a dependency on aztec/ethereum, which is only needed to grab the L1ContractAddresses type.

Maybe we can just move the L1ContractAddresses to aztec/types, so we don't introduce the new dependency (since we want js to be as slim as possible), and don't have these new issues with webpack?

spalladino · 2023-09-25T18:18:52Z

yarn-project/aztec.js/src/aztec_rpc_client.ts

@@ -32,7 +33,7 @@ export const createAztecRpcClient = (url: string, fetch = makeFetch([1, 2, 3], t
      NotePreimage,
      AuthWitness,
    },
-    { Tx, TxReceipt, L2BlockL2Logs },
+    { Tx, TxReceipt, L2BlockL2Logs, NodeInfo },


Actually, do we need to add NodeInfo here? If we keep it as an interface instead of a type, and we initialise it as a plain object, doesn't the automagic (de)serialiser go into each field and serialise that, without having to register the class here?

If we can remove it and just keep NodeInfo as a type or interface (and not a class) we can roll back the new dependency on aztec/ethereum completely.

spalladino · 2023-09-25T18:24:02Z

yarn-project/aztec-sandbox/src/sandbox.ts

+/**
+ * Helper function that retrieves the addresses of configured deployed contracts.
+ */
+function retrieveL1Contracts(config: AztecNodeConfig, account: Account): Promise<DeployL1Contracts> {


Took me a while to understand that this function does not retrieve the addresses, it just creates the wallet and public clients and bundles everything together. I'd rename the function, or rather remove it in favor of a helper that just assembles the wallet and public clients (and we reuse that for deployL1Contracts as well), and use it along with the config to create the DeployL1Contracts object directly.

spalladino · 2023-09-25T18:27:33Z

yarn-project/aztec-sandbox/src/sandbox.ts

+ * Does not start any HTTP services nor populate any initial accounts.
+ * @param config - Optional Sandbox settings.
+ */
+export async function createP2PSandbox() {


Is the only difference with createSandbox that this one does not deploy the L1 contracts? If so, I'd go back to a single createSandbox function, and add an option into the SandboxConfig like skipDeployL1 that controls that behavior. Otherwise, it feels like we have a lot of duplication between createP2PSandbox and createSandbox.

spalladino · 2023-09-25T18:33:35Z

yarn-project/types/package.json

@@ -31,6 +31,7 @@
  },
  "dependencies": {
    "@aztec/circuits.js": "workspace:^",
+    "@aztec/ethereum": "workspace:^",


aztec/ethereum brings a bunch of annoying L1 dependencies (such as our l1-artifacts) that we shouldn't need to pull in everywhere, given types is imported everywhere. It'd be great if we can remove this new dependency.

I think we should be able to roll back L1ContractAddresses to be just a type or interface (and define it within types), and let the converter work its magic by working on a plain object:

aztec-packages/yarn-project/foundation/src/json-rpc/convert.test.ts

Lines 62 to 66 in 471159b

test('converts a plain object', () => {

const obj = { a: 10, b: [20, 30], c: 'foo' };

const cc = new ClassConverter();

expect(convertFromJsonObj(cc, convertToJsonObj(cc, obj))).toEqual(obj);

});

This PR addresses the comments of PR [2486](#2486). The feature set here is: 1. Some refactoring of packages dependencies and l1 contract address configuration. 2. Entrypoints to start standalone instances of Aztec Node and Aztec RPC Server. 3. Some initial terraform for Aztec Node deployments. # Checklist: Remove the checklist to signal you've completed it. Enable auto-merge if the PR is ready to merge. - [ ] If the pull request requires a cryptography review (e.g. cryptographic algorithm implementations) I have added the 'crypto' tag. - [ ] I have reviewed my diff in github, line by line and removed unexpected formatting changes, testing logs, or commented-out code. - [ ] Every change is related to the PR description. - [ ] I have [linked](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue) this pull request to relevant issues (if any exist).

This PR addresses the comments of PR [2486](AztecProtocol/aztec-packages#2486). The feature set here is: 1. Some refactoring of packages dependencies and l1 contract address configuration. 2. Entrypoints to start standalone instances of Aztec Node and Aztec RPC Server. 3. Some initial terraform for Aztec Node deployments. # Checklist: Remove the checklist to signal you've completed it. Enable auto-merge if the PR is ready to merge. - [ ] If the pull request requires a cryptography review (e.g. cryptographic algorithm implementations) I have added the 'crypto' tag. - [ ] I have reviewed my diff in github, line by line and removed unexpected formatting changes, testing logs, or commented-out code. - [ ] Every change is related to the PR description. - [ ] I have [linked](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue) this pull request to relevant issues (if any exist).

PhilWindle added 14 commits September 8, 2023 09:34

WIP

9b02d53

WIP

5798a4a

Merge branch 'master' into pw/devnet

2107fb6

WIP

bb04391

Merge branch 'master' into pw/devnet

61594a1

Fix

ed810e7

Merge fixes

5e6e8b2

Fixes

3b9e08e

Fix

4518654

fix

1343d91

Update version

f2821aa

yarn lock

0223f16

Fix

a11de0f

WIP

395f24e

PhilWindle added 11 commits September 24, 2023 12:36

Fix

6dfee39

Fix

7fbc059

Serialise node info

51ab998

Fixes

236634e

Fixes

ec7ef72

WIP

f65c0fa

Merge branch 'master' into pw/devnet

f98b077

WIP

657b780

Merge branch 'master' into pw/devnet

ee9f7d2

Remove old terraform

c9f7c27

WIP

5c7336f

PhilWindle changed the title ~~feat: Network~~ feat: Create standalone instances of Aztec Node and RPC Server Sep 25, 2023

PhilWindle marked this pull request as ready for review September 25, 2023 15:57

Cleanup

471159b

spalladino reviewed Sep 25, 2023

View reviewed changes

PhilWindle mentioned this pull request Sep 26, 2023

feat: Standalone Aztec Node and RPC Server #2522

Merged

4 tasks

PhilWindle closed this Sep 26, 2023

ludamad deleted the pw/devnet branch August 22, 2024 14:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Create standalone instances of Aztec Node and RPC Server #2486

feat: Create standalone instances of Aztec Node and RPC Server #2486

PhilWindle commented Sep 23, 2023 •

edited

Loading

socket-security bot commented Sep 24, 2023

spalladino left a comment

spalladino Sep 25, 2023

spalladino Sep 25, 2023

PhilWindle Sep 25, 2023

spalladino Sep 25, 2023

spalladino Sep 25, 2023

spalladino Sep 25, 2023

spalladino Sep 25, 2023

spalladino Sep 25, 2023

spalladino Sep 25, 2023

spalladino Sep 25, 2023

spalladino Sep 25, 2023

	test('converts a plain object', () => {
	const obj = { a: 10, b: [20, 30], c: 'foo' };
	const cc = new ClassConverter();
	expect(convertFromJsonObj(cc, convertToJsonObj(cc, obj))).toEqual(obj);
	});

feat: Create standalone instances of Aztec Node and RPC Server #2486

feat: Create standalone instances of Aztec Node and RPC Server #2486

Conversation

PhilWindle commented Sep 23, 2023 • edited Loading

Checklist:

socket-security bot commented Sep 24, 2023

spalladino left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PhilWindle commented Sep 23, 2023 •

edited

Loading