Skip to content

Commit

Permalink
feat: photon rpc + new cli (#594)
Browse files Browse the repository at this point in the history
* feat: non-inclusion gnark circuits (#559)

feat: sol decompression (#567)

* feat: sol de compression

* feat: JS compress and decompress lamports (#572)

---------

Co-authored-by: Swen Schäferjohann <[email protected]>

ci: add github ci syntax linter (#574)

chore: Remove patched arkworks crates (#575)

We are using only upstream 0.4.0 arkworks crates now, there is no need
to point to patched 0.3.0 crates.

feat: gnark circuits inclusion + non-inclusion 2-in-1 (#569)

* feat: inclusion + non-inclusion gnark circuits

* Refactor gnark server spawning and killing process.

add actions for initSolOmnibusAccount, compressLamports, decompressLamports + happy path tests (#578)

refactor: rename gnark proof inputs to plural form (#579)

* refactor: rename gnark proof inputs to plural form

* formatting

feat: add SOL compression and decompression cli commands (#580)

* Add SOL compression and decompression commands

* Use getTestRpc instead of Connection in decompress and compress cli cmds.

sync interface

rpc-interface coerce from photon response. open: tests

all endpoints except token

wip: debug delay in photon indexing

wip: RPC tests working. debug: merkleproof, notest: token

wip: add sleep

wip: fix cli build

fresh install changes gnark-prover

merkleproof confirmed for 2 consecutive leaves

wip

wip: built

rm compress test dupe

* cli, stateless.js, ctoken working with photon

* lint

* stdout

* cleanup logs

* initTestEnvIfNeeded with prover and indexer

* cli cmds: create-mint and mint-to working (happy path)

* transfer cli command works

* cli tests working: createmint, mint-to, transfer, balance

* compress, decompress, initsolpool working.

* linted

* cleanup

* disable vitest:browser for prover.js

* update readme: add usage instructions

* upd cli readme

* fix typo

* Update key loading logic and add test-validator options

* fix cli tests + update oclif

* Replace Connection with getTestRpc in cli command

* fix create-mint

* new cli + photon rpc

* remove manual prover.sh invocation in compressed-token/package.json

* clean up

* Add CLI test to light-sdk-tests.yml workflow

* Update cli/README.md

Co-authored-by: Michal Rostecki <[email protected]>

* fix cli prove test

* add "build @lightprotocol/programs" step to setup-and-build action

* Update cli/test/commands/config/index.test.ts

Co-authored-by: Swen Schäferjohann <[email protected]>

* Update test script in package.json

The `test` script in the package.json file was updated to include `test-prove` and `test-init-sol-pool`.

* Enable in-browser tests for prover.js package

* Update test command in cli/package.json

* rm commented code, address review

---------

Co-authored-by: Swen <[email protected]>
Co-authored-by: Michal Rostecki <[email protected]>
Co-authored-by: Swen Schäferjohann <[email protected]>
  • Loading branch information
4 people authored Apr 12, 2024
1 parent 2b79a37 commit 72c4439
Show file tree
Hide file tree
Showing 56 changed files with 3,988 additions and 3,385 deletions.
1 change: 0 additions & 1 deletion .github/actionlint.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
self-hosted-runner:
# Labels of self-hosted runner in array of strings.
labels:
- buildjet-16vcpu-ubuntu-2204
- buildjet-2vcpu-ubuntu-2204
# Configuration variables in array of strings defined in your repository or
# organization. `null` means disabling configuration variables check.
Expand Down
1 change: 1 addition & 0 deletions .github/actions/setup-and-build/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ runs:
mkdir -p /home/runner/.config/solana/
solana-keygen new --no-bip39-passphrase -o /home/runner/.config/solana/id.json
npx nx build @lightprotocol/hasher.rs
npx nx build @lightprotocol/programs
- name: Check for git changes
shell: bash
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/light-sdk-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
sub-tests: '[
"@lightprotocol/circuit-lib.js:test",
"@lightprotocol/prover.js:test",
"@lightprotocol/cli:test",
]'
services:
redis:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/prover-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version-file: './gnark-prover/go.mod'
go-version-file: "./gnark-prover/go.mod"

- name: Download keys
run: |
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@

/cli/bin

**/photon.log

# Yarn
**/.yarn
**/.pnpm
Expand Down
81 changes: 34 additions & 47 deletions cli/README.md
Original file line number Diff line number Diff line change
@@ -1,75 +1,62 @@
# Light CLI

Official CLI to interact with Light Protocol v3 and build Private Solana Programs (https://github.com/Lightprotocol/light-protocol)
CLI to interact with Light Protocol and use generalized ZK compression.

## Installation

To use Light CLI, you need to have Node.js (version 12 or later) and npm (Node Package Manager) installed on your machine. Follow the steps below to install [Your CLI Name] globally:
To use Light CLI, you need to have Node.js (version 12 or later) and npm (Node Package Manager) installed on your machine.

1. Open your terminal or command prompt.
2. Run the following command:
Please compile the CLI from source.

```shell
npm install -g @lightprotocol/cli
```
`. ./scripts/devenv`

3. After the installation is complete, you can verify the installation by running:
`./scripts/install.sh`

```shell
light --version
```
`./scripts/build.sh`

## Usage

The CLI lets you initialize a PSP scaffold project and customize, build, and test it. (`light psp:...` subcommand)
Note: currently, you have to start the light-test-validator, gnark-prover, and photon indexer outside the CLI binary:

It also provides utilities for local development, such as running a local Solana-test-validator with all necessary Light accounts and pre-loaded programs. (`light test-validator`).
`cd js/stateless.js`

You can also execute common user actions such as compressing and sending private transfers. (`light compress`, `light transfer`, `light decompress`).
`pnpm run pretest:e2e`

For the full list of available commands, see below:
This will reset and start the validator, prover, and indexer on a clean ledger.

### Commands
Alternatively, to start only the light-test-validator and the gnark-prover, run:

- `help`: Display help information.
- `account`: Get the current account details
- `airdrop`: Perform a native Solana or SPL airdrop to a user.
- `balance`: Retrieve the balance, inbox balance, or utxos for the user.
- `config`: Update the configuration values.
- `history`: Retrieve transaction history for the user.
- `accept-utxos`: Merge multiple utxos into a single UTXO.
- `compress`: Compress tokens for a user.
- `compress:sol`: Compress sol for a user.
- `compress:spl`: Compress spl tokens for a user.
- `transfer`: Transfer tokens to a recipient.
- `decompress`: Decompress tokens for a user.
`./cli/test_bin/run test-validator -p -i && pnpm gnark-prover`

- `decompress:sol`: Decompress sol for a user.
- `decompress:spl`: Decompress spl tokens for a user.
Note: the CLI currently expects the photon indexer to run at port: 8784, and the gnark-prover at port: 3001

- `test-validator`: Starts a solana-test-validator and with an initialized light environment. Use in a separate terminal instead of solana-test-validator.
Once you've started all services, in the same or a separate terminal window, go to the cli directory:

- merkleTree:
`cd cli`

- `mt:authority`: Initialize, set, or get the Merkle Tree Authority.
- `mt:configuration`: Update the configuration of the Merkle Tree NFTs, permissionless SPL tokens, and lock duration.
- `mt:initialize`: Initialize the Merkle Tree.
- `mt:pool`: Register a new pool type [default, spl, sol].
- `mt:verifier`: Register a new verifier for a Merkle Tree.
Ensure that the CLI is built.
Also ensure that you have a local solana wallet set up at ~/.config/solana/id.json. (see solana documentation for how to create one). This wallet will be used by the CLI as default feePayer and mintAuthority.

- psp:
- `psp:init`: Initialize, set, or get the Merkle Tree Authority
- `psp:build`: Update the configuration of the Merkle Tree NFTs, permissionless SPL tokens, and lock duration.
- `psp:test`: Perform the PSP tests.
Run `solana address` using the solana-cli to print your id.json/wallet address.
To ensure you have enough localnet funds: run `solana aidrop 10000000`

## PSP Guide
You can now create test-data against the test-ledger and photon with the following commands:

You can find a comprehensive tutorial for building a custom Private Solana Program [here](https://docs.lightprotocol.com/getting-started/creating-a-custom-psp).
`./test_bin/run create-mint`

## License
This will create a random mint and print its mint address.
You can then mint some tokens to your wallet.

@lightprotocol/cli is released under the GNU General Public License v3.0. See the LICENSE file for more details.
`./test_bin/run mint-to --mint "YOUR_MINT_ADDRESS_BASE58" --amount 4242 --to "YOUR_WALLET_ADDRESS_BASE58"`

## Contact
Next, you can transfer some of your compressed tokens to another solana wallet:

If you have questions, suggestions, or feedback, join the developer community on [Discord](https://discord.gg/J3KvDfZpyp), or reach out at hello[at]lightprotocol[dot]com.
`./test_bin/run transfer --mint "YOUR_MINT_ADDRESS_BASE58" --amount 3 --to "5bdFnXU47QjzGpzHfXnxcEi5WXyxzEAZzd1vrE39bf1W"`

Other commands include:

`./test_bin/run init-sol-pool` (must be run once before compressing lamports)

`./test_bin/run compress-sol --amount 1000 --to "YOUR_WALLET_ADDRESS_BASE58"`

`./test_bin/run decompress-sol --amount 42 --to "YOUR_WALLET_ADDRESS_BASE58"`
66 changes: 34 additions & 32 deletions cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -17,55 +17,56 @@
],
"dependencies": {
"@coral-xyz/anchor": "0.28.0",
"@lightprotocol/compressed-token": "workspace:*",
"@lightprotocol/hasher.rs": "workspace:*",
"@lightprotocol/stateless.js": "workspace:*",
"@lightprotocol/compressed-token": "workspace:*",
"@oclif/core": "^2.15",
"@oclif/plugin-autocomplete": "^2.3.8",
"@oclif/plugin-help": "^5.2.15",
"@oclif/plugin-not-found": "^2.4",
"@oclif/plugin-plugins": "^3.8.4",
"@oclif/core": "^3.26.2",
"@oclif/plugin-autocomplete": "^3.0.13",
"@oclif/plugin-help": "^6.0.20",
"@oclif/plugin-not-found": "^3.1.2",
"@oclif/plugin-plugins": "^5.0.7",
"@project-serum/anchor": "^0.26.0",
"@solana-developers/helpers": "^1.4.2",
"@solana/spl-token": "^0.3.7",
"@solana/web3.js": "^1.87.6",
"@typescript-eslint/eslint-plugin": "^6.7.3",
"@typescript-eslint/parser": "^6.7.3",
"@solana-developers/helpers": "^1.5.1",
"@solana/spl-token": "^0.3.11",
"@solana/web3.js": "^1.91.4",
"@typescript-eslint/eslint-plugin": "^7.6.0",
"@typescript-eslint/parser": "^7.6.0",
"case-anything": "^2.1.13",
"chai": "^4.3.10",
"chai": "^4.4.1",
"cli-progress": "^3.12.0",
"cli-spinners": "^2.9.0",
"dotenv": "^16.0.3",
"ffjavascript": "^0.2.62",
"cli-spinners": "^2.9.2",
"dotenv": "^16.4.5",
"ffjavascript": "^0.2.63",
"find-process": "^1.4.7",
"fs": "^0.0.1-security",
"node-fetch": "^3.3.1",
"fs": "0.0.1-security",
"node-fetch": "^3.3.2",
"path": "^0.12.7",
"snake-case": "^3.0.4",
"tar": "^6.1.15",
"tar": "^6.2.1",
"ts-mocha": "^10.0.0",
"tweetnacl": "^1.0.3",
"zlib": "^1.0.5"
},
"devDependencies": {
"@lightprotocol/programs": "workspace:*",
"@oclif/test": "2.3.33",
"@types/bn.js": "^5.1.1",
"@types/chai": "^4.3.9",
"@types/cli-progress": "^3.11.3",
"@types/mocha": "^10.0.3",
"@types/node": "^20.10.2",
"@types/tar": "^6.1.5",
"axios": "^1.5.0",
"@oclif/test": "2.3.9",
"@types/bn.js": "^5.1.5",
"@types/chai": "^4.3.14",
"@types/cli-progress": "^3.11.5",
"@types/mocha": "^10.0.6",
"@types/node": "^20.12.7",
"@types/tar": "^6.1.12",
"axios": "^1.6.8",
"chai": "^4.3.10",
"eslint": "^8.50.0",
"eslint-config-oclif": "^5",
"eslint-config-oclif-typescript": "^2",
"eslint": "8.57.0",
"eslint-config-oclif": "5.1.1",
"eslint-config-oclif-typescript": "3.1.4",
"eslint-config-prettier": "9.1.0",
"mocha": "^10.2.0",
"oclif": "^3.17.2",
"oclif": "4.8.0",
"prettier": "^3.0.3",
"shx": "^0.3.4",
"ts-node": "^10.9.1",
"ts-node": "^10.9.2",
"tslib": "^2.6.2",
"typescript": "5.3.2"
},
Expand Down Expand Up @@ -97,11 +98,12 @@
"test-mint-to": "mocha ./test/commands/mint-to/index.test.ts -t 10000000 --exit",
"test-transfer": "mocha ./test/commands/transfer/index.test.ts -t 10000000 --exit",
"test-balance": "mocha ./test/commands/balance/index.test.ts -t 10000000 --exit",
"test-init-sol-pool": "mocha ./test/commands/init-sol-pool/index.test.ts -t 10000000 --exit",
"test-compress-sol": "mocha ./test/commands/compress-sol/index.test.ts -t 10000000 --exit",
"test-decompress-sol": "mocha ./test/commands/decompress-sol/index.test.ts -t 10000000 --exit",
"kill": "killall solana-test-validator || true && killall solana-test-val || true && sleep 1",
"test-cli": "pnpm test-config && pnpm kill",
"test": "pnpm kill && pnpm test-cli && pnpm test-utils && pnpm test-create-mint && pnpm test-mint-to && pnpm test-transfer && pnpm test-balance && pnpm test-compress-sol && pnpm test-decompress-sol",
"test": "pnpm kill && pnpm test-prove && pnpm test-cli && pnpm test-utils && pnpm test-create-mint && pnpm test-mint-to && pnpm test-transfer && pnpm test-balance && pnpm test-init-sol-pool && pnpm test-compress-sol && pnpm test-decompress-sol",
"install-local": "pnpm build && pnpm global remove @lightprotocol/cli || true && pnpm global add $PWD",
"version": "oclif readme && git add README.md"
},
Expand Down
2 changes: 1 addition & 1 deletion cli/scripts/photon2bin.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ set -eux
root_dir="$(git rev-parse --show-toplevel)";
cli_dir="${root_dir}/cli"

cargo install --root "$cli_dir" photon-indexer
cargo install --root "$cli_dir" photon-indexer --version 0.11.0
14 changes: 6 additions & 8 deletions cli/src/commands/balance/index.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
import { Command, Flags } from "@oclif/core";
import { CustomLoader, getSolanaRpcUrl } from "../../utils/utils";
import { getCompressedTokenAccountsForTest } from "@lightprotocol/compressed-token";
import { PublicKey } from "@solana/web3.js";
import { getTestRpc } from "@lightprotocol/stateless.js";
import { createRpc } from "@lightprotocol/stateless.js";

class BalanceCommand extends Command {
static summary = "Get balance";
Expand All @@ -11,11 +10,11 @@ class BalanceCommand extends Command {

static flags = {
owner: Flags.string({
description: "Public Key of the compressed token owner.",
description: "Address of the compressed token owner.",
required: true,
}),
mint: Flags.string({
description: "Mint of the compressed token account.",
description: "Mint address of the compressed token account.",
required: true,
}),
};
Expand All @@ -29,11 +28,10 @@ class BalanceCommand extends Command {
try {
const refMint = new PublicKey(flags["mint"]);
const refOwner = new PublicKey(flags["owner"]);
const rpc = await getTestRpc(getSolanaRpcUrl());
const tokenAccounts = await getCompressedTokenAccountsForTest(
rpc,
const rpc = createRpc(getSolanaRpcUrl());
const tokenAccounts = await rpc.getCompressedTokenAccountsByOwner(
refOwner,
refMint,
{ mint: refMint },
);

loader.stop(false);
Expand Down
27 changes: 12 additions & 15 deletions cli/src/commands/compress-sol/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ import {
generateSolanaTransactionURL,
getSolanaRpcUrl,
} from "../../utils/utils";
import { Connection, PublicKey, LAMPORTS_PER_SOL } from "@solana/web3.js";
import { Rpc, compressLamports, getTestRpc } from "@lightprotocol/stateless.js";
import { PublicKey } from "@solana/web3.js";
import { compressLamports, createRpc } from "@lightprotocol/stateless.js";

class MintToCommand extends Command {
class CompressSolCommand extends Command {
static summary = "Compress SOL.";

static examples = ["$ light compress-sol --to PublicKey --amount 10"];
Expand All @@ -19,15 +19,15 @@ class MintToCommand extends Command {
required: true,
}),
amount: Flags.integer({
description: "Amount to mint, in SOL.",
description: "Amount to compress, in lamports.",
required: true,
}),
};

static args = {};

async run() {
const { flags } = await this.parse(MintToCommand);
const { flags } = await this.parse(CompressSolCommand);
const to = flags["to"];
const amount = flags["amount"];
if (!to || !amount) {
Expand All @@ -36,28 +36,25 @@ class MintToCommand extends Command {

const loader = new CustomLoader(`Performing compress-sol...\n`);
loader.start();

let txId;
try {
const toPublicKey = new PublicKey(to);
const payer = defaultSolanaWalletKeypair();

const rpc = await getTestRpc(getSolanaRpcUrl());
const txId = await compressLamports(
rpc,
payer,
amount * LAMPORTS_PER_SOL,
toPublicKey,
);
const rpc = createRpc(getSolanaRpcUrl());
txId = await compressLamports(rpc, payer, amount, toPublicKey);

loader.stop(false);
console.log(
"\x1b[compress-sol:\x1b[0m ",
"\x1b[32mtxId:\x1b[0m ",
generateSolanaTransactionURL("tx", txId, "custom"),
);
console.log("compress-sol successful");
} catch (error) {
console.log("compress-sol failed", txId);
this.error(`Failed to compress-sol!\n${error}`);
}
}
}

export default MintToCommand;
export default CompressSolCommand;
Loading

0 comments on commit 72c4439

Please sign in to comment.