Add solana-tokens (solana-labs#10011)

* Initial commit * Execute transfers * Refactor for testing * Cleanup readme * Rewrite * Cleanup * Cleanup * Cleanup client * Use a Null Client to move prints closer to where messages are sent * Upgrade Solana * Move core functionality into its own module * Handle transaction errors * Merge allocations * Fixes * Cleanup readme * Fix markdown * Add example input * Add integration test - currently fails * Add integration test * Add metrics * Use RpcClient in dry-run, just don't send messages * More metrics * Fix dry run with no keys * Only require one approval if fee-payer is the sender keypair * Fix bugs * Don't create the transaction log if nothing to put into it; otherwise the next innvocation won't add the header * Apply previous transactions to allocations with matching recipients * Bail out of any account already has a balance * Polish * Add new 'balances' command * 9 decimal places * Add missing file * Better dry-run; keypair options now optional * Change field name from 'bid' to 'accepted' Also, tolerate precision change from 2 decimal places to 4 * Write to transaction log immediately * Rename allocations_csv to bids_csv So that we can bypass bids_csv with an allocations CSV file * Upgrade Solana * Remove faucet from integration test * Cleaner integration test Won't work until this lands and is released: solana-labs#9717 * Update README * Add TravicCI script to build and test (#1) * Add distribute-stake command (#2) * Distribute -> DistributeTokens (#3) * Cache cargo deps (#4) * Add docs (#5) * Switch to latest Solana 1.1 release (#7) * distribute -> distribute-tokens (#9) * Switch from CSV to a pickledb database (#8) * Switch from CSV to a pickledb database * Allow PickleDb errors to bubble up * Dedup * Hoist db * Add finalized field to TransactionInfo * Don't allow RPC client to resign transactions * Remove dead code * Use transport::Result * Record unconfirmed transaction * Fix: separate stake account per allocation * Catch transport errors * Panic if we attempt to replay a transaction that hasn't been finalized * Attempt to fix CI PickleDb isn't calling flush() or close() after writing to files. No issue on MacOS, but looks racy in CI. * Revert "Attempt to fix CI" This reverts commit 1632394f636c54402b3578120e8817dd1660e19b. * Poll for signature before returning * Add --sol-for-fees option for stake distributions * Add --allocations-csv option (#14) * Add allocations-csv option * Add tests or GTFO * Apply review feedback * apply feedback * Add read_allocations function * Update arg_parser.rs * Fix balances command (#17) * Fix balances command * Fix readme * Add --force to transfer to non-empty accounts (#18) * Add --no-wait (#16) * Add ThinClient methods to implement --no-wait * Plumb --no-wait through No tests yet * Check transaction status on startup * Easier to test * Wait until transaction is finalized before checking if it failed with an error It's possible that a minority fork thinks it failed. * Add unit tests * Remove dead code and rustfmt * Don't flush database to file if doing a dry-run * Continue when transactions not yet finalized (#20) If those transactions are dropped, the next run will execute them. * Return the number of confirmations (#21) * Add read_allocations() unit-test (#22) Delete the copy-pasted top-level test. Fixes #19 * Add a CSV printer (#23) * Remove all the copypasta (#24) * Move resolve_distribute_stake_args into its own function * Add stake args to token args * Unify option names * Move Command::DistributeStake into DistributeTokens * Remove process_distribute_stake * Only unique signers * Use sender keypair to fund new fee-payer accounts * Unify distribute_tokens and distribute_stake * Rename print-database command to transaction-log (#25) * Send all transactions as quickly as possible, then wait (#26) * Send all transactions as quickly as possible, then wait * Exit when finalized or blockhashes have expired * Don't need blockhash in the CSV output * Better types CSV library was choking on Pubkey as a type. PickleDb doesn't have that problem. * Resend if blockhash has not expired * Attempt to fix CI * Move log to stderr * Add constructor, tuck away client (#30) * Add constructor, tuck away client * Fix unwrap() caught by CI * Fix optional option flagged as required * Bunch of cleanup (#31) * Remove untested --no-wait feature * Make --transactions-db an option, not an arg So that in the future, we can make it optional * Remove more untested features Too many false positives in that santity check. Use --dry-run instead. * Add dry-run mode to ThinClient * Cleaner dry-run * Make key parameters required Just don't use them in --dry-run * Add option to write the transaction log --dry-run doesn't write to the database. Use this option if you want a copy of the transaction log before the final run. * Revert --transaction-log addition Implement #27 first * Fix CI * Update readme * Fix CI in copypasta * Sort transaction log by finalized date (#33) * Make --transaction-db option implicit (#34) * Move db functionality into its own module (#35) * Move db functionality into its own module * Rename tokens module to commands * Version bump * Upgrade Solana * Add solana-tokens to build * Remove Cargo.lock * Remove vscode file * Remove TravisCI build script * Install solana-tokens Co-authored-by: Dan Albert <[email protected]>
aeyakovenko · May 13, 2020 · e09f517 · e09f517
1 parent 1eb40c3
commit e09f517
Show file tree

Hide file tree

Showing 14 changed files with 1,902 additions and 0 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -56,6 +56,7 @@ members = [
     "stake-accounts",
     "stake-monitor",
     "sys-tuner",
+    "tokens",
     "transaction-status",
     "upload-perf",
     "net-utils",

diff --git a/scripts/cargo-install-all.sh b/scripts/cargo-install-all.sh
@@ -67,6 +67,7 @@ if [[ $CI_OS_NAME = windows ]]; then
     solana-install-init
     solana-keygen
     solana-stake-accounts
+    solana-tokens
   )
 else
   ./fetch-perf-libs.sh
@@ -100,6 +101,7 @@ else
     solana-stake-accounts
     solana-stake-monitor
     solana-sys-tuner
+    solana-tokens
     solana-validator
     solana-watchtower
   )

diff --git a/tokens/.gitignore b/tokens/.gitignore
@@ -0,0 +1,2 @@
+target/
+*.csv
diff --git a/tokens/Cargo.toml b/tokens/Cargo.toml
@@ -0,0 +1,34 @@
+[package]
+name = "solana-tokens"
+description = "Blockchain, Rebuilt for Scale"
+authors = ["Solana Maintainers <[email protected]>"]
+edition = "2018"
+version = "1.2.0"
+repository = "https://github.com/solana-labs/solana"
+license = "Apache-2.0"
+homepage = "https://solana.com/"
+
+[dependencies]
+chrono = { version = "0.4", features = ["serde"] }
+clap = "2.33.0"
+console = "0.10.3"
+csv = "1.1.3"
+dirs = "2.0.2"
+indexmap = "1.3.2"
+indicatif = "0.14.0"
+itertools = "0.9.0"
+pickledb = "0.4.1"
+serde = { version = "1.0", features = ["derive"] }
+solana-clap-utils = { path = "../clap-utils", version = "1.2.0" }
+solana-cli-config = { path = "../cli-config", version = "1.2.0" }
+solana-client = { path = "../client", version = "1.2.0" }
+solana-remote-wallet = { path = "../remote-wallet", version = "1.2.0" }
+solana-runtime = { path = "../runtime", version = "1.2.0" }
+solana-sdk = { path = "../sdk", version = "1.2.0" }
+solana-stake-program = { path = "../programs/stake", version = "1.2.0" }
+solana-transaction-status = { path = "../transaction-status", version = "1.2.0" }
+tempfile = "3.1.0"
+thiserror = "1.0"
+
+[dev-dependencies]
+solana-core = { path = "../core", version = "1.2.0" }
diff --git a/tokens/README.md b/tokens/README.md
@@ -0,0 +1,105 @@
+# Distribute Solana tokens
+
+A user may want to make payments to multiple accounts over multiple iterations.
+The user will have a spreadsheet listing public keys and token amounts, and
+some process for transferring tokens to them, and ensuring that no more than the
+expected amount are sent. The command-line tool here automates that process.
+
+## Distribute tokens
+
+Send tokens to the recipients in `<BIDS_CSV>`.
+
+Example bids.csv:
+
+```text
+primary_address,bid_amount_dollars
+6Vo87BaDhp4v4GHwVDhw5huhxVF8CyxSXYtkUwVHbbPv,6.6
+```
+
+```bash
+solana-tokens distribute-tokens --from <KEYPAIR> --dollars-per-sol <NUMBER> --from-bids --input-csv <BIDS_CSV> --fee-payer <KEYPAIR>
+```
+
+Example transaction log before:
+
+```text
+recipient,amount,signature
+6Vo87BaDhp4v4GHwVDhw5huhxVF8CyxSXYtkUwVHbbPv,30,1111111111111111111111111111111111111111111111111111111111111111
+```
+
+Send tokens to the recipients in `<BIDS_CSV>` if the distribution is
+not already recordered in the transaction log.
+
+```bash
+solana-tokens distribute-tokens --from <KEYPAIR> --dollars-per-sol <NUMBER> --from-bids --input-csv <BIDS_CSV> --fee-payer <KEYPAIR>
+```
+
+Example output:
+
+```text
+Recipient                                     Amount
+6Vo87BaDhp4v4GHwVDhw5huhxVF8CyxSXYtkUwVHbbPv  70
+3ihfUy1n9gaqihM5bJCiTAGLgWc5zo3DqVUS6T736NLM  42
+UKUcTXgbeTYh65RaVV5gSf6xBHevqHvAXMo3e8Q6np8k  43
+```
+
+
+Example transaction log after:
+
+```bash
+solana-tokens transaction-log --output-path transactions.csv
+```
+
+```text
+recipient,amount,signature
+6Vo87BaDhp4v4GHwVDhw5huhxVF8CyxSXYtkUwVHbbPv,30,1111111111111111111111111111111111111111111111111111111111111111
+6Vo87BaDhp4v4GHwVDhw5huhxVF8CyxSXYtkUwVHbbPv,70,1111111111111111111111111111111111111111111111111111111111111111
+3ihfUy1n9gaqihM5bJCiTAGLgWc5zo3DqVUS6T736NLM,42,1111111111111111111111111111111111111111111111111111111111111111
+UKUcTXgbeTYh65RaVV5gSf6xBHevqHvAXMo3e8Q6np8k,43,1111111111111111111111111111111111111111111111111111111111111111
+```
+
+### Calculate what tokens should be sent
+
+List the differences between a list of expected distributions and the record of what
+transactions have already been sent.
+
+```bash
+solana-tokens distribute-tokens --dollars-per-sol <NUMBER> --dry-run --from-bids --input-csv <BIDS_CSV>
+```
+
+Example bids.csv:
+
+```text
+primary_address,bid_amount_dollars
+6Vo87BaDhp4v4GHwVDhw5huhxVF8CyxSXYtkUwVHbbPv,6.6
+6Vo87BaDhp4v4GHwVDhw5huhxVF8CyxSXYtkUwVHbbPv,15.4
+3ihfUy1n9gaqihM5bJCiTAGLgWc5zo3DqVUS6T736NLM,9.24
+UKUcTXgbeTYh65RaVV5gSf6xBHevqHvAXMo3e8Q6np8k,9.46
+```
+
+Example output:
+
+```text
+Recipient                                     Amount
+6Vo87BaDhp4v4GHwVDhw5huhxVF8CyxSXYtkUwVHbbPv  70
+3ihfUy1n9gaqihM5bJCiTAGLgWc5zo3DqVUS6T736NLM  42
+UKUcTXgbeTYh65RaVV5gSf6xBHevqHvAXMo3e8Q6np8k  43
+```
+
+## Distribute stake accounts
+
+Distributing tokens via stake accounts works similarly to how tokens are distributed. The
+big difference is that new stake accounts are split from existing ones. By splitting,
+the new accounts inherit any lockup or custodian settings of the original.
+
+```bash
+solana-tokens distribute-stake --stake-account-address <ACCOUNT_ADDRESS> \
+    --input-csv <ALLOCATIONS_CSV> \
+    --stake-authority <KEYPAIR> --withdraw-authority <KEYPAIR> --fee-payer <KEYPAIR>
+```
+
+Currently, this will subtract 1 SOL from each allocation and store it the
+recipient address. That SOL can be used to pay transaction fees on staking
+operations such as delegating stake. The rest of the allocation is put in
+a stake account. The new stake account address is output in the transaction
+log.