replace api_key with gcp secret #15080

yuunlimm · 2024-10-24T22:41:33Z

Description

This is to fix trigger processor tests workflow where it's failing to generate files due to having no api keys specified in the yuaml file. we have added a new secret to GCP and this change updates the workflow to pull the tokens from GCP and replace the placeholder string with the tokens.

How Has This Been Tested?

https://github.com/aptos-labs/aptos-core/actions/runs/11525821798/job/32088828463

Key Areas to Review

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

I have read and followed the CONTRIBUTING doc
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I identified and added all stakeholders and component owners affected by this change as reviewers
I tested both happy and unhappy path of the functionality
I have made corresponding changes to the documentation

trunk-io · 2024-10-24T22:41:36Z

⏱️ 7h 24m total CI duration on this PR

Slowest 15 Jobs	Cumulative Duration	Recent Runs
rust-cargo-deny	44m	🟩 🟩 🟩 🟩 🟩 (+21 more)
execution-performance / test-target-determinator	36m	🟩 🟩 🟩 🟩 🟩 (+3 more)
check-dynamic-deps	33m	🟩 🟩 🟩 🟩 🟩 (+21 more)
test-target-determinator	30m	🟩 🟩 🟩 🟩 🟩 (+3 more)
check	26m	🟩 🟩 🟩 🟩 🟩 (+3 more)
dispatch_event	24m	🟥
dispatch_event	22m	🟥
dispatch_event	22m	🟥
dispatch_event	21m	🟥
general-lints	12m	🟩 🟩 🟩 🟩 🟩 (+21 more)
fetch-last-released-docker-image-tag	11m	🟩 🟩 🟩 🟩 🟩 (+3 more)
semgrep/ci	11m	🟩 🟩 🟩 🟩 🟩 (+21 more)
forge-compat-test / forge	9m	🟥
dispatch_event	7m	🟥
dispatch_event	6m	🟥

🚨 1 job on the last run was significantly faster/slower than expected

Job	Duration	vs 7d avg	Delta
execution-performance / single-node-performance	10s	20m

_{settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io}

rtso · 2024-10-28T17:29:08Z

ecosystem/indexer-grpc/indexer-transaction-generator/template_config.yaml

There's multiple of the example .yaml files. Can we clean them up so there's only 1?

rtso · 2024-10-28T17:29:29Z

ecosystem/indexer-grpc/indexer-transaction-generator/example_tests/imported_transactions.yaml

Whenever we import new transactions for the testing framework, do we need to update this file with the versions/filename? I haven't been updating this file bc I thought it was used as an example yaml file.

If so, maybe we can update the folder to something else instead of example_tests, maybe like imported_transactions?

yes we should renamed this folder, example_tests is a very tricky name lol

…ary example config yaml

graphite-app · 2024-10-28T20:48:45Z

.github/workflows/indexer-processor-testing.yaml

+          sed -i "s/TESTNET_API_KEY/${{ steps.api_key_tokens.outputs.testnet_api_key }}/g" ./imported_transactions/imported_transactions.yaml
+          sed -i "s/MAINNET_API_KEY/${{ steps.api_key_tokens.outputs.mainnet_api_key }}/g" ./imported_transactions/imported_transactions.yaml
+
+          cat ./imported_transactions/imported_transactions.yaml  # Print the updated file for verification


The cat command here prints API keys to public workflow logs, creating a security risk. While the sed commands confirm the file modifications are working, exposing sensitive credentials should be avoided. Consider removing this debug line or redirecting the output to a secure location if verification is needed.

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.

github-actions · 2024-10-28T23:08:04Z

✅ Forge suite `realistic_env_max_load` success on `8da9876c7f369523f86d4836a44b8fa002712e79`

two traffics test: inner traffic : committed: 14396.96 txn/s, latency: 2759.34 ms, (p50: 2700 ms, p70: 2700, p90: 3000 ms, p99: 3300 ms), latency samples: 5474100
two traffics test : committed: 100.05 txn/s, latency: 1405.27 ms, (p50: 1400 ms, p70: 1400, p90: 1500 ms, p99: 2500 ms), latency samples: 1720
Latency breakdown for phase 0: ["MempoolToBlockCreation: max: 2.007, avg: 1.556", "ConsensusProposalToOrdered: max: 0.331, avg: 0.294", "ConsensusOrderedToCommit: max: 0.369, avg: 0.355", "ConsensusProposalToCommit: max: 0.660, avg: 0.648"]
Max non-epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 0.94s no progress at version 2578728 (avg 0.20s) [limit 15].
Max epoch-change gap was: 0 rounds at version 0 (avg 0.00) [limit 4], 8.43s no progress at version 2578726 (avg 8.43s) [limit 15].
Test Ok

github-actions · 2024-10-28T23:10:17Z

✅ Forge suite `framework_upgrade` success on `9a44d73b5a6cd123562f466ec6852ebd4fdb7305` ==> `8da9876c7f369523f86d4836a44b8fa002712e79`

Compatibility test results for 9a44d73b5a6cd123562f466ec6852ebd4fdb7305 ==> 8da9876c7f369523f86d4836a44b8fa002712e79 (PR)
Upgrade the nodes to version: 8da9876c7f369523f86d4836a44b8fa002712e79
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1404.09 txn/s, submitted: 1407.99 txn/s, failed submission: 3.91 txn/s, expired: 3.91 txn/s, latency: 2181.26 ms, (p50: 2100 ms, p70: 2400, p90: 2700 ms, p99: 3900 ms), latency samples: 122240
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1361.77 txn/s, submitted: 1365.15 txn/s, failed submission: 3.38 txn/s, expired: 3.38 txn/s, latency: 2181.03 ms, (p50: 2100 ms, p70: 2400, p90: 3300 ms, p99: 4500 ms), latency samples: 120720
5. check swarm health
Compatibility test for 9a44d73b5a6cd123562f466ec6852ebd4fdb7305 ==> 8da9876c7f369523f86d4836a44b8fa002712e79 passed
Upgrade the remaining nodes to version: 8da9876c7f369523f86d4836a44b8fa002712e79
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1369.00 txn/s, submitted: 1374.09 txn/s, failed submission: 5.09 txn/s, expired: 5.09 txn/s, latency: 2119.43 ms, (p50: 2100 ms, p70: 2400, p90: 3000 ms, p99: 4800 ms), latency samples: 123660
Test Ok

github-actions · 2024-10-28T23:14:37Z

✅ Forge suite `compat` success on `9a44d73b5a6cd123562f466ec6852ebd4fdb7305` ==> `8da9876c7f369523f86d4836a44b8fa002712e79`

Compatibility test results for 9a44d73b5a6cd123562f466ec6852ebd4fdb7305 ==> 8da9876c7f369523f86d4836a44b8fa002712e79 (PR)
1. Check liveness of validators at old version: 9a44d73b5a6cd123562f466ec6852ebd4fdb7305
compatibility::simple-validator-upgrade::liveness-check : committed: 15218.89 txn/s, latency: 2201.82 ms, (p50: 1900 ms, p70: 2100, p90: 3600 ms, p99: 5400 ms), latency samples: 499840
2. Upgrading first Validator to new version: 8da9876c7f369523f86d4836a44b8fa002712e79
compatibility::simple-validator-upgrade::single-validator-upgrading : committed: 6425.04 txn/s, latency: 4389.66 ms, (p50: 4700 ms, p70: 4900, p90: 5700 ms, p99: 6000 ms), latency samples: 123280
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 6504.13 txn/s, latency: 4996.63 ms, (p50: 5300 ms, p70: 5500, p90: 6500 ms, p99: 7000 ms), latency samples: 223800
3. Upgrading rest of first batch to new version: 8da9876c7f369523f86d4836a44b8fa002712e79
compatibility::simple-validator-upgrade::half-validator-upgrading : committed: 5888.13 txn/s, latency: 4917.93 ms, (p50: 5700 ms, p70: 5900, p90: 6000 ms, p99: 6100 ms), latency samples: 112400
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 6179.26 txn/s, latency: 5232.76 ms, (p50: 5800 ms, p70: 5900, p90: 6000 ms, p99: 6100 ms), latency samples: 213980
4. upgrading second batch to new version: 8da9876c7f369523f86d4836a44b8fa002712e79
compatibility::simple-validator-upgrade::rest-validator-upgrading : committed: 8373.54 txn/s, latency: 3388.49 ms, (p50: 3700 ms, p70: 4200, p90: 4400 ms, p99: 4900 ms), latency samples: 152040
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 8887.18 txn/s, latency: 3580.17 ms, (p50: 3500 ms, p70: 3700, p90: 6100 ms, p99: 6300 ms), latency samples: 292040
5. check swarm health
Compatibility test for 9a44d73b5a6cd123562f466ec6852ebd4fdb7305 ==> 8da9876c7f369523f86d4836a44b8fa002712e79 passed
Test Ok

yuunlimm force-pushed the yuunlimm/replace_api_key branch 13 times, most recently from 62b78e9 to 8d2d6bd Compare October 25, 2024 21:53

yuunlimm marked this pull request as ready for review October 25, 2024 21:58

yuunlimm requested a review from a team as a code owner October 25, 2024 21:58

yuunlimm requested a review from a team October 25, 2024 21:58

larry-aptos approved these changes Oct 25, 2024

View reviewed changes

yuunlimm enabled auto-merge (squash) October 28, 2024 16:49

This comment has been minimized.

Sign in to view

rtso approved these changes Oct 28, 2024

View reviewed changes

yuunlimm added 4 commits October 28, 2024 13:41

replace api_key with gcp secret

f28d3be

add new json to trigger test

d1a287d

auth before checkout

c56a97e

fix path

d654151

rename the example_tests -> imported_transactions and remove unnecess…

6543923

…ary example config yaml

yuunlimm force-pushed the yuunlimm/replace_api_key branch from 8d2d6bd to 6543923 Compare October 28, 2024 20:42

update workflow

6b8f834

graphite-app bot reviewed Oct 28, 2024

View reviewed changes

yuunlimm force-pushed the yuunlimm/replace_api_key branch from 9c86e8b to f429c53 Compare October 28, 2024 21:08

This comment has been minimized.

Sign in to view

fix

d4d7f74

yuunlimm force-pushed the yuunlimm/replace_api_key branch from f429c53 to d4d7f74 Compare October 28, 2024 21:41

This comment has been minimized.

Sign in to view

Merge branch 'main' into yuunlimm/replace_api_key

8da9876

This comment has been minimized.

Sign in to view

yuunlimm merged commit 0e7ef1c into main Oct 28, 2024
48 checks passed

yuunlimm deleted the yuunlimm/replace_api_key branch October 28, 2024 23:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

replace api_key with gcp secret #15080

replace api_key with gcp secret #15080

yuunlimm commented Oct 24, 2024 •

edited

Loading

trunk-io bot commented Oct 24, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rtso Oct 28, 2024

rtso Oct 28, 2024

yuunlimm Oct 28, 2024

graphite-app bot Oct 28, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Oct 28, 2024

github-actions bot commented Oct 28, 2024

github-actions bot commented Oct 28, 2024

replace api_key with gcp secret #15080

replace api_key with gcp secret #15080

Conversation

yuunlimm commented Oct 24, 2024 • edited Loading

Description

How Has This Been Tested?

Key Areas to Review

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

trunk-io bot commented Oct 24, 2024 • edited Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

rtso Oct 28, 2024

Choose a reason for hiding this comment

rtso Oct 28, 2024

Choose a reason for hiding this comment

yuunlimm Oct 28, 2024

Choose a reason for hiding this comment

graphite-app bot Oct 28, 2024

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented Oct 28, 2024

✅ Forge suite realistic_env_max_load success on 8da9876c7f369523f86d4836a44b8fa002712e79

github-actions bot commented Oct 28, 2024

✅ Forge suite framework_upgrade success on 9a44d73b5a6cd123562f466ec6852ebd4fdb7305 ==> 8da9876c7f369523f86d4836a44b8fa002712e79

github-actions bot commented Oct 28, 2024

✅ Forge suite compat success on 9a44d73b5a6cd123562f466ec6852ebd4fdb7305 ==> 8da9876c7f369523f86d4836a44b8fa002712e79

yuunlimm commented Oct 24, 2024 •

edited

Loading

trunk-io bot commented Oct 24, 2024 •

edited

Loading

✅ Forge suite `realistic_env_max_load` success on `8da9876c7f369523f86d4836a44b8fa002712e79`

✅ Forge suite `framework_upgrade` success on `9a44d73b5a6cd123562f466ec6852ebd4fdb7305` ==> `8da9876c7f369523f86d4836a44b8fa002712e79`

✅ Forge suite `compat` success on `9a44d73b5a6cd123562f466ec6852ebd4fdb7305` ==> `8da9876c7f369523f86d4836a44b8fa002712e79`