Run TS SDK tests against local testnets built from prod branches, split API and Rust node API client checks into separate job

[banool]

Description

Currently we run the TS SDK tests against a local testnet built from the same commit. So if you make a PR, it will run a local testnet built from the code in that PR and then run the tests against it. This helps check for compatibility between the SDK and main but that's actually not really necessary. Instead, what matters is confirming that the SDK is compatible with each of the production networks. As such, in this PR I make it that the SDK runs against local testnets built from the production networks (devnet, testnet, mainnet).

To ensure compatibility we run these tests on main and on PRs to check for changes to the SDK that could break compatibility with the production networks.

One nice benefit of this change is the SDK tests no longer have to wait for the whole image build phase, it can just run immediately.

I also switch up publish_ans_contract.ts to make the retries actually work in CI. They didn't previously because we weren't deleting the ANS directory properly, so now we use temp directories. The way I do it has to workaround the following issues that GitHub the company probably won't fix any time soon:

• https://github.com/orgs/community/discussions/58633 (https://support.github.com/ticket/personal/0/2217441)
• https://stackoverflow.com/a/76523941/3846032

After removing the TS SDK tests, I spit the remaining stuff into two separate workflows too:

1. I put the API + spec + generated TS code compatibility tests into their own workflow.
2. I put the Rust SDK client tests into their own workflow. This workflow works the same as the TS SDK tests, in which we run the code against three different local testnets.

Note: With this new setup, it means that some of the tests as they are now don't work against testnet and mainnet because the fungible asset module is not deployed there yet. To fix this, I made the tests run conditionally depending on the NETWORK env var. I also improved the error messages by making the test suite check that the txns are successful.

This addresses #8740.

Test Plan

CI. I tested it with pull_request instead of pull_request_target and the relevant tests (the api compatibility tests) worked.

The Rust node API client tests are flaky because of this toolchain issue. It is not related to this PR, I have seen them pass.

I also ran the TS SDK tests locally and they worked.

[0xmaayan]

BEAUTIFUL!

[0xmaayan]

nice!

[0xmaayan]

mktemp create a temp directory? how is it a "temp" directory? is it get self destroyed once the process is done?

[banool]

Not quite, it gets cleaned up by the OS at some point, where that exact point is vague and mysterious but will happen eventually. On Mac OS it's roughly 3 days or when you're running out of space.

[github-actions]

✅ Forge suite land_blocking success on a7685294c0b0ddab509ee2f33886b65eaa00d584

performance benchmark : committed: 5425 txn/s, submitted: 5426 txn/s, expired: 1 txn/s, latency: 7251 ms, (p50: 5100 ms, p90: 15300 ms, p99: 27400 ms), latency samples: 2316558
Max round gap was 1 [limit 4] at version 868034. Max no progress secs was 3.952009 [limit 10] at version 868034.
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> a7685294c0b0ddab509ee2f33886b65eaa00d584

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> a7685294c0b0ddab509ee2f33886b65eaa00d584 (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : committed: 8433 txn/s, latency: 4082 ms, (p50: 3700 ms, p90: 6700 ms, p99: 8800 ms), latency samples: 303620
2. Upgrading first Validator to new version: a7685294c0b0ddab509ee2f33886b65eaa00d584
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 4294 txn/s, latency: 7489 ms, (p50: 8800 ms, p90: 9900 ms, p99: 10200 ms), latency samples: 163180
3. Upgrading rest of first batch to new version: a7685294c0b0ddab509ee2f33886b65eaa00d584
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 4765 txn/s, latency: 6655 ms, (p50: 7100 ms, p90: 8700 ms, p99: 9100 ms), latency samples: 181100
4. upgrading second batch to new version: a7685294c0b0ddab509ee2f33886b65eaa00d584
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 6238 txn/s, latency: 5318 ms, (p50: 4900 ms, p90: 8500 ms, p99: 10300 ms), latency samples: 212120
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> a7685294c0b0ddab509ee2f33886b65eaa00d584 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite framework_upgrade success on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> a7685294c0b0ddab509ee2f33886b65eaa00d584

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> a7685294c0b0ddab509ee2f33886b65eaa00d584 (PR)
Upgrade the nodes to version: a7685294c0b0ddab509ee2f33886b65eaa00d584
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 4987 txn/s, latency: 6358 ms, (p50: 6300 ms, p90: 9200 ms, p99: 16300 ms), latency samples: 189540
5. check swarm health
Compatibility test for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> a7685294c0b0ddab509ee2f33886b65eaa00d584 passed
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[banool]

All other checks are green except for https://github.com/aptos-labs/aptos-core/actions/runs/5336576362/jobs/9671938125?pr=8769. This is a shadow of the past, just kept around by pull_request_target, it will be gone when I land this. Force landing now.