Fix node compatibility test CI

[banool]

Description

The issue was introduced by merging a bunch of run commands into one. How it used to work was:

1. cd ./ecosystem/typescript/sdk && pnpm generate-client -o /tmp/generated_client
2. git diff --no-index --ignore-space-at-eol --ignore-blank-lines ./ecosystem/typescript/sdk/src/generated/ /tmp/generated_client/

Now that they are merged, the git diff command in step 2 is trying to access ./ecosystem/typescript/sdk/src/generated/ but at this point we're already cd'd into this directory, so the path isn't there.

This PR fixes the issue by separating them into separate run commands again.

Test Plan

Because this uses workflow_dispatch I'll have to do the thing where I switch to pull_request temporarily. Stay posted.

Update: The fix works, see a successful run here: https://github.com/aptos-labs/aptos-core/actions/runs/5826782139.

[JoshLind]

Thanks @banool 😄

[github-actions]

✅ Forge suite compat success on aptos-node-v1.5.1 ==> 0ace148dab11a18e482740c6800ad6496333631d

Compatibility test results for aptos-node-v1.5.1 ==> 0ace148dab11a18e482740c6800ad6496333631d (PR)
1. Check liveness of validators at old version: aptos-node-v1.5.1
compatibility::simple-validator-upgrade::liveness-check : committed: 4861 txn/s, latency: 6528 ms, (p50: 6600 ms, p90: 9300 ms, p99: 10800 ms), latency samples: 194440
2. Upgrading first Validator to new version: 0ace148dab11a18e482740c6800ad6496333631d
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1782 txn/s, latency: 16175 ms, (p50: 18900 ms, p90: 22200 ms, p99: 22500 ms), latency samples: 92700
3. Upgrading rest of first batch to new version: 0ace148dab11a18e482740c6800ad6496333631d
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1441 txn/s, latency: 16482 ms, (p50: 18800 ms, p90: 22300 ms, p99: 22600 ms), latency samples: 92260
4. upgrading second batch to new version: 0ace148dab11a18e482740c6800ad6496333631d
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3148 txn/s, latency: 9910 ms, (p50: 10200 ms, p90: 13800 ms, p99: 14200 ms), latency samples: 135400
5. check swarm health
Compatibility test for aptos-node-v1.5.1 ==> 0ace148dab11a18e482740c6800ad6496333631d passed
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions]

❌ Forge suite framework_upgrade failure on aptos-node-v1.5.1 ==> 0ace148dab11a18e482740c6800ad6496333631d

Compatibility test results for aptos-node-v1.5.1 ==> 0ace148dab11a18e482740c6800ad6496333631d (PR)
Upgrade the nodes to version: 0ace148dab11a18e482740c6800ad6496333631d
Test Failed: Failed to get StatefulSet: ApiError: statefulsets.apps "aptos-node-3-validator" not found: NotFound (ErrorResponse { status: "Failure", message: "statefulsets.apps \"aptos-node-3-validator\" not found", reason: "NotFound", code: 404 })

Stack backtrace:
   0: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/result.rs:1961:27
      aptos_forge::backend::k8s::stateful_set::wait_stateful_set::{{closure}}
             at ./testsuite/forge/src/backend/k8s/stateful_set.rs:69:5
      aptos_forge::backend::k8s::stateful_set::scale_stateful_set_replicas::{{closure}}
             at ./testsuite/forge/src/backend/k8s/stateful_set.rs:243:6
   1: <aptos_forge::backend::k8s::node::K8sNode as aptos_forge::interface::node::Node>::start::{{closure}}
             at ./testsuite/forge/src/backend/k8s/node.rs:137:84
   2: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/future/future.rs:125:9
      <aptos_forge::backend::k8s::swarm::K8sSwarm as aptos_forge::interface::swarm::Swarm>::upgrade_validator::{{closure}}
             at ./testsuite/forge/src/backend/k8s/swarm.rs:262:27
   3: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/future/future.rs:125:9
      aptos_testcases::batch_update::{{closure}}
             at ./testsuite/testcases/src/lib.rs:51:60
      tokio::park::thread::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/park/thread.rs:267:54
      tokio::coop::with_budget::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/coop.rs:102:9
      std::thread::local::LocalKey<T>::try_with
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/thread/local.rs:270:16
      std::thread::local::LocalKey<T>::with
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/thread/local.rs:246:9
      tokio::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/coop.rs:95:13
      tokio::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/coop.rs:72:5
      tokio::park::thread::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/park/thread.rs:267:31
   4: tokio::runtime::enter::Enter::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/runtime/enter.rs:152:13
      tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/runtime/scheduler/multi_thread/mod.rs:79:9
   5: tokio::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/runtime/mod.rs:492:44
   6: <aptos_testcases::framework_upgrade::FrameworkUpgrade as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/framework_upgrade.rs:56:9
   7: aptos_forge::runner::Forge<F>::run::{{closure}}
             at ./testsuite/forge/src/runner.rs:481:42
      aptos_forge::runner::run_test
             at ./testsuite/forge/src/runner.rs:549:11
      aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:481:30
   8: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:403:11
      forge::main
             at ./testsuite/forge-cli/src/main.rs:329:21
   9: core::ops::function::FnOnce::call_once
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ops/function.rs:250:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/sys_common/backtrace.rs:135:18
  10: std::rt::lang_start::{{closure}}
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/rt.rs:166:18
  11: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ops/function.rs:284:13
      std::panicking::try::do_call
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/rt.rs:148:20
  12: main
  13: __libc_start_main
  14: _start
Trailing Log Lines:
      std::panicking::try
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/rt.rs:148:20
  12: main
  13: __libc_start_main
  14: _start


Swarm logs can be found here: See fgi output for more information.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ApiError: namespaces "forge-framework-upgrade-pr-9609" not found: NotFound (ErrorResponse { status: "Failure", message: "namespaces \"forge-framework-upgrade-pr-9609\" not found", reason: "NotFound", code: 404 })

Caused by:
    namespaces "forge-framework-upgrade-pr-9609" not found: NotFound

Stack backtrace:
   0: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/result.rs:1961:27
      aptos_forge::backend::k8s::cluster_helper::delete_k8s_cluster::{{closure}}
             at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:289:13
      aptos_forge::backend::k8s::cluster_helper::uninstall_testnet_resources::{{closure}}
             at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:399:48
   1: tokio::park::thread::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/park/thread.rs:267:54
      tokio::coop::with_budget::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/coop.rs:102:9
      std::thread::local::LocalKey<T>::try_with
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/thread/local.rs:270:16
      std::thread::local::LocalKey<T>::with
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/thread/local.rs:246:9
      tokio::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/coop.rs:95:13
      tokio::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/coop.rs:72:5
      tokio::park::thread::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/park/thread.rs:267:31
      tokio::runtime::enter::Enter::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/runtime/enter.rs:152:13
   2: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/runtime/scheduler/multi_thread/mod.rs:79:9
   3: tokio::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.21.2/src/runtime/mod.rs:492:44
   4: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
             at ./testsuite/forge/src/backend/k8s/swarm.rs:670:13
   5: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ptr/mod.rs:497:1
   6: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ptr/mod.rs:497:1
   7: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:494:9
   8: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:403:11
      forge::main
             at ./testsuite/forge-cli/src/main.rs:329:21
   9: core::ops::function::FnOnce::call_once
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ops/function.rs:250:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/sys_common/backtrace.rs:135:18
  10: std::rt::lang_start::{{closure}}
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/rt.rs:166:18
  11: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ops/function.rs:284:13
      std::panicking::try::do_call
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:500:40
      std::panicking::try
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:464:19
      std::panic::catch_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/rt.rs:148:20
  12: main
  13: __libc_start_main
  14: _start', testsuite/forge/src/backend/k8s/swarm.rs:672:18
stack backtrace:
   0: rust_begin_unwind
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/panicking.rs:67:14
   2: core::result::unwrap_failed
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/result.rs:1651:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/result.rs:1076:23
   4: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
             at ./testsuite/forge/src/backend/k8s/swarm.rs:670:13
   5: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ptr/mod.rs:497:1
   6: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ptr/mod.rs:497:1
   7: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:494:9
   8: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:403:11
   9: forge::main
             at ./testsuite/forge-cli/src/main.rs:329:21
  10: core::ops::function::FnOnce::call_once
             at /rustc/eb26296b556cef10fb713a38f3d16b9886080f26/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Debugging output:

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking

[github-actions]

✅ Forge suite realistic_env_max_load success on 0ace148dab11a18e482740c6800ad6496333631d

two traffics test: inner traffic : committed: 5916 txn/s, latency: 6620 ms, (p50: 6300 ms, p90: 8200 ms, p99: 15000 ms), latency samples: 2562060
two traffics test : committed: 100 txn/s, latency: 3031 ms, (p50: 2900 ms, p90: 3600 ms, p99: 5600 ms), latency samples: 1780
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.270, avg: 0.216", "QsPosToProposal: max: 0.357, avg: 0.201", "ConsensusProposalToOrdered: max: 0.662, avg: 0.632", "ConsensusOrderedToCommit: max: 0.551, avg: 0.522", "ConsensusProposalToCommit: max: 1.205, avg: 1.155"]
Max round gap was 1 [limit 4] at version 22308. Max no progress secs was 3.633121 [limit 10] at version 1098572.
Test Ok

• Grafana dashboard
• Humio Logs
• Test runner output
• Test run is land-blocking