stellar#40: add new scenario for diagnostic event tests on cli and js

.vscode/launch.json

@@ -0,0 +1,27 @@
+{
+    // example of how to debug and step through a system test feature using IDE
+    // need to have a running instance of quickstart:soroban-dev for rpc url
+    "version": "0.2.0",
+    "configurations": [
+        {
+			"name": "Debug Dapp Test",
+			"type": "go",
+			"request": "launch",
+			"host": "127.0.0.1",
+			"mode": "test",
+			"program": "${workspaceFolder}/features/dapp_develop/dapp_develop_test.go",
+			"cwd": "${workspaceFolder}",
+            "env": {
+                "SorobanExamplesGitHash": "v0.7.0",
+                "SorobanExamplesRepoURL": "https://github.com/stellar/soroban-examples.git",
+                "TargetNetworkPassPhrase": "Standalone Network ; February 2017",
+                "TargetNetworkSecretKey": "SC5O7VZUXDJ6JBDSZ74DSERXL7W3Y5LTOAMRF7RQRL3TAGAPS7LUVG3L",
+                "TargetNetworkPublicKey": "GBZXN7PIRZGNMHGA7MUUUF4GWPY5AYPV6LY4UV2GL6VJGIQRXFDNMADI",
+                "TargetNetworkRPCURL":"http://localhost:8000/soroban/rpc",
+                "LocalCore": "true",
+                "VerboseOutput": "true",
+                "FeaturePath": "./features/dapp_develop"
+            },
+		},
+    ]
+}

CHANGELOG.md

@@ -1,5 +1,7 @@
 # System Test Releases
 
+
+
 #### 1.0.7
 
 * Reorg the dockerfile for better cache-ability. [system-test, #31](https://github.com/stellar/system-test/pull/31).

Dockerfile

@@ -1,7 +1,7 @@
 ARG QUICKSTART_IMAGE_REF=stellar/quickstart:soroban-dev
 ARG SOROBAN_CLI_IMAGE_REF=stellar/system-test-soroban-cli:dev
 
-FROM golang:1.19 as go
+FROM golang:1.20 as go
 
 RUN ["mkdir", "-p", "/test"] 
 RUN ["mkdir", "-p", "/test/bin"] 
@@ -54,6 +54,10 @@ RUN groupadd --gid $USER_GID $USERNAME \
 RUN ["mkdir", "-p", "/home/tester"]
 USER tester
 WORKDIR /home/tester
+RUN mkdir -p ~/.ssh
+RUN chmod 700 ~/.ssh
+RUN echo "HOST *" > ~/.ssh/config
+RUN echo "StrictHostKeyChecking no" >> ~/.ssh/config
 
 # Install Node.js
 RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash
@@ -68,7 +72,7 @@ RUN npm i -g ts-node
 ARG JS_SOROBAN_CLIENT_NPM_VERSION
 ADD package*.json /home/tester/
 RUN npm ci && npm install "soroban-client@${JS_SOROBAN_CLIENT_NPM_VERSION}"
-ADD invoke.ts /home/tester/bin/
+ADD *.ts /home/tester/bin/
 RUN ["sudo", "chmod", "+x", "/home/tester/bin/invoke.ts"]
 
 FROM base as build

README.md

@@ -125,27 +125,27 @@ The default target network for system tests is a new/empty instance of standalon
 `--TargetNetworkTestAccountPublic "{your test account key pair info}"`  
 
 Debug mode, the docker container will exit with error code when any pre-setup or test fails to pass,
-you can enable DEBUG_MODE flag, and the container will stay running, prompting you for enter key before shutting down, make sure you invoke docker with `-it` so the prompt will reach your command line. While container is kept running, you can shell into it via `docker exec -it <container id or name>` and view log files of core, rpc, horizon all of which are located in container at `/var/log/supervisor`.  
+you can enable DEBUG_MODE flag, and the container will stay running, prompting you for enter key before shutting down, make sure you invoke docker with `-it` so the prompt will reach your command line. While container is kept running, you can shell into it via `docker exec -it <container id or name>` and view log files of services in the stack such as core, rpc located in container at `/var/log/supervisor`.  
 `--DebugMode=true`
 
 
 The docker run follows standard exit code conventions, so if all tests pass in the container run, exit code from command line execution will be 0, otherwise, if any failures in container or tests, then exit code will be greater than 0.
 
 
 ### Development mode and running tests directly from checked out system-test repo.
-This approach allows to run the tests from source code directly on host as go tests, no docker image.  
-Tests will use `soroban` cli tool from the host path. You need to set `TargetNetworkRPCURL` to a running instance of soroban rpc.
+This approach allows to run the tests from source code directly on host as go tests, no docker image is used. 
 
 #### Prerequisites:
 
  1. go 1.18 or above - https://go.dev/doc/install
  2. rust toolchain(cargo and rustc), install the version per testing requirements or stable, - use rustup - https://www.rust-lang.org/tools/install 
- 3. target network stack for the tests to execute against - need a soroban-rpc instance. You can use an existing/running instance if reachable or can use the quickstart image `stellar/quickstart:soroban-dev` from dockerhub to run the latest stable target network stack locally, or build quickstart with specific versions of core, horizon and soroban rpc first [following these instructions](https://github.com/stellar/quickstart#building-custom-images) and run `stellar/quickstart:dev` locally.
+ 3. `soroban`, compile or install via cargo crate a version of soroban cli onto your machine and accessible from PATH.
+ 4. target network stack for the tests to access soroban-rpc instance. You can use an existing/running instance if reachable or can use the quickstart image `stellar/quickstart:soroban-dev` from dockerhub to run the latest stable target network stack locally, or build quickstart with specific versions of core, horizon and soroban rpc first [following these instructions](https://github.com/stellar/quickstart#building-custom-images) and run `stellar/quickstart:dev` locally.
      ```
      docker run --rm -it -p 8000:8000 --name stellar stellar/quickstart:dev --standalone --enable-soroban-rpc --enable-core-artificially-accelerate-time-for-testing
      ```
- 4. locally checkout stellar/system-test GH repo and go into top folder - `git clone https://github.com/stellar/system-test.git;cd system-test`
- 5. compile or install via cargo crate a version of soroban cli onto your machine and accessible from PATH.
+ 5. locally checkout stellar/system-test GH repo and go into top folder - `git clone https://github.com/stellar/system-test.git;cd system-test`
+ 
 
 #### Running tests 
 ```
@@ -168,3 +168,8 @@ This example uses a feature/scenario filter also to limit which tests are run.
 
 * the verbose output of BDD scenerio results for tests is dependent on go's testing verbose output rules, need to specify -v and a directory with single package, if multiple packages detected on directory location, then go won't print verbose output for each package, i.e. you wont see the BDD scenerio summaries printed, just the standard one liner for summary of package pass/fail status.
 
+#### Debugging tests 
+
+Have provided a debug config [launch.json](.vscode/launch.json) for example reference on how to run a test with the go/dlv debugger to step through, etc.
+
+

e2e.go

@@ -27,6 +27,10 @@ type E2EConfig struct {
 	TargetNetworkPassPhrase string
 	TargetNetworkSecretKey  string
 	TargetNetworkPublicKey  string
+	// if true, means the core is running in same container as tests
+	LocalCore bool
+	// the relative feature file path
+	FeaturePath string
 }
 
 const (
@@ -77,12 +81,29 @@ type RPCLedgerEntryResponse struct {
 	Error  *RPCError         `json:"error,omitempty"`
 }
 
+type LatestLedgerResult struct {
+	// Hash of the latest ledger as a hex-encoded string
+	Hash string `json:"id"`
+	// Stellar Core protocol version associated with the ledger.
+	ProtocolVersion uint32 `json:"protocolVersion,string"`
+	// Sequence number of the latest ledger.
+	Sequence uint32 `json:"sequence"`
+}
+
+type RPCLatestLedgerResponse struct {
+	Result LatestLedgerResult `json:"result"`
+	Error  *RPCError          `json:"error,omitempty"`
+}
+
 const TestTmpDirectory = "test_tmp_workspace"
 
 func InitEnvironment() (*E2EConfig, error) {
 	var flagConfig = &E2EConfig{}
 	var err error
 
+	if flagConfig.FeaturePath, err = getEnv("FeaturePath"); err != nil {
+		return nil, err
+	}
 	if flagConfig.SorobanExamplesGitHash, err = getEnv("SorobanExamplesGitHash"); err != nil {
 		return nil, err
 	}
@@ -104,6 +125,9 @@ func InitEnvironment() (*E2EConfig, error) {
 	if verboseOutput, err := getEnv("VerboseOutput"); err == nil {
 		flagConfig.VerboseOutput, _ = strconv.ParseBool(verboseOutput)
 	}
+	if LocalCore, err := getEnv("LocalCore"); err == nil {
+		flagConfig.LocalCore, _ = strconv.ParseBool(LocalCore)
+	}
 
 	return flagConfig, nil
 }
@@ -180,6 +204,32 @@ func (a *Asserter) Errorf(format string, args ...interface{}) {
 	a.Err = fmt.Errorf(format, args...)
 }
 
+func QueryNetworkState(e2eConfig *E2EConfig) (LatestLedgerResult, error) {
+	getLatestLedger := []byte(`{
+           "jsonrpc": "2.0",
+           "id": 10235,
+           "method": "getLatestLedger"
+        }`)
+
+	resp, err := http.Post(e2eConfig.TargetNetworkRPCURL, "application/json", bytes.NewBuffer(getLatestLedger))
+	if err != nil {
+		return LatestLedgerResult{}, fmt.Errorf("soroban rpc get latest ledger had error %e", err)
+	}
+
+	var rpcResponse RPCLatestLedgerResponse
+	decoder := json.NewDecoder(resp.Body)
+	err = decoder.Decode(&rpcResponse)
+	if err != nil {
+		return LatestLedgerResult{}, fmt.Errorf("soroban rpc get latest ledger, not able to parse response, %v, %e", resp, err)
+	}
+	if rpcResponse.Error != nil {
+		return LatestLedgerResult{}, fmt.Errorf("soroban rpc get latest ledger, error on response, %v, %e", resp, err)
+	}
+
+	return rpcResponse.Result, nil
+
+}
+
 func QueryAccount(e2eConfig *E2EConfig, publicKey string) (*AccountInfo, error) {
 	decoded, err := strkey.Decode(strkey.VersionByteAccountID, publicKey)
 	if err != nil {

events.ts

@@ -0,0 +1,47 @@
+#!/usr/bin/env ts-node-script
+
+import { ArgumentParser } from 'argparse';
+import * as SorobanClient from 'soroban-client';
+const xdr = SorobanClient.xdr;
+
+async function main() {
+  const parser = new ArgumentParser({ description: 'Get contract events' })
+  parser.add_argument('--id', { dest: 'contractId', required: true, help: 'Contract ID' });
+  parser.add_argument('--rpc-url', { dest: 'rpcUrl', required: true, help: 'RPC URL' });
+  parser.add_argument('--size', { dest: 'size', required: true, help: 'Max Number of events to fetch' });
+  parser.add_argument('--ledgerFrom', { dest: 'ledgerFrom', help: 'Ledget Start' });
+
+  const {
+    contractId,
+    rpcUrl,
+    size,
+    ledgerFrom,
+  } = parser.parse_args() as Record<string, string>;
+
+  const server = new SorobanClient.Server(rpcUrl, { allowHttp: true });
+
+  let filters = [];
+
+  if (contractId != null) {
+    filters.push({
+      contractIds: [ contractId ]
+    });
+  }
+
+  let response = await server
+          .getEvents({ 
+            startLedger: Number(ledgerFrom), 
+            filters: filters,
+            limit: Number(size)});
+
+  if (!response.events) {
+      throw new Error(`No events in response: ${JSON.stringify(response)}`);
+  }
+
+  console.log(JSON.stringify(response.events));  
+}
+
+main().catch(err => {
+  console.error(JSON.stringify(err));
+  throw err;
+});

features/dapp_develop/cli.go

@@ -1,6 +1,7 @@
 package dapp_develop
 
 import (
+	"encoding/json"
 	"fmt"
 	"strings"
 
@@ -74,3 +75,46 @@ func invokeContractFromCliToolWithConfig(deployedContractId, contractName, funct
 
 	return stdOut, nil
 }
+
+func getEventsFromCliTool(ledgerFrom uint32, deployedContractId string, size uint32, e2eConfig *e2e.E2EConfig) ([]map[string]interface{}, error) {
+
+	args := []string{
+		"events",
+		"--start-ledger", fmt.Sprint(ledgerFrom),
+		"--count", fmt.Sprint(size),
+		"--id", deployedContractId,
+		"--rpc-url", e2eConfig.TargetNetworkRPCURL,
+		"--network-passphrase", e2eConfig.TargetNetworkPassPhrase,
+		"--output", "json",
+	}
+
+	envCmd := cmd.NewCmd("soroban", args...)
+
+	status, stdOutLines, err := e2e.RunCommand(envCmd, e2eConfig)
+	var jsonEvents []map[string]interface{}
+	var stdOutLinesTrimmed []string
+
+	if status != 0 || err != nil {
+		return jsonEvents, fmt.Errorf("soroban cli get events had error %v, %v", status, err)
+	}
+
+	if len(stdOutLines) > 0 {
+		// some output was produced to console by the cli events command,
+		// it could represent a correct or bad result, but need to remove the last line of it regardless
+		// because if it's correct results will have a last line of 'Latest Ledger: xxxx', which needs to be removed to
+		// parse the rest as valid json,
+		stdOutLinesTrimmed = stdOutLines[:len(stdOutLines)-1]
+	}
+
+	// put commas between any json event objects if more than one found
+	stdOutEventsValidJson := strings.ReplaceAll(strings.Join(stdOutLinesTrimmed, "\n"), `\n}\n{\n`, `\n}\n,\n{\n`)
+	// wrap the json objects in json array brackets
+	stdOutEventsValidJson = "[" + stdOutEventsValidJson + "]"
+
+	err = json.Unmarshal([]byte(stdOutEventsValidJson), &jsonEvents)
+	if err != nil {
+		return jsonEvents, fmt.Errorf("soroban cli get events console output %v was not parseable as event json, %e", strings.Join(stdOutLines, "\n"), err)
+	}
+
+	return jsonEvents, nil
+}

features/dapp_develop/dapp_develop.feature

@@ -1,15 +1,32 @@
 Feature: DApp Contract Development
 
+
+
+Scenario Outline: DApp developer retrieves contract events
+  Given I am using an rpc instance that has captive core config, ENABLE_SOROBAN_DIAGNOSTIC_EVENTS=true
+  And I used cargo to compile example contract <ContractExampleSubPath>
+  And I used rpc to verify my account is on the network
+  And I used rpc to get network latest ledger
+  And I used cli to deploy contract <ContractCompiledFileName> using my secret key
+  And I invoke function <FunctionName> on <ContractName> with request parameters <FunctionParams> from tool <Tool> using my secret key
+  Then The result should be to receive <EventCount> contract events and <DiagEventCount> diagnostic events for <ContractName> from <Tool>
+
+  Examples: 
+        | Tool         | ContractExampleSubPath | ContractName                  | ContractCompiledFileName             | FunctionName | FunctionParams         | RootIdentityName | Result             | EventCount | DiagEventCount |
+        | NODEJS       | hello_world            | soroban-hello-world-contract  | soroban_hello_world_contract.wasm    | hello        | Aloha                  | r1               | ["Hello","Aloha"]  | 0          | 1              |
+        | CLI          | hello_world            | soroban-hello-world-contract  | soroban_hello_world_contract.wasm    | hello        | --to=Aloha             | r1               | ["Hello","Aloha"]  | 0          | 1              |
+
+
 Scenario Outline: DApp developer compiles, installs, deploys and invokes a contract
-Given I used cargo to compile example contract <ContractExampleSubPath>
+  Given I used cargo to compile example contract <ContractExampleSubPath>
   And I used rpc to verify my account is on the network
   And I used cli to install contract <ContractCompiledFileName> on network using my secret key
   And I used cli to deploy contract <ContractCompiledFileName> by installed hash using my secret key
-  When I invoke function <FunctionName> on <ContractName> with request parameter <Param1> from <Tool> using my secret key
-  Then the result should be <Result>
+  When I invoke function <FunctionName> on <ContractName> with request parameters <FunctionParams> from tool <Tool> using my secret key
+  Then The result should be <Result>
 
   Examples: 
-        | Tool         | ContractExampleSubPath | ContractName                  | ContractCompiledFileName             | FunctionName | Param1         | Result             |
+        | Tool         | ContractExampleSubPath | ContractName                  | ContractCompiledFileName             | FunctionName | FunctionParams | Result             |
         | NODEJS       | hello_world            | soroban-hello-world-contract  | soroban_hello_world_contract.wasm    | hello        | Aloha          | ["Hello","Aloha"]  |
         | CLI          | hello_world            | soroban-hello-world-contract  | soroban_hello_world_contract.wasm    | hello        | --to=Aloha     | ["Hello","Aloha"]  |
         | NODEJS       | increment              | soroban-increment-contract    | soroban_increment_contract.wasm      | increment    |                | 1                  |
@@ -20,15 +37,15 @@ Scenario Outline: DApp developer compiles, deploys and invokes a contract
   Given I used cargo to compile example contract <ContractExampleSubPath>
   And I used rpc to verify my account is on the network
   And I used cli to deploy contract <ContractCompiledFileName> using my secret key
-  When I invoke function <FunctionName> on <ContractName> with request parameter <Param1> from <Tool> using my secret key
-  Then the result should be <Result>
+  When I invoke function <FunctionName> on <ContractName> with request parameters <FunctionParams> from tool <Tool> using my secret key
+  Then The result should be <Result>
 
   Examples: 
-        | Tool         | ContractExampleSubPath | ContractName                  | ContractCompiledFileName             | FunctionName | Param1     | Result             |
-        | NODEJS       | hello_world            | soroban-hello-world-contract  | soroban_hello_world_contract.wasm    | hello        | Aloha      | ["Hello","Aloha"]  |
-        | CLI          | hello_world            | soroban-hello-world-contract  | soroban_hello_world_contract.wasm    | hello        | --to=Aloha | ["Hello","Aloha"]  |
-        | NODEJS       | increment              | soroban-increment-contract    | soroban_increment_contract.wasm      | increment    |            | 1                  |
-        | CLI          | increment              | soroban-increment-contract    | soroban_increment_contract.wasm      | increment    |            | 1                  |
+        | Tool         | ContractExampleSubPath | ContractName                  | ContractCompiledFileName             | FunctionName | FunctionParams | Result             |
+        | NODEJS       | hello_world            | soroban-hello-world-contract  | soroban_hello_world_contract.wasm    | hello        | Aloha          | ["Hello","Aloha"]  |
+        | CLI          | hello_world            | soroban-hello-world-contract  | soroban_hello_world_contract.wasm    | hello        | --to=Aloha     | ["Hello","Aloha"]  |
+        | NODEJS       | increment              | soroban-increment-contract    | soroban_increment_contract.wasm      | increment    |                | 1                  |
+        | CLI          | increment              | soroban-increment-contract    | soroban_increment_contract.wasm      | increment    |                | 1                  |
 
 
 Scenario Outline: DApp developer uses config states, compiles, deploys and invokes contract with authorizations
@@ -38,9 +55,9 @@ Scenario Outline: DApp developer uses config states, compiles, deploys and invok
   And I used cli to add Network Config <NetworkConfigName> for rpc and standalone
   And I used cli to add Identity <RootIdentityName> for my secret key
   And I used cli to add Identity <TesterIdentityName> for tester secret key
-  And I used cli to deploy contract <ContractCompiledFileName> using my Identity <RootIdentityName> and Network Config <NetworkConfigName>
-  When I invoke function <FunctionName> on <ContractName> with request parameters <FunctionParams> from <Tool> using tester Identity <TesterIdentityName> as invoker and Network Config <NetworkConfigName>
-  Then the result should be <Result>
+  And I used cli to deploy contract <ContractCompiledFileName> using Identity <RootIdentityName> and Network Config <NetworkConfigName>
+  When I invoke function <FunctionName> on <ContractName> with request parameters <FunctionParams> from tool <Tool> using Identity <TesterIdentityName> as invoker and Network Config <NetworkConfigName>
+  Then The result should be <Result>
 
   Examples: 
         | Tool         | ContractExampleSubPath | ContractName                  | ContractCompiledFileName      | FunctionName     | FunctionParams                              | RootIdentityName  | TesterIdentityName  | NetworkConfigName   | Result |