E2E Tests (Release 24.12)

[obalandi]

Please open and fill in this issue in your product repository to document the status of E2E tests.

**Show success/failure ** by attaching screenshots and/or result documents/failure messages.

Close this issue once the status of the E2E tests has been documented

Testers(s) and Role(s):

• Oguzhan Balandi (T-Systems, CONSUMER, SUPPLIER)
• Christoph Jung (T-Systems, OEM, PROVIDER, PRODUCER)

Deployment/Umbrella Chart Coordinates:

• https://github.com/big-data-spaces/knowledge-agents-deployment/tree/release/cx-2408/infrastructure

Tested Sub-Charts/Versions:

• tractusx-connector-azure-vault: 0.8.0-RC4
• agent-plane-azure-vault: 1.14.24-SNAPSHOT
• conforming-agent: 1.14.24-SNAPSHOT
• matchmaking-agent: 1.14.24-SNAPSHOT
• remoting-agent: 1.14.24-SNAPSHOT
• provisioning-agent: 1.14.24-SNAPSHOT
• aas-bridge: 1.14.24-SNAPSHOT

Retest with:

• agent-plane-azure-vault: 0.8.0-RC4
• matchmaking-agent: 1.14.24-SNAPSHOT

Release Management Reference Issue:

eclipse-tractusx/sig-release#902

Preconditions / Integration Tests

• For Release 24.12 Integration Test various scenarios need to be performed by the testing parties depending on their role (CONSUMER, OEM, PROVIDER, PRODUCER, SUPPLIER).
• Example performances of the integration steps can be found in the postman collection

CONSUMER of a sample OEM and PROVIDER:

• Deploy Agent-Enabled Connector
• Perform Remote Catalogue Request (OEM and PROVIDER)
• Check Federated Catalogue (need to list OEM and PROVIDER assets)

OEM for a sample SUPPLIER, access Provisioning Agents and register a graph

• Deploy Agent-Enabled Connector
• Perform Remote Catalogue Request (against SUPPLIER)
• Check Federated Catalogue (needs to see SUPPLIER and PROVIDER assets)
• Deploy Provisioning Agent
• Access Provisioning Agent (Reliability)
• Access Provisioning Agent (Supply Chain)
• Register Graph Asset(s)
• Register Graph Policy
• Register Graph Contract

SUPPLIER for a sample PRODUCER and OEM, access Remoting Agent and register a graph

• Deploy Agent-Enabled Connector
• Perform Remote Catalogue Request (against PRODUCER and OEM)
• Check Federated Catalogue (of PRODUCER and OEM)
• Deploy Remoting Agent with RUL binding
• Access Remoting Agent (Remaining Useful Life)
• Register Graph Asset
• Upload Graph Asset (Supply Chain)
• Register Graph Policy
• Register Graph Contract

PRODUCER for a sample SUPPLIER and register a graph

• Deploy Agent-Enabled Connector
• Perform Remote Catalogue Request
• Check Federated Catalogue (needs to see SUPPLIER and PROVIDER)
• Upload Graph Asset (Supply Chain)
• Register Graph Policy
• Register Graph Contract

PROVIDER for sample OEM, SUPPLIER and PRODUCER and register a Skill

• Deploy Agent-Enabled Connector
• Perform Remote Catalogue Request (against OEM, SUPPLIER and PRODUCER)
• Register Skill Asset
• Register Skill Policy
• Register Skill Contract

Test Plan

• The Test Steps are documented as API interactions in this postman collection.
• The concrete Test environment is referred in this postman environment. The necessary secrets will be communicated by the test management in private.

PROVIDER Test

• E2E_02_00_01_OEM_GRAPH_POLICY OEM Creates a Policy for Providing Graphs and Skills to the E2E participants
• E2E_02_00_02_OEM_GRAPH_CONTRACT OEM Creates a Contract Definition for Providing Graphs and Skills to the E2E participants
• E2E_02_00_03_OEM_RELIABILITY_GRAPH OEM Creates a Graph Asset with Telematics Data
• E2E_02_01_00_SUPPLIER_GRAPH_POLICY SUPPLIER Creates a Policy for Providing Graphs to the E2E participants
• E2E_02_01_01_SUPPLIER_GRAPH_CONTRACT SUPPLIER Creates a Contract Definition for Providing Graphs to the E2E participants
• E2E_02_01_02_SUPPLIER_BEHAVIOUR_GRAPH SUPPLIER Creates a Graph Asset with RUL Prognosis
• E2E_02_01_03_SUPPLIER_BEHAVIOUR_GRAPH_VARIANT SUPPLIER Creates a Graph Asset with HI Prognosis
• E2E_02_02_01_OPEN_POLICY PROVIDER Creates a Permissive Policy
• E2E_02_02_02_OPEN_CONTRACT PROVIDER Creates a Permissive Contract Definition
• E2E_02_02_03_REGISTER_RUL_SKILL_REMOTE PROVIDER Creates a RUL Skill in PROVIDER Mode
• E2E_02_02_04_REGISTER_RUL_SKILL_LOCAL PROVIDER Creates a RUL Skill in CONSUMER Mode
• E2E_02_02_06_REGISTER_BEH_SKILL_PROVIDER OEM Creates a Behaviour Skill in PROVIDER Mode
• E2E_02_03_01_OEM_FEDERATED_CATALOGUE OEM Checks SUPPLIER Information in Federated Catalogue
• E2E_02_03_02_OEM_CALL_REMOTE OEM Checks Successful Calling SUPPLIER Prognosis Logic
• E2E_02_03_03_OEM_CALL_REMOTE_FAIL_INVALID_GRAPH OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Wrong Address)
• E2E_02_03_04_OEM_CALL_REMOTE_FAIL_INVALID_SYNTAX OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Invalid Query)
• E2E_02_03_05_OEM_CALL_REMOTE_FAIL_NS OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Non-Matched Semantics)
• E2E_02_04_01_PROVIDER_FEDERATED_CATALOGUE PROVIDER Checks OEM Information in Federated Catalogue
• E2E_02_04_02_PROVIDER_CALL_REMOTE PROVIDER Checks Successful Calling OEM Telematics Data
• E2E_02_04_03_PROVIDER_CALL_REMOTE_FAIL_INVALID_GRAPH PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Wrong Address)
• E2E_02_04_04_PROVIDER_CALL_REMOTE_FAIL_INVALID_SYNTAX PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Invalid Query)
• E2E_02_04_05_PROVIDER_CALL_REMOTE_FAIL_NS PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Non-Matched Semantics)
• E2E_02_06_01_LIST_SKILLS PROVIDER lists all required skills for the E2E Test
• E2E_02_06_02_AAS_DESCRIPTION PROVIDER successfully lists AAS capabilties
• [E2E_02_06_03_AAS_ALL_SHELLS](https://www.postman.com/catena-x/workspace/catena-x-knowledge-agents/request/2757771-e5ffecc4-e3bf-4561-b512-4c535afa5e38?action=share&source=copy-link&creator=2757771&ctx=documentation PROVIDER successfully lists all twins inside Traceability Graph
• E2E_02_06_04_AAS_ALL_SUBMODELS PROVIDER successfully lists all submodels inside Traceability Graph
• E2E_02_06_05_ONE_SHELL PROVIDER successfully looks up single twin from the Traceability Graph
• E2E_02_06_06_NOTEXISTS_SHELL PROVIDER fails looking up non-existing twin
• E2E_02_06_07_ONE_SUBMODEL PROVIDER successfully looks up single submodel from the Traceability Graph
• E2E_02_06_07_NOTEXISTS_SUBMODEL PROVIDER fails looking up non-existing submodel

CONSUMER Test

• E2E_02_02_05_REGISTER_RUL_SKILL_CONSUMER CONSUMER Creates a Non-Federated RUL Skill
• E2E_02_05_01_RUL_SKILL_ADHOC CONSUMER Calls an Ad-hoc Skill Over OEM and SUPPLIER
• E2E_02_05_02_RUL_SKILL CONSUMER Calls Own Predefined RUL Skill
• E2E_02_05_03_RUL_SKILL_WRONG_VEHICLES CONSUMER Calls Own Predefined RUL Skill with an invalid parameterization
• E2E_02_05_04_RUL_SKILL_REMOTE_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)
• E2E_02_05_05_RUL_SKILL_REMOTE_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)
• E2E_02_05_06_RUL_SKILL_REMOTE_WRONG_MODE CONSUMER Calls PROVIDER Skill (Wrong Mode)
• E2E_02_05_07_RUL_SKILL_LOCAL_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)
• E2E_02_05_08_RUL_SKILL_LOCAL_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)
• E2E_02_05_09_RUL_SKILL_LOCAL_WRONG_MODE CONSUMER Calls PROVIDER Skill (Wrong Mode)
• E2E_02_05_10_BT_SKILL_RUL CONSUMER Calls Polymorphic OEM Skill (Variant 1)
• E2E_02_05_11_BT_SKILL_HI CONSUMER Calls Polymorphic OEM Skill (Variant 2)

Hints

Information Sharing

[drcgjung]

Integration Test Results (as of 2024-10-10)

CONSUMER+PROVIDER of a sample OEM and PROVIDER and PRODUCER

• Deploy Agent-Enabled Connector

• Perform Remote Catalogue Request (against OEM, SUPPLIER and PRODUCER)

• Check Federated Catalogue (need to list OEM and PROVIDER assets)

• Register Skill Asset

• Register Skill Policy

• Register Skill Contract

OEM for a sample SUPPLIER, access Provisioning Agents and register a graph

• Deploy Agent-Enabled Connector

• Deploy Provisioning Agent

• Perform Remote Catalogue Request (against SUPPLIER)

• Check Federated Catalogue (needs to see SUPPLIER and PROVIDER assets)

• Access Provisioning Agent (Reliability)

• Access Provisioning Agent (Supply Chain)

• Register Graph Asset(s)

• Register Graph Policy

• Register Graph Contract

SUPPLIER+PRODUCER for a sample PRODUCER/SUPPLIER and OEM, access Remoting Agent and register a graph

• Deploy Agent-Enabled Connector
• Deploy Remoting Agent with RUL binding

• Perform Remote Catalogue Request (against PRODUCER and OEM)

• Check Federated Catalogue (of PRODUCER and OEM)

• Access Remoting Agent (Remaining Useful Life)

• Register Graph Asset

• Upload Graph Asset (Supply Chain)

• Register Graph Policy

• Register Graph Contract

[drcgjung]

@obalandi Deployment and Integration Tests are successfully done (see above). You could check the integration test related marks now.

[drcgjung]

E2E Tests (as of 2024-10-10)

Provider Tests

• E2E_02_00_01_OEM_GRAPH_POLICY OEM Creates a Policy for Providing Graphs and Skills to the E2E participants

• E2E_02_00_02_OEM_GRAPH_CONTRACT OEM Creates a Contract Definition for Providing Graphs and Skills to the E2E participants

• E2E_02_00_03_OEM_RELIABILITY_GRAPH OEM Creates a Graph Asset with Telematics Data

• E2E_02_01_00_SUPPLIER_GRAPH_POLICY SUPPLIER Creates a Policy for Providing Graphs to the E2E participants

• E2E_02_01_01_SUPPLIER_GRAPH_CONTRACT SUPPLIER Creates a Contract Definition for Providing Graphs to the E2E participants

• E2E_02_01_02_SUPPLIER_BEHAVIOUR_GRAPH SUPPLIER Creates a Graph Asset with RUL Prognosis

• E2E_02_01_03_SUPPLIER_BEHAVIOUR_GRAPH_VARIANT SUPPLIER Creates a Graph Asset with HI Prognosis

• E2E_02_02_01_OPEN_POLICY PROVIDER Creates a Permissive Policy

• E2E_02_02_02_OPEN_CONTRACT PROVIDER Creates a Permissive Contract Definition

• E2E_02_02_03_REGISTER_RUL_SKILL_REMOTE PROVIDER Creates a RUL Skill in PROVIDER Mode

• E2E_02_02_04_REGISTER_RUL_SKILL_LOCAL PROVIDER Creates a RUL Skill in CONSUMER Mode

• E2E_02_02_06_REGISTER_BEH_SKILL_PROVIDER OEM Creates a Behaviour Skill in PROVIDER Mode

• E2E_02_03_01_OEM_FEDERATED_CATALOGUE OEM Checks SUPPLIER Information in Federated Catalogue

• E2E_02_03_02_OEM_CALL_REMOTE OEM Checks Successful Calling SUPPLIER Prognosis Logic

• E2E_02_03_03_OEM_CALL_REMOTE_FAIL_INVALID_GRAPH OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Wrong Address)

• E2E_02_03_04_OEM_CALL_REMOTE_FAIL_INVALID_SYNTAX OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Invalid Query)

• E2E_02_03_05_OEM_CALL_REMOTE_FAIL_NS OEM Checks Successful Unsuccessful Calling SUPPLIER Prognosis Logic (Non-Matched Semantics)

• E2E_02_04_01_PROVIDER_FEDERATED_CATALOGUE PROVIDER Checks OEM Information in Federated Catalogue

• E2E_02_04_02_PROVIDER_CALL_REMOTE PROVIDER Checks Successful Calling OEM Telematics Data

• E2E_02_04_03_PROVIDER_CALL_REMOTE_FAIL_INVALID_GRAPH PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Wrong Address)

• E2E_02_04_04_PROVIDER_CALL_REMOTE_FAIL_INVALID_SYNTAX PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Invalid Query)

• E2E_02_04_05_PROVIDER_CALL_REMOTE_FAIL_NS PROVIDER Checks Successful Unsuccessful Calling OEM Telematics Data (Non-Matched Semantics)

• E2E_02_06_01_LIST_SKILLS PROVIDER lists all required skills for the E2E Test
  

• E2E_02_06_02_AAS_DESCRIPTION PROVIDER successfully lists AAS capabilties

• [E2E_02_06_03_AAS_ALL_SHELLS](https://www.postman.com/catena-x/workspace/catena-x-knowledge-agents/request/2757771-e5ffecc4-e3bf-4561-b512-4c535afa5e38?action=share&source=copy-link&creator=2757771&ctx=documentation PROVIDER successfully lists all twins inside Traceability Graph

• E2E_02_06_04_AAS_ALL_SUBMODELS PROVIDER successfully lists all submodels inside Traceability Graph

• E2E_02_06_05_ONE_SHELL PROVIDER successfully looks up single twin from the Traceability Graph

• E2E_02_06_06_NOTEXISTS_SHELL PROVIDER fails looking up non-existing twin

• E2E_02_06_07_ONE_SUBMODEL PROVIDER successfully looks up single submodel from the Traceability Graph

• E2E_02_06_07_NOTEXISTS_SUBMODEL PROVIDER fails looking up non-existing submodel

CONSUMER Test

• E2E_02_02_05_REGISTER_RUL_SKILL_CONSUMER CONSUMER Creates a Non-Federated RUL Skill

• E2E_02_05_01_RUL_SKILL_ADHOC CONSUMER Calls an Ad-hoc Skill Over OEM and SUPPLIER

• E2E_02_05_02_RUL_SKILL CONSUMER Calls Own Predefined RUL Skill

• E2E_02_05_03_RUL_SKILL_WRONG_VEHICLES CONSUMER Calls Own Predefined RUL Skill with an invalid parameterization

• E2E_02_05_04_RUL_SKILL_REMOTE_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)

• E2E_02_05_05_RUL_SKILL_REMOTE_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)

• E2E_02_05_06_RUL_SKILL_REMOTE_WRONG_MODE CONSUMER Calls PROVIDER Skill (Wrong Mode)

• E2E_02_05_07_RUL_SKILL_LOCAL_IMPLICIT CONSUMER Calls PROVIDER Skill (Implicit Mode)

• E2E_02_05_08_RUL_SKILL_LOCAL_EXPLICIT CONSUMER Calls PROVIDER Skill (Explicit Mode)

• E2E_02_05_09_RUL_SKILL_LOCAL_WRONG_MODE CONSUMER Calls PROVIDER Skill (Wrong Mode)

• E2E_02_05_10_BT_SKILL_RUL CONSUMER Calls Polymorphic OEM Skill (Variant 1)

• E2E_02_05_11_BT_SKILL_HI CONSUMER Calls Polymorphic OEM Skill (Variant 2)

[drcgjung]

@obalandi now the e2e tests also have been successfully documented.