diff --git a/asciidoc/testing/testing_overview.adoc b/asciidoc/testing/testing_overview.adoc index a48efeab3..ff53f33cd 100644 --- a/asciidoc/testing/testing_overview.adoc +++ b/asciidoc/testing/testing_overview.adoc @@ -8,15 +8,51 @@ keywords: testing, QA, Ensono Stacks, automation, performance, accessibility, vi Building testing frameworks from scratch requires a lot of boilerplate code. It is also hard to know how to get started without examples. -To ensure the quality and reliability of your software projects, Ensono Stacks includes a range of integrated testing frameworks. These frameworks are designed to be easily bootstrapped and customized to fit your specific project needs. +To ensure the quality and reliability of your software projects, Ensono Stacks includes a range of integrated testing frameworks. These frameworks are designed to be easily bootstrapped and customized to fit your specific project needs. The tests integrated within each stack is summarised in this table: + +|=== +| Testing Type | .Net | Java | Frontend/NX | Data +| Unit | ✔ | ✔ | ✔ | ✔ +| Component | ✔ | ✔ | N/A | ✖ +| Integration | ✔ | ✔ | N/A | ✖ +| Contract | ✖ | ✔ | ✖ | N/A +| Data Validation | N/A | N/A | N/A | ✔ +| Functional | ✔ | ✔ | ✔ | ✖ +| End-to-end | N/A* | N/A* | ✔ | ✔ +| Linting | ✔ | ✔ | ✔ | ✔ +| Vulnerability Scanning | ✔ | ✔ | ✔ | ✔ +| Static code analysis | ✔ | ✔ | ✔ | ✔ +| Security | ✔ | ✔ | ✔ | ✔ +| Performance* | ✖ | ✖ | ✖ | ✖ +|=== + +`*` _Tests are available as a standalone solution that can be applied to the Stack._ == Types of Testing -This summary outlines key areas of testing included in the Ensono Stacks. +This summary outlines key areas of testing integrated within Ensono Stacks as well as some standalone solutions for testing Stacks applications. + +=== Unit Testing + +All of our Stacks come with Unit testing built in. This ensures that each individual unit (function/method) works in isolation. +Examples: + +- link:https://github.com/Ensono/stacks-java/blob/master/java/src/test/java/com/amido/stacks/workloads/menu/api/v1/CategoryControllerTest.java[Java stack] +- link:https://github.com/Ensono/stacks-dotnet/tree/master/src/cqrs/src/api/xxENSONOxx.xxSTACKSxx.API.UnitTests[DotNet stack] +- link:https://github.com/Ensono/stacks-data/tree/main/tests/unit[Data stack] +- link:https://github.com/Ensono/stacks-nx-plugins/blob/main/packages/common/test/src/lib/stacks-attributes.spec.ts[NX plugins] + +=== Component Testing + +Our .Net and Java Stacks come with Component testing built in. Components tests take the individual component (I.e. Service bus implementation, API, Function App, etc.) and test it as a whole in a white box format. All external dependencies are mocked. +This allows us to ensure that, before deployment, our component is functioning as expected in isolation, reducing the risk of deployment and liklihood that post-deployment issues are due to this components functionality. +Examples: +- link:https://github.com/Ensono/stacks-java/blob/master/java/src/test/java/com/amido/stacks/workloads/actuator/ActuatorTest.java[Java stack] +- link:https://github.com/Ensono/stacks-dotnet/tree/master/src/cqrs/src/api/xxENSONOxx.xxSTACKSxx.API.ComponentTests[DotNet stack] === Performance Testing -Performance testing ensures that your application can handle the expected load and perform well under stress. Ensono Stacks uses https://gatling.io/[Gatling] for performance testing. Further information can be found link:./performance_testing_gatling.md[here]. +Performance testing ensures that your application can handle the expected load and perform well under stress. Ensono Stacks uses link:https://gatling.io/[Gatling] for performance testing. Further information can be found link:./performance_testing_gatling.md[here]. === UI Testing @@ -41,28 +77,44 @@ The Ensono Stacks Java includes an API testing framework using Serenity BDD and ==== Contract Testing with Pact -Contract testing ensures that the interactions between services are working as expected. Ensono Stacks uses https://docs.pact.io/[Pact] for consumer-driven contract testing. For further information please refer to the link:./contract_testing_pact.md[API Contract Testing with Pact] documentation. +Contract testing ensures that the interactions between services are working as expected. Ensono Stacks uses link:https://docs.pact.io/[Pact] for consumer-driven contract testing. For further information please refer to the link:./contract_testing_pact.md[API Contract Testing with Pact] documentation. -=== ServiceBus Testing +=== Functional Testing + +==== ServiceBus Testing Ensono Stacks .NET includes comprehensive functional and unit tests for ServiceBus operations to ensure the correct handling of topics, queues, and messages. -The GitHub repository folder containing the ServiceBus unit tests can be found https://github.com/Ensono/stacks-dotnet/tree/master/src/shared/xxENSONOxx.xxSTACKSxx.Shared.Messaging.Azure.ServiceBus.Tests[here]. These tests ensure that the various components of the ServiceBus, such as validators, serializers, routers, listeners, and custom processors, function correctly and handle different scenarios as expected. +The GitHub repository folder containing the ServiceBus unit tests can be found link:https://github.com/Ensono/stacks-dotnet/tree/master/src/shared/xxENSONOxx.xxSTACKSxx.Shared.Messaging.Azure.ServiceBus.Tests[here]. These tests ensure that the various components of the ServiceBus, such as validators, serializers, routers, listeners, and custom processors, function correctly and handle different scenarios as expected. -Functional tests can be found in the GitHub repository folder https://github.com/Ensono/stacks-dotnet/tree/master/src/func-cosmosdb-worker/src/tests/Functional/xxENSONOxx.xxSTACKSxx.Worker.FunctionalTests/Tests[here]. Key areas of coverage include: +Functional tests can be found in the GitHub repository folder link:https://github.com/Ensono/stacks-dotnet/tree/master/src/func-cosmosdb-worker/src/tests/Functional/xxENSONOxx.xxSTACKSxx.Worker.FunctionalTests/Tests[here]. Key areas of coverage include: - Adding messages to the queue - Checking the existence of topics - Reading messages from the queue - Clearing messages from the queue -=== CosmosDB Testing +==== CosmosDB Testing Ensono Stacks .NET includes unit and integration tests for CosmosDB operations to ensure correct data handling and repository functionality. These include: - Unit tests for verifying the repository implementation and its functionality using a mock repository. The unit test example are in the CosmosDbMenuRepositoryTests.cs in this repository link:https://github.com/Ensono/stacks-dotnet/blob/master/src/cqrs/src/api/xxENSONOxx.xxSTACKSxx.Infrastructure.UnitTests/[folder]. - Integration tests for CosmosDB Document Storage, validating communication with CosmosDB, data encoding/decoding, and various search operations, including tests for single/multiple fields, ordering, partition keys, and SQL queries. The integration tests are in the repository folder link:https://github.com/Ensono/stacks-dotnet/tree/master/src/cqrs/src/api/xxENSONOxx.xxSTACKSxx.Infrastructure.IntegrationTests/CosmosDb/Integration[here]. +=== SonarCloud + +Ensono Stacks integrates with SonarCloud for our Static Code Analysis. This analysis includes scans for: +- Security +- Reliability +- Maintainability +- Test coverage +- Duplicated code +- Coding standards + +=== Infrastructure Testing + +Within the AKS infrastructure, there is a suite of link:https://github.com/Ensono/stacks-infrastructure-aks/tree/master/deploy/tests[infrastructure tests] using the tool link:https://github.com/inspec/inspec[InSpec]. These tests check the expected state of resources deployed into Azure to ensure the quality of the infrastructure that our Stacks applications are deploy to. + == Getting Started To get started with testing in Ensono Stacks, refer to the detailed documentation for each testing tool and framework. Each guide provides setup instructions, configuration details, and example tests to help you integrate testing into your development workflow. diff --git a/asciidoc/workloads/azure/backend/netcore/provider_contract_testing.adoc b/asciidoc/workloads/azure/backend/netcore/provider_contract_testing.adoc new file mode 100644 index 000000000..1a111d6d7 --- /dev/null +++ b/asciidoc/workloads/azure/backend/netcore/provider_contract_testing.adoc @@ -0,0 +1,156 @@ +--- +id: provider_contract_testing_dotnet +title: Contract Testing - .Net Provider +sidebar_label: Contract Testing +hide_title: false +hide_table_of_contents: false +description: Dotnet provider contract tests +keywords: + - dotnet api tests + - rest + - api + - contract + - pact + - pactflow + - pipeline + - dotnet + - testing +--- + +== Contract Testing the API + +The link:../quickstart/web_api/create_project_netcore.md[.Net WebAPI] Stack includes a Provider implementation of Bi-Directional Contract Testing to accelerate teams getting started with contract testing. + +=== What is Bi-Directional Contract Testing + +Bi-Directional Contract Testing is a type of static contract testing where two contracts - one representing consumer expectations, and another representing the provider's capability - are compared to ensure they are compatible. + +[quote] +Bi-Directional contract testing replaces Consumer-Driven contract testing which is a more complicated but more thorough version of contract testing. Bi-Directional contract testing allows a provider to publish their OpenAPI specification instead of a Pact contract. When the provider and/or the consumer publish their contract, PactFlow will verify that the two contracts (consumer Pact and provider OpenAPI spec) are compatible. + +The general steps for a **Provider** in Bi-Directional contract testing are as follows: + +1. Create the provider contract (i.e., OpenAPI specification) either manually or generated by code +2. (Optional) The provider contract is tested against the provider itself to ensure the API satisfies the contract +3. The contract is published to PactFlow +4. Run can-i-deploy to check the compatibility with its consumers + +In .Net WebAPI Stack, the OpenAPI specification is generated based upon the code, so there is no requirement for it to include verification tests (step 2) as we know the implementation and specification match. + +The following diagram shows an example flow of a Bi-Directional contract testing implementation with 1 provider and 1 consumer. + +image::../../../../../../static/img/StacksQA_Bi-Directional-Contract-Testing.png[Bi-Directional Contract Testing] + +=== Getting Started + +When creating a project using the link:../quickstart/web_api/create_project_netcore.md[WebAPI template], the `ci.yml` pipeline will include the steps required for Contract Testing as a provider. + +Within the contract testing steps, there are a number of variables that are required. In order for the pipeline to work as expected, the variable values will need to be updated before executing the pipeline. +Variables can be found within `./build/azDevOps/azure/ci-vars.yml` + +[source,yaml] +---- + # Pact Contract Tests + - name: PACT_BROKER_BASE_URL + value: 'https://ensono-stacks.pactflow.io' + - name: PACTICIPANT_NAME + value: 'stacks-provider' + - name: OAS_FILE + value: ./src/simple-api/contracts/openapi-v1.yaml +---- + +- `PACT_BROKER_BASE_URL` needs to be updated to use a new instance of PactFlow. Details on creating a PactFlow instance can be found link:https://pactflow.io/pricing/[here]. +- `PACTICIPANT_NAME` should be the name of your application +- `OAS_FILE` is the relative path to the generated OpenAPI specification + +In addition to these variables, the pipeline steps require a `PACT_BROKER_TOKEN`. This is a secret value and should be added to your pipeline as a secret environment variable. +You can get the value of this from within the Settings of your PactFlow broker instance. + +=== Pipeline Overview + +==== Publish OpenAPI specification + +In order to publish Pacts, we need to first install the Docker image. Once we have the image, we can run the `publish-provider-contract` command. + +[source,yaml] +---- + - task: Bash@3 + displayName: 'Contract Tests: Pull Pact CLI Docker image' + inputs: + targetType: inline + script: | + docker pull pactfoundation/pact-cli:latest + + - task: Bash@3 + displayName: 'Contract Tests: Publish OpenAPI spec to PactFlow' + inputs: + targetType: inline + script: | + docker run --rm \ + -w ${PWD} \ + -v ${PWD}:${PWD} \ + -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ + -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ + pactfoundation/pact-cli:latest \ + pactflow publish-provider-contract \ + $(OAS_FILE) \ + --provider $(PACTICIPANT_NAME) \ + --provider-app-version $(version_number) \ + --branch $(Build.SourceBranchName) \ + --verification-results $(OAS_FILE) \ + --verifier "Verification not necessary when the OpenAPI spec is generated by the API code" \ + --tag $(version_number) $(Build.SourceBranchName) \ + --content-type "application/yaml" \ + --verification-exit-code 0 \ + --verification-results-content-type "application/yaml" \ + --verification-results-format "yaml" + env: + PACTFLOW_TOKEN: $(PACT_BROKER_TOKEN) +---- + +When the pact is being published, attributes including the branch name, version and tags are added to the command. These all help identify contracts within the PactFlow broker. These are essential to allow consumers to target the correct version of the provider contract in PactFlow when running `can-i-deploy`. + +==== Execute `can-i-deploy` + +`can-i-deploy` is a command provided by PactFlow that checks the state of relationships between consumers and providers registered in PactFlow. If the contracts are compatible, `can-i-deploy` will succeed, otherwise it will return a failure. + +[source,yaml] +---- + - task: Bash@3 + displayName: 'Contract Tests: can-i-deploy to dev' + inputs: + targetType: inline + script: | + docker run --rm \ + -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ + -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ + pactfoundation/pact-cli:latest \ + broker can-i-deploy \ + --pacticipant $(PACTICIPANT_NAME) \ + --version $(version_number) \ + --to-environment $(Environment.ShortName) +---- + +Arguments provided identify which provider and version of the contract is being validated against the registered consumer contracts. This will check the against contracts on the environment we are trying to deploy to (set in `--to-environment`). + +==== Register a deployment + +After we have deployed our API (with a new OpenAPI specification version), we need to let PactFlow know what environment that specification has been deployed to. This is essential when **consumers** are running `can-i-deploy` as they will need to confirm when they deploy to that environment, the contracts are compatible. + +[source,yaml] +---- + - task: Bash@3 + displayName: 'Contract Tests: Record-deployment to dev' + condition: succeeded() + inputs: + targetType: inline + script: | + docker run --rm \ + -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ + -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ + pactfoundation/pact-cli:latest \ + broker record-deployment \ + --pacticipant $(PACTICIPANT_NAME) \ + --version $(version_number) \ + --environment $(Environment.ShortName) +---- \ No newline at end of file diff --git a/docs/testing/testing_overview.md b/docs/testing/testing_overview.md index f4b2a0351..ec6297386 100644 --- a/docs/testing/testing_overview.md +++ b/docs/testing/testing_overview.md @@ -16,11 +16,47 @@ keywords: Building testing frameworks from scratch requires a lot of boilerplate code. It is also hard to know how to get started without examples. -To ensure the quality and reliability of your software projects, Ensono Stacks includes a range of integrated testing frameworks. These frameworks are designed to be easily bootstrapped and customized to fit your specific project needs. +To ensure the quality and reliability of your software projects, Ensono Stacks includes a range of integrated testing frameworks. These frameworks are designed to be easily bootstrapped and customized to fit your specific project needs. The tests integrated within each stack is summarised in this table: +| Testing Type | .Net | Java | Frontend/NX | Data | +|------------------------|------|------|-------------|------| +| Unit | ✔ | ✔ | ✔ | ✔ | +| Component | ✔ | ✔ | N/A | ✖ | +| Integration | ✔ | ✔ | N/A | ✖ | +| Contract | ✔ | ✔ | ✖ | N/A | +| Data Validation | N/A | N/A | N/A | ✔ | +| Functional | ✔ | ✔ | ✔ | ✖ | +| End-to-end | N/A* | N/A*| ✔ | ✔ | +| Linting | ✔ | ✔ | ✔ | ✔ | +| Vulnerability Scanning | ✔ | ✔ | ✔ | ✔ | +| Static code analysis | ✔ | ✔ | ✔ | ✔ | +| Security | ✔ | ✔ | ✔ | ✔ | +| Performance* | ✖ | ✖ | ✖ | ✖ | + +`*` _Tests are available as a standalone solution that can be applied to the Stack._ ## Types of Testing -This summary outlines key areas of testing included in the Ensono Stacks. +This summary outlines key areas of testing integrated within Ensono Stacks as well as some standalone solutions for testing Stacks applications. + +### Unit Testing + +All of our Stacks come with Unit testing built in. This ensures that each individual unit (function/method) works in isolation. +Examples: + +- [Java stack](https://github.com/Ensono/stacks-java/blob/master/java/src/test/java/com/amido/stacks/workloads/menu/api/v1/CategoryControllerTest.java) +- [DotNet stack](https://github.com/Ensono/stacks-dotnet/tree/master/src/cqrs/src/api/xxENSONOxx.xxSTACKSxx.API.UnitTests) +- [Data stack](https://github.com/Ensono/stacks-data/tree/main/tests/unit) +- [NX plugins](https://github.com/Ensono/stacks-nx-plugins/blob/main/packages/common/test/src/lib/stacks-attributes.spec.ts) + +### Component Testing + +Our .Net and Java Stacks come with Component testing built in. Components tests take the individual component (I.e. Service bus implementation, API, Function App, etc.) and test it as a whole in a white box format. All external dependencies are mocked. +This allows us to ensure that, before deployment, our component is functioning as expected in isolation, reducing the risk of deployment and liklihood that post-deployment issues are due to this components functionality. +Examples: + +- [Java stack](https://github.com/Ensono/stacks-java/blob/master/java/src/test/java/com/amido/stacks/workloads/actuator/ActuatorTest.java) +- [DotNet stack](https://github.com/Ensono/stacks-dotnet/tree/master/src/cqrs/src/api/xxENSONOxx.xxSTACKSxx.API.ComponentTests) + ### Performance Testing @@ -49,9 +85,11 @@ The Ensono Stacks Java includes an API testing framework using Serenity BDD and #### Contract Testing with Pact -Contract testing ensures that the interactions between services are working as expected. Ensono Stacks uses [Pact](https://docs.pact.io/) for consumer-driven contract testing. For further information please refer to the [API Contract Testing with Pact](./contract_testing_pact.md) documentation. +Contract testing ensures that the interactions between services are working as expected. Ensono Stacks uses [Pact](https://docs.pact.io/) for Bi-Directional contract testing. For further information please refer to the [API Contract Testing with Pact](./contract_testing_pact.md) documentation. + +### Functional Testing -### ServiceBus Testing +#### ServiceBus Testing Ensono Stacks .NET includes comprehensive functional and unit tests for ServiceBus operations to ensure the correct handling of topics, queues, and messages. @@ -64,7 +102,7 @@ Functional tests can be found in the GitHub repository folder [here](https://git - Reading messages from the queue - Clearing messages from the queue -### CosmosDB Testing +#### CosmosDB Testing Ensono Stacks .NET includes unit and integration tests for CosmosDB operations to ensure correct data handling and repository functionality. These include: @@ -72,6 +110,20 @@ Ensono Stacks .NET includes unit and integration tests for CosmosDB operations t - Integration tests for CosmosDB Document Storage, validating communication with CosmosDB, data encoding/decoding, and various search operations, including tests for single/multiple fields, ordering, partition keys, and SQL queries. The integration tests are in the repository folder [here](https://github.com/Ensono/stacks-dotnet/tree/master/src/cqrs/src/api/xxENSONOxx.xxSTACKSxx.Infrastructure.IntegrationTests/CosmosDb/Integration). +### SonarCloud + +Ensono Stacks integrates with SonarCloud for our Static Code Analysis. This analysis includes scans for: + +- Security +- Reliability +- Maintainability +- Test coverage +- Duplicated code +- Coding standards + +### Infrastructure Testing + +Within the AKS infrastructure, there is a suite of [infrastructure tests](https://github.com/Ensono/stacks-infrastructure-aks/tree/master/deploy/tests) using the tool [InSpec](https://github.com/inspec/inspec). These tests check the expected state of resources deployed into Azure to ensure the quality of the infrastructure that our Stacks applications are deploy to. ## Getting Started diff --git a/docs/workloads/azure/backend/netcore/testing/provider_contract_testing.md b/docs/workloads/azure/backend/netcore/testing/provider_contract_testing.md new file mode 100644 index 000000000..3d075e4e9 --- /dev/null +++ b/docs/workloads/azure/backend/netcore/testing/provider_contract_testing.md @@ -0,0 +1,169 @@ +--- +id: provider_contract_testing_dotnet +title: Contract Testing - .Net Provider +sidebar_label: Contract Testing +hide_title: false +hide_table_of_contents: false +description: Dotnet provider contract tests +keywords: + - dotnet api tests + - rest + - api + - contract + - pact + - pactflow + - pipeline + - dotnet + - testing +--- + + +## Contract Testing the API + +The [Ensono.Stacks.Templates](../../netcore/introduction_netcore) included templates for a [WebAPI](../../netcore/quickstart/web_api/create_project_netcore) and a [WebAPI with CQRS](../../netcore/quickstart/web_api_cqrs/create_project_netcore). Each of these templates will include a Provider implementation of Bi-Directional Contract Testing to accelerate teams getting started with contract testing. + +### What is Bi-Directional Contract Testing + +Bi-Directional Contract Testing is a type of static contract testing where two contracts - one representing consumer expectations, and another representing the provider's capability - are compared to ensure they are compatible. + +> Bi-Directional contract testing replaces Consumer-Driven contract testing which is a more complicated but more thorough version of contract testing. Bi-Directional contract testing allows a provider to publish their OpenAPI specification instead of a Pact contract. When the provider and/or the consumer publish their contract, PactFlow will verify that the two contracts (consumer Pact and provider OpenAPI spec) are compatible. + +The general steps for a **Provider** in Bi-Directional contract testing are as follows: + +1. Create the provider contract (i.e. OpenAPI specification) either manually or generated by code +2. (Optional) The provider contract is tested against the provider itself to ensure the API satisfies the contract +3. The contract is published to PactFlow +4. Run can-i-deploy to check the compatibility with its consumers + +In .Net WebAPI Stack, the OpenAPI specification is generated based upon the code, so there is no requirement for it to include verification tests (step 2) as we know the implementation and specification match. + +The following diagram shows an example flow of a Bi-Directional contract testing implementation with 1 provider and 1 consumer. + +![Bi-Directional Contract Testing](../../../../../../static/img/StacksQA_Bi-Directional-Contract-Testing.png) + + +### Getting Started + +When creating an API project using the [Ensono.Stacks.Templates](../../netcore/introduction_netcore), the created `ci.yml` pipeline will include the steps required for Contract Testing as a provider. + +Within the contract testing steps, there are a number of variables that are required. In order for the pipeline to work as expected, the variable values will need to be updated before executing the pipeline. +Variables can be found within `./build/azDevOps/azure/ci-vars.yml` + +```yaml + # Pact Contract Tests + - name: PACT_BROKER_BASE_URL + value: 'https://ensono-stacks.pactflow.io' + - name: PACTICIPANT_NAME + value: 'stacks-provider' + - name: OAS_FILE + value: ./src/simple-api/contracts/openapi-v1.yaml + - name: run_contract_tests + value: flag +``` + +- `PACT_BROKER_BASE_URL` needs to be updated to use a new instance of PactFlow. Details on creating a PactFlow instance can be found [here](https://pactflow.io/pricing/). +- `PACTICIPANT_NAME` should be the name of your application +- `OAS_FILE` is the relative path to the generated OpenAPI specification +- `run_contract_tests` is a flag to enable/disable all contract testing steps. This is false by default and should only be set to true once users have a new PactFlow instance. + +In addition to these variables, the pipeline steps require a `PACT_BROKER_TOKEN`. This is a secret value and should be added to your pipeline as a secret environment variable. +You can get the value of this from within the Settings of your PactFlow broker instance. + +### Pipeline Overview + +#### Publish OpenAPI specification + +In order to publish Pacts, we need to first install the Docker image. Once we have the image, we can run the `publish-provider-contract` command. + +```yaml + - task: Bash@3 + displayName: 'Contract Tests: Pull Pact CLI Docker image' + condition: eq(variables.run_contract_test, true) + inputs: + targetType: inline + script: | + docker pull pactfoundation/pact-cli:latest + + - task: Bash@3 + displayName: 'Contract Tests: Publish OpenAPI spec to PactFlow' + condition: eq(variables.run_contract_test, true) + inputs: + targetType: inline + script: | + docker run --rm \ + -w ${PWD} \ + -v ${PWD}:${PWD} \ + -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ + -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ + pactfoundation/pact-cli:latest \ + pactflow publish-provider-contract \ + $(OAS_FILE) \ + --provider $(PACTICIPANT_NAME) \ + --provider-app-version $(version_number) \ + --branch $(Build.SourceBranchName) \ + --verification-results $(OAS_FILE) \ + --verifier "Verification not necessary when the OpenAPI spec is generated by the API code" \ + --tag $(version_number) $(Build.SourceBranchName) \ + --content-type "application/yaml" \ + --verification-exit-code 0 \ + --verification-results-content-type "application/yaml" \ + --verification-results-format "yaml" + env: + PACTFLOW_TOKEN: $(PACT_BROKER_TOKEN) +``` + +When the pact is being published, attributes including the branch name, version and tags are added to the command. These all help identify contracts within the PactFlow broker. These are essential to allow consumers to target the correct version of the provider contract in PactFlow when running `can-i-deploy`. + +#### Execute `can-i-deploy` + +`can-i-deploy` is a command provided by PactFlow that checks the state of relationships between consumers and providers registered in PactFlow. If the contracts are compatible, `can-i-deploy` will succeed, otherwise it will return a failure. +The docker image needs to be pulled here again as this is executed on a new build agent within Azure DevOps. + +```yaml + - job: canideploy_dev + condition: eq(variables.run_contract_test, true) + steps: + - task: Bash@3 + displayName: 'Pull Pact CLI Docker image' + inputs: + targetType: inline + script: | + docker pull pactfoundation/pact-cli:latest + + - task: Bash@3 + displayName: 'Contract Tests: can-i-deploy to dev' + inputs: + targetType: inline + script: | + docker run --rm \ + -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ + -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ + pactfoundation/pact-cli:latest \ + broker can-i-deploy \ + --pacticipant $(PACTICIPANT_NAME) \ + --version $(version_number) \ + --to-environment $(Environment.ShortName) +``` + +Arguments provided identify which provider and version of the contract is being validated against the registered consumer contracts. This will check the against contracts on the environment we are trying to deploy to (set in `--to-environment`). + +#### Register a deployment + +After we have deployed our API (with a new OpenAPI specification version), we need to let PactFlow know what environment that specification has been deployed to. This is essential when **consumers** are running `can-i-deploy` as they will need to confirm when they deploy to that environment, the contracts are compatible. + +```yaml + - task: Bash@3 + displayName: 'Contract Tests: Record-deployment to dev' + condition: and(succeeded(), eq(variables.run_contract_test, true)) + inputs: + targetType: inline + script: | + docker run --rm \ + -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ + -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ + pactfoundation/pact-cli:latest \ + broker record-deployment \ + --pacticipant $(PACTICIPANT_NAME) \ + --version $(version_number) \ + --environment $(Environment.ShortName) +``` diff --git a/sidebars.js b/sidebars.js index 417ef10e5..27a8c95ac 100644 --- a/sidebars.js +++ b/sidebars.js @@ -213,6 +213,7 @@ module.exports = { ], }, "workloads/azure/backend/netcore/testing/functional_testing_netcore", + "workloads/azure/backend/netcore/testing/provider_contract_testing_dotnet", "workloads/azure/backend/netcore/infrastructure_netcore", "workloads/azure/backend/netcore/pipeline_netcore", "workloads/azure/backend/netcore/infrastructure_aws_netcore", diff --git a/static/img/StacksQA_Bi-Directional-Contract-Testing.png b/static/img/StacksQA_Bi-Directional-Contract-Testing.png new file mode 100644 index 000000000..d29edbd89 Binary files /dev/null and b/static/img/StacksQA_Bi-Directional-Contract-Testing.png differ