-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* feat: updated QA landing page with summary of testing in each stack and more detail on extra testing types * fix: Add a disclaimer on performance and e2e tests * fix: added infrastructure testing references * minor descriptive wording added * feat: added documentation for contract testing in .net * fix: updated links to include API and API with CQRS * fix: PR feedback fixes * fix: correct nx unit test link to use main * fix: linting errors ---------
- Loading branch information
1 parent
65ab32c
commit 0c48f7c
Showing
6 changed files
with
443 additions
and
13 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
156 changes: 156 additions & 0 deletions
156
asciidoc/workloads/azure/backend/netcore/provider_contract_testing.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,156 @@ | ||
| --- | ||
| id: provider_contract_testing_dotnet | ||
| title: Contract Testing - .Net Provider | ||
| sidebar_label: Contract Testing | ||
| hide_title: false | ||
| hide_table_of_contents: false | ||
| description: Dotnet provider contract tests | ||
| keywords: | ||
| - dotnet api tests | ||
| - rest | ||
| - api | ||
| - contract | ||
| - pact | ||
| - pactflow | ||
| - pipeline | ||
| - dotnet | ||
| - testing | ||
| --- | ||
|
|
||
| == Contract Testing the API | ||
|
|
||
| The link:../quickstart/web_api/create_project_netcore.md[.Net WebAPI] Stack includes a Provider implementation of Bi-Directional Contract Testing to accelerate teams getting started with contract testing. | ||
|
|
||
| === What is Bi-Directional Contract Testing | ||
|
|
||
| Bi-Directional Contract Testing is a type of static contract testing where two contracts - one representing consumer expectations, and another representing the provider's capability - are compared to ensure they are compatible. | ||
|
|
||
| [quote] | ||
| Bi-Directional contract testing replaces Consumer-Driven contract testing which is a more complicated but more thorough version of contract testing. Bi-Directional contract testing allows a provider to publish their OpenAPI specification instead of a Pact contract. When the provider and/or the consumer publish their contract, PactFlow will verify that the two contracts (consumer Pact and provider OpenAPI spec) are compatible. | ||
|
|
||
| The general steps for a **Provider** in Bi-Directional contract testing are as follows: | ||
|
|
||
| 1. Create the provider contract (i.e., OpenAPI specification) either manually or generated by code | ||
| 2. (Optional) The provider contract is tested against the provider itself to ensure the API satisfies the contract | ||
| 3. The contract is published to PactFlow | ||
| 4. Run can-i-deploy to check the compatibility with its consumers | ||
|
|
||
| In .Net WebAPI Stack, the OpenAPI specification is generated based upon the code, so there is no requirement for it to include verification tests (step 2) as we know the implementation and specification match. | ||
|
|
||
| The following diagram shows an example flow of a Bi-Directional contract testing implementation with 1 provider and 1 consumer. | ||
|
|
||
| image::../../../../../../static/img/StacksQA_Bi-Directional-Contract-Testing.png[Bi-Directional Contract Testing] | ||
|
|
||
| === Getting Started | ||
|
|
||
| When creating a project using the link:../quickstart/web_api/create_project_netcore.md[WebAPI template], the `ci.yml` pipeline will include the steps required for Contract Testing as a provider. | ||
|
|
||
| Within the contract testing steps, there are a number of variables that are required. In order for the pipeline to work as expected, the variable values will need to be updated before executing the pipeline. | ||
| Variables can be found within `./build/azDevOps/azure/ci-vars.yml` | ||
|
|
||
| [source,yaml] | ||
| ---- | ||
| # Pact Contract Tests | ||
| - name: PACT_BROKER_BASE_URL | ||
| value: 'https://ensono-stacks.pactflow.io' | ||
| - name: PACTICIPANT_NAME | ||
| value: 'stacks-provider' | ||
| - name: OAS_FILE | ||
| value: ./src/simple-api/contracts/openapi-v1.yaml | ||
| ---- | ||
|
|
||
| - `PACT_BROKER_BASE_URL` needs to be updated to use a new instance of PactFlow. Details on creating a PactFlow instance can be found link:https://pactflow.io/pricing/[here]. | ||
| - `PACTICIPANT_NAME` should be the name of your application | ||
| - `OAS_FILE` is the relative path to the generated OpenAPI specification | ||
|
|
||
| In addition to these variables, the pipeline steps require a `PACT_BROKER_TOKEN`. This is a secret value and should be added to your pipeline as a secret environment variable. | ||
| You can get the value of this from within the Settings of your PactFlow broker instance. | ||
|
|
||
| === Pipeline Overview | ||
|
|
||
| ==== Publish OpenAPI specification | ||
|
|
||
| In order to publish Pacts, we need to first install the Docker image. Once we have the image, we can run the `publish-provider-contract` command. | ||
|
|
||
| [source,yaml] | ||
| ---- | ||
| - task: Bash@3 | ||
| displayName: 'Contract Tests: Pull Pact CLI Docker image' | ||
| inputs: | ||
| targetType: inline | ||
| script: | | ||
| docker pull pactfoundation/pact-cli:latest | ||
| - task: Bash@3 | ||
| displayName: 'Contract Tests: Publish OpenAPI spec to PactFlow' | ||
| inputs: | ||
| targetType: inline | ||
| script: | | ||
| docker run --rm \ | ||
| -w ${PWD} \ | ||
| -v ${PWD}:${PWD} \ | ||
| -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ | ||
| -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ | ||
| pactfoundation/pact-cli:latest \ | ||
| pactflow publish-provider-contract \ | ||
| $(OAS_FILE) \ | ||
| --provider $(PACTICIPANT_NAME) \ | ||
| --provider-app-version $(version_number) \ | ||
| --branch $(Build.SourceBranchName) \ | ||
| --verification-results $(OAS_FILE) \ | ||
| --verifier "Verification not necessary when the OpenAPI spec is generated by the API code" \ | ||
| --tag $(version_number) $(Build.SourceBranchName) \ | ||
| --content-type "application/yaml" \ | ||
| --verification-exit-code 0 \ | ||
| --verification-results-content-type "application/yaml" \ | ||
| --verification-results-format "yaml" | ||
| env: | ||
| PACTFLOW_TOKEN: $(PACT_BROKER_TOKEN) | ||
| ---- | ||
|
|
||
| When the pact is being published, attributes including the branch name, version and tags are added to the command. These all help identify contracts within the PactFlow broker. These are essential to allow consumers to target the correct version of the provider contract in PactFlow when running `can-i-deploy`. | ||
|
|
||
| ==== Execute `can-i-deploy` | ||
|
|
||
| `can-i-deploy` is a command provided by PactFlow that checks the state of relationships between consumers and providers registered in PactFlow. If the contracts are compatible, `can-i-deploy` will succeed, otherwise it will return a failure. | ||
|
|
||
| [source,yaml] | ||
| ---- | ||
| - task: Bash@3 | ||
| displayName: 'Contract Tests: can-i-deploy to dev' | ||
| inputs: | ||
| targetType: inline | ||
| script: | | ||
| docker run --rm \ | ||
| -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ | ||
| -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ | ||
| pactfoundation/pact-cli:latest \ | ||
| broker can-i-deploy \ | ||
| --pacticipant $(PACTICIPANT_NAME) \ | ||
| --version $(version_number) \ | ||
| --to-environment $(Environment.ShortName) | ||
| ---- | ||
|
|
||
| Arguments provided identify which provider and version of the contract is being validated against the registered consumer contracts. This will check the against contracts on the environment we are trying to deploy to (set in `--to-environment`). | ||
|
|
||
| ==== Register a deployment | ||
|
|
||
| After we have deployed our API (with a new OpenAPI specification version), we need to let PactFlow know what environment that specification has been deployed to. This is essential when **consumers** are running `can-i-deploy` as they will need to confirm when they deploy to that environment, the contracts are compatible. | ||
|
|
||
| [source,yaml] | ||
| ---- | ||
| - task: Bash@3 | ||
| displayName: 'Contract Tests: Record-deployment to dev' | ||
| condition: succeeded() | ||
| inputs: | ||
| targetType: inline | ||
| script: | | ||
| docker run --rm \ | ||
| -e PACT_BROKER_BASE_URL=$(PACT_BROKER_BASE_URL) \ | ||
| -e PACT_BROKER_TOKEN=$(PACT_BROKER_TOKEN) \ | ||
| pactfoundation/pact-cli:latest \ | ||
| broker record-deployment \ | ||
| --pacticipant $(PACTICIPANT_NAME) \ | ||
| --version $(version_number) \ | ||
| --environment $(Environment.ShortName) | ||
| ---- |
Oops, something went wrong.