This workshop is aimed at demonstrating core features and benefits of contract testing with Pact.
Whilst contract testing can be applied retrospectively to systems, we will follow the consumer driven contracts approach in this workshop - where a new consumer and provider are created in parallel to evolve a service over time, especially where there is some uncertainty with what is to be built.
This workshop should take from 1 to 2 hours, depending on how deep you want to go into each topic.
Workshop outline:
- step 1: create consumer: Create our consumer before the Provider API even exists
- step 2: unit test: Write a unit test for our consumer
- step 3: pact test: Write a Pact test for our consumer
- step 4: pact verification: Verify the consumer pact with the Provider API
- step 5: fix consumer: Fix the consumer's bad assumptions about the Provider
- step 6: pact test: Write a pact test for
404(missing User) in consumer - step 7: provider states: Update API to handle
404case - step 8: pact test: Write a pact test for the
401case - step 9: pact test: Update API to handle
401case - step 10: request filters: Fix the provider to support the
401case - step 11: pact broker: Implement a broker workflow for integration with CI/CD
NOTE: Each step is tied to, and must be run within, a git branch, allowing you to progress through each stage incrementally. For example, to move to step 2 run the following: git checkout step2
If running this as a team workshop format, you may want to take a look through the learning objectives.
There are two components in scope for our workshop.
- Admin Service (Consumer). Does Admin-y things, and often needs to communicate to the User service. But really, it's just a placeholder for a more useful consumer (e.g. a website or another microservice) - it doesn't do much!
- User Service (Provider). Provides useful things about a user, such as listing all users and getting the details of individuals.
For the purposes of this workshop, we won't implement any functionality of the Admin Service, except the bits that require User information.
Project Structure
The key packages are shown below:
├── consumer # Contains the Admin Service Team (client) project
├── model # Shared domain model
├── pact # The directory of the Pact Standalone CLI
├── provider # The User Service Team (provider) projectWe need to first create an HTTP client to make the calls to our provider service:
NOTE: even if the API client had been been graciously provided for us by our Provider Team, it doesn't mean that we shouldn't write contract tests - because the version of the client we have may not always be in sync with the deployed API - and also because we will write tests on the output appropriate to our specific needs.
This User Service expects a user path parameter, and then returns some simple json back:
You can see the client public interface we created in the consumer/client package:
type Client struct {
BaseURL *url.URL
httpClient *http.Client
}
// GetUser gets a single user from the API
func (c *Client) GetUser(id int) (*model.User, error) {
}We can run the client with make run-consumer - it should fail with an error, because the Provider is not running.
Move on to step 2: Write a unit test for our consumer
Now lets create a basic test for our API client. We're going to check 2 things:
- That our client code hit the expected endpoint
- That the response is marshalled into a
Userobject, with the correct ID
consumer/client/client_test.go
func TestClientUnit_GetUser(t *testing.T) {
userID := 10
// Setup mock server
server := httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
assert.Equal(t, req.URL.String(), fmt.Sprintf("/user/%d", userID))
user, _ := json.Marshal(model.User{
FirstName: "Sally",
LastName: "McDougall",
ID: userID,
Type: "admin",
Username: "smcdougall",
})
rw.Write([]byte(user))
}))
defer server.Close()
// Setup client
u, _ := url.Parse(server.URL)
client := &Client{
BaseURL: u,
}
user, err := client.GetUser(userID)
assert.NoError(t, err)
// Assert basic fact
assert.Equal(t, user.ID, userID)
}
Let's run this spec and see it all pass:
$ make unit
--- 🔨Running Unit tests
go test -count=1 github.com/pact-foundation/pact-workshop-go/consumer/client -run 'TestClientUnit'
ok github.com/pact-foundation/pact-workshop-go/consumer/client 10.196s
Meanwhile, our provider team has started building out their API in parallel. Let's run our client against our provider (you'll need two terminals to do this):
# Terminal 1
$ make run-provider
2019/10/28 18:24:37 API starting: port 8080 ([::]:8080)
# Terminal 2
make run-consumer
2019/10/28 18:25:57 api unavailable
exit status 1
make: *** [run-consumer] Error 1
Doh! The Provider doesn't know about /users/:id. On closer inspection, the provider only knows about /user/:id and /users.
We need to have a conversation about what the endpoint should be, but first...
Move on to step 3
Let us add Pact to the project and write a consumer pact test for the GET /users/:id endpoint. Note how similar it looks to our unit test:
consumer/client/client_pact_test.go:
t.Run("the user exists", func(t *testing.T) {
id := 10
pact.
AddInteraction().
Given("User sally exists").
UponReceiving("A request to login with user 'sally'").
WithRequest(request{
Method: "GET",
Path: term("/users/10", "/user/[0-9]+"),
Headers: headersWithToken,
}).
WillRespondWith(dsl.Response{
Status: 200,
Body: dsl.Match(model.User{}),
Headers: commonHeaders,
})
err := pact.Verify(func() error {
user, err := client.WithToken("2019-01-01").GetUser(id)
// Assert basic fact
if user.ID != id {
return fmt.Errorf("wanted user with ID %d but got %d", id, user.ID)
}
return err
})
if err != nil {
t.Fatalf("Error on Verify: %v", err)
}
})
This test starts a mock server a random port that acts as our provider service. To get this to work we update the URL in the Client that we create, after initialising Pact.
Running this test still passes, but it creates a pact file which we can use to validate our assumptions on the provider side, and have conversation around.
$ make consumerA pact file should have been generated in pacts/goadminservice-gouserservice.json
Move on to step 4
We now need to validate the pact generated by the consumer is valid, by executing it against the running service provider, which should fail:
$ make provider
--- 🔨Running Provider Pact tests
go test -count=1 -tags=integration github.com/pact-foundation/pact-workshop-go/provider -run "TestPactProvider"
2019/10/30 11:29:49 API starting: port 62059 ([::]:62059)
--- FAIL: TestPactProvider (11.30s)
pact.go:416: Verifying a pact between GoAdminService and GoUserService Given User sally exists A request to login with user 'sally' with GET /users/10 returns a response which has a matching body
Actual: [{"firstName":"Jean-Marie","lastName":"de La Beaujardière😀😍","username":"sally","type":"admin","id":10}]
Diff
--------------------------------------
Key: - is expected
+ is actual
Matching keys and values are not shown
-{
- "firstName": "Sally",
- "id": 10,
- "lastName": "McSmiley Face😀😍",
- "type": "admin",
- "username": "sally"
-}
+[
+ {
+ "firstName": "Jean-Marie",
+ "lastName": "de La Beaujardière😀😍",
+ "username": "sally",
+ "type": "admin",
+ "id": 10
+ },
+]
Description of differences
--------------------------------------
* Expected a Hash (like {"firstName"=>"Sally", "id"=>10, "lastName"=>"McSmiley Face😀😍", "type"=>"admin", "username"=>"sally"}) but got an Array ([{"firstName"=>"Jean-Marie", "lastName"=>"de La Beaujardière😀😍", "username"=>"sally", "type"=>"admin", "id"=>10}]) at $
user_service_test.go:43: error verifying provider: exit status 1The test has failed, as the expected path /users/:id is actually triggering the /users endpoint (which we don't need), and returning a list of Users instead of a single User. We incorrectly believed our provider was following a RESTful design, but the authors were too lazy to implement a better routing solution 🤷🏻♂️.
The correct endpoint should be /user/:id.
Move on to step 5*
Let's update the consumer test and client to hit the correct path, and run the provider verification also:
$ make consumer
--- 🔨Running Consumer Pact tests
go test github.com/pact-foundation/pact-workshop-go/consumer/client -run '^TestClientPact'
ok github.com/pact-foundation/pact-workshop-go/consumer/client 21.983s
$ make provider
--- 🔨Running Provider Pact tests
go test -count=1 -tags=integration github.com/pact-foundation/pact-workshop-go/provider -run "TestPactProvider"
ok github.com/pact-foundation/pact-workshop-go/provider 22.138s
Yay - green ✅!
Move on to step 6*
We're now going to add another scenario - what happens when we make a call for a user that doesn't exist? We assume we'll get a 404, because that is the obvious thing to do.
Let's write a test for this scenario, and then generate an updated pact file.
consumer/client/client_pact_test.go:
t.Run("the user does not exist", func(t *testing.T) {
pact.
AddInteraction().
Given("User sally does not exist").
UponReceiving("A request to login with user 'sally'").
WithRequest(request{
Method: "GET",
Path: term("/user/10", "/user/[0-9]+"),
Headers: headersWithToken,
}).
WillRespondWith(dsl.Response{
Status: 404,
Headers: commonHeaders,
})
err := pact.Verify(func() error {
_, err := client.WithToken("2019-01-01").GetUser(10)
return err
})
assert.Equal(t, ErrNotFound, err)
})Notice that our new test looks almost identical to our previous test, and only differs on the expectations of the response - the HTTP request expectations are exactly the same.
$ make consumer
go test github.com/pact-foundation/pact-workshop-go/consumer/client -run '^TestClientPact'
ok github.com/pact-foundation/pact-workshop-go/consumer/client 21.983s
What does our provider have to say about this new test:
--- 🔨Running Provider Pact tests
go test -count=1 -tags=integration github.com/pact-foundation/pact-workshop-go/provider -run "TestPactProvider"
2019/10/30 13:46:32 API starting: port 64046 ([::]:64046)
--- FAIL: TestPactProvider (11.56s)
pact.go:416: Verifying a pact between GoAdminService and GoUserService Given User sally does not exist A request to login with user 'sally' with GET /user/10 returns a response which has status code 404
expected: 404
got: 200
(compared using eql?)
user_service_test.go:43: error verifying provider: exit status 1
We expected this failure, because the user we are requesing does in fact exist! What we want to test for, is what happens if there is a different state on the Provider. This is what is referred to as "Provider states", and how Pact gets around test ordering and related issues.
We could resolve this by updating our consumer test to use a known non-existent User, but it's worth understanding how Provider states work more generally.
Move on to step 7
Our code already deals with missing users and sends a 404 response, however our test data fixture always has Sally (user 10) in our database.
In this step, we will add a state handler (StateHandlers) to our Pact tests, which will update the state of our data store depending on which states.
States are invoked prior to the actual test function is invoked. You can see the full lifecycle here.
We're going to add handlers for our two states - when Sally does and does not exist.
var stateHandlers = types.StateHandlers{
"User sally exists": func() error {
userRepository = sallyExists
return nil
},
"User sally does not exist": func() error {
userRepository = sallyDoesNotExist
return nil
},
}Let's see how we go now:
$ make provider
--- 🔨Running Provider Pact tests
go test -count=1 -tags=integration github.com/pact-foundation/pact-workshop-go/provider -run "TestPactProvider"
ok github.com/pact-foundation/pact-workshop-go/provider 22.138s
Move on to step 8
It turns out that not everyone should be able to use the API. After a discussion with the team, it was decided that a time-bound bearer token would suffice.
In the case a valid bearer token is not provided, we expect a 401. Let's update the consumer test cases to pass the bearer token, and capture this new 401 scenario.
$ make consumer
--- 🔨Running Consumer Pact tests
go test github.com/pact-foundation/pact-workshop-go/consumer/client -run '^TestClientPact'
ok github.com/pact-foundation/pact-workshop-go/consumer/client 21.983s
We should now have two interactions in our pact file.
Our verification now fails, as our consumer is sending a Bearer token that is not yet understood by our provider.
$ make provider
--- 🔨Running Provider Pact tests
go test -count=1 -tags=integration github.com/pact-foundation/pact-workshop-go/provider -run "TestPactProvider"
2019/10/30 13:28:47 API starting: port 63875 ([::]:63875)
2019/10/30 13:28:59 [WARN] state handler not found for state:
--- FAIL: TestPactProvider (11.54s)
pact.go:416: Verifying a pact between GoAdminService and GoUserService A request to login with user 'sally' with GET /user/10 returns a response which has status code 401
expected: 401
got: 200
(compared using eql?)
user_service_test.go:43: error verifying provider: exit status 1
Move on to step 9
Like most tokens, our bearer token is going to be dependent on the date/time it was generated. For the purposes of our API, it's rather crude:
func getAuthToken() string {
return fmt.Sprintf("Bearer %s", time.Now().Format("2006-01-02T15:04"))
}This means that a client must present an HTTP Authorization header that looks as follows:
Authorization: Bearer 2006-01-02T15:04
We have created a small middleware to wrap our functions and return a 401:
func IsAuthenticated(h http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") == getAuthToken() {
h.ServeHTTP(w, r)
} else {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(http.StatusUnauthorized)
}
}
}Let's test this out:
$ make provider
--- 🔨Running Provider Pact tests
go test -count=1 -tags=integration github.com/pact-foundation/pact-workshop-go/provider -run "TestPactProvider"
2019/10/30 14:08:11 API starting: port 64214 ([::]:64214)
2019/10/30 14:08:22 [WARN] state handler not found for state: User is not authenticated
--- FAIL: TestPactProvider (11.55s)
pact.go:416: Verifying a pact between GoAdminService and GoUserService Given User sally exists A request to login with user 'sally' with GET /user/10 returns a response which has status code 200
expected: 200
got: 401
(compared using eql?)
pact.go:416: Verifying a pact between GoAdminService and GoUserService Given User sally exists A request to login with user 'sally' with GET /user/10 returns a response which has a matching body
757: unexpected token at 'null'
pact.go:416: Verifying a pact between GoAdminService and GoUserService Given User sally does not exist A request to login with user 'sally' with GET /user/10 returns a response which has status code 404
expected: 404
got: 401
Oh, dear. Both tests are now failing. Can you understand why?
Move on to step 10
Because our pact file has static data in it, our bearer token is now out of date, so when Pact verification passes it to the Provider we get a 401. There are multiple ways to resolve this - mocking or stubbing out the authentication component is a common one. In our use case, we are going to use a process referred to as Request Filtering, using a RequestFilter.
NOTE: This is an advanced concept and should be used carefully, as it has the potential to invalidate a contract by bypassing its constraints. See https://github.com/pact-foundation/pact-go#request-filtering for more details on this.
The approach we are going to take to inject the header is as follows:
- If we receive any Authorization header, we override the incoming request with a valid (in time) Authorization header, and continue with whatever call was being made
- If we don't recieve a header, we do nothing
NOTE: We are not considering the 403 scenario in this example.
Here is the request filter:
// Simulates the neeed to set a time-bound authorization token,
// such as an OAuth bearer token
func fixBearerToken(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Only set the correct bearer token, if one was provided in the first place
if r.Header.Get("Authorization") != "" {
r.Header.Set("Authorization", fmt.Sprintf("Bearer %s", getAuthToken()))
}
next.ServeHTTP(w, r)
})
}We can now run the Provider tests
$ make provider
--- 🔨Running Provider Pact tests
go test -count=1 -tags=integration github.com/pact-foundation/pact-workshop-go/provider -run "TestPactProvider"
ok github.com/pact-foundation/pact-workshop-go/provider 22.138s
Move on to step 11
We've been publishing our pacts from the consumer project by essentially sharing the file system with the provider. But this is not very manageable when you have multiple teams contributing to the code base, and pushing to CI. We can use a Pact Broker to do this instead.
Using a broker simplifies the management of pacts and adds a number of useful features, including some safety enhancements for continuous delivery which we'll see shortly.
First, in the consumer project we need to tell Pact about our broker. We've created a small utility to push the pact files to the broker:
$ make publish
--- 📝 Publishing Pacts
go run consumer/client/pact/publish.go
Publishing Pact files to broker /Users/matthewfellows/development/pact-workshop-go/pacts test.pact.dius.com.au
2019/10/30 15:23:09 [INFO]
2019/10/30 15:23:09 [INFO] Tagging version 1.0.0 of GoAdminService as "master"
2019/10/30 15:23:09 [INFO] Publishing GoAdminService/GoUserService pact to pact broker at https://test.pact.dius.com.au
2019/10/30 15:23:09 [INFO] The given version of pact is already published. Overwriting...
2019/10/30 15:23:09 [INFO] The latest version of this pact can be accessed at the following URL (use this to configure the provider verification):
2019/10/30 15:23:09 [INFO] https://test.pact.dius.com.au/pacts/provider/GoUserService/consumer/GoAdminService/latest
2019/10/30 15:23:09 [INFO]
2019/10/30 15:23:09 [DEBUG] response from publish <nil>
Pact contract publishing complete!
Head over to https://test.pact.dius.com.au and login with
to see your published contracts.Have a browse around the broker and see your newly published contract!
All we need to do for the provider is update where it finds its pacts, from local URLs, to one from a broker.
_, err := pact.VerifyProvider(t, types.VerifyRequest{
ProviderBaseURL: fmt.Sprintf("http://127.0.0.1:%d", port),
Tags: []string{"master"},
FailIfNoPactsFound: false,
Verbose: false,
// Use this if you want to test without the Pact Broker
// PactURLs: []string{filepath.FromSlash(fmt.Sprintf("%s/goadminservice-gouserservice.json", os.Getenv("PACT_DIR")))},
BrokerURL: fmt.Sprintf("%s://%s", os.Getenv("PACT_BROKER_PROTO"), os.Getenv("PACT_BROKER_URL")),
BrokerUsername: os.Getenv("PACT_BROKER_USERNAME"),
BrokerPassword: os.Getenv("PACT_BROKER_PASSWORD"),
PublishVerificationResults: true,
ProviderVersion: "1.0.0",
StateHandlers: stateHandlers,
RequestFilter: fixBearerToken,
})Let's run the provider verification one last time after this change:
$ make provider
--- 🔨Running Provider Pact tests
go test -count=1 -tags=integration github.com/pact-foundation/pact-workshop-go/provider -run "TestPactProvider"
ok github.com/pact-foundation/pact-workshop-go/provider 58.047s
As part of this process, the results of the verification - the outcome (boolean) and the detailed information about the failures at the interaction level - are published to the Broker also.
This is one of the Broker's more powerful features. Referred to as Verifications, it allows providers to report back the status of a verification to the broker. You'll get a quick view of the status of each consumer and provider on a nice dashboard. But, it is much more important than this!
With just a simple use of the pact-broker can-i-deploy tool - the Broker will determine if a consumer or provider is safe to release to the specified environment.
You can run the can-i-deploy checks as follows:
$ make deploy-consumer
--- ✅ Checking if we can deploy consumer
Computer says yes \o/
CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS?
---------------|-----------|---------------|-----------|---------
GoAdminService | 1.0.0 | GoUserService | 1.0.0 | true
All required verification results are published and successful
$ make deploy-provider
--- ✅ Checking if we can deploy provider
Computer says yes \o/
CONSUMER | C.VERSION | PROVIDER | P.VERSION | SUCCESS?
---------------|-----------|---------------|-----------|---------
GoAdminService | 1.0.0 | GoUserService | 1.0.0 | true
All required verification results are published and successfulThat's it - you're now a Pact pro. Go build 🔨