Skip to content

Commit

Permalink
Update test/adding_tests.md
Browse files Browse the repository at this point in the history
Update test/adding_tests.md and test/README.md

Fixes #6239
  • Loading branch information
MIBc committed Dec 27, 2019
1 parent 545c67b commit e81bbd9
Show file tree
Hide file tree
Showing 2 changed files with 63 additions and 34 deletions.
17 changes: 12 additions & 5 deletions test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,15 +160,16 @@ New test images should be placed in `./test/test_images`.
These flags are useful for running against an existing cluster, making use of
your existing [environment setup](../DEVELOPMENT.md#setup-your-environment).

Tests importing [`github.com/knative/serving/test`](#test-library) recognize
Tests importing [`knative.dev/serving/test`](#test-library) recognize
these flags:

- [All flags added by `knative/pkg/test`](https://github.com/knative/pkg/tree/master/test#flags)
- [`--dockerrepo`](#overriding-docker-repo)
- [`--tag`](#using-a-docker-tag)
- [`--ingressendpoint`](#using-a-custom-ingress-endpoint)
- [All flags added by `knative/pkg/test`](https://github.com/knative/pkg/tree/master/test#flags) such as:
- [`--dockerrepo`](#overriding-docker-repo)
- [`--tag`](#using-a-docker-tag)
- [`--ingressendpoint`](#using-a-custom-ingress-endpoint)
- [`--resolvabledomain`](#using-a-resolvable-domain)
- [`--https`](#using-https)
- [`--ingressClass`](#using-ingress-class)

### Overridding docker repo

Expand Down Expand Up @@ -230,3 +231,9 @@ directly against `Route.Status.Domain` and does not need to spoof the `Host`.
### Using https

You can use the `--https` flag to have all tests run with https.

### Using ingress class

The `--ingressClass` argument lets you specify the ingress class. The default
value is `istio.ingress.networking.knative.dev`.

80 changes: 51 additions & 29 deletions test/adding_tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ This library exists partially in this directory and partially in

The libs in this dir can:

- [Use common test flags](#use-common-test-flags)
- [Get access to client objects](#get-access-to-client-objects)
- [Make requests against deployed services](#make-requests-against-deployed-services)
- [Check Knative Serving resources](#check-knative-serving-resources)
Expand All @@ -40,6 +39,7 @@ The libs in this dir can:

See [`knative/pkg/test`](https://github.com/knative/pkg/tree/master/test) to:

- [Use common test flags](#use-common-test-flags)
- Output logs
- Emit metrics
- Ensure test cleanup
Expand All @@ -49,24 +49,31 @@ See [`knative/pkg/test`](https://github.com/knative/pkg/tree/master/test) to:
These flags are useful for running against an existing cluster, making use of
your existing [environment setup](../DEVELOPMENT.md#setup-your-environment).

By importing `github.com/knative/serving/test` you get access to a global
By importing `knative.dev/pgk/test` you get access to a global
variable called `test.Flags` which holds the values of
[the command line flags](./README.md#flags).

```go
imagePath := strings.Join([]string{test.ServingFlags.DockerRepo, image}, "/"))
imagePath := strings.Join([]string{test.Flags.DockerRepo, image}, "/"))
```

_See [e2e_flags.go](./e2e_flags.go)._
_See [e2e_flags.go](https://github.com/knative/pkg/blob/master/test/e2e_flags.go)._

### Get access to client objects

To initialize client objects that you can use
[the command line flags](#use-flags) that describe the environment:
the command line flags that describe the environment:

```go
func setup(t *testing.T) *test.Clients {
clients, err := test.NewClients(kubeconfig, cluster, namespaceName)
import (
testing

knative.dev/serving/test
pkgTest "knative.dev/pkg/test"
)

func Setup(t *testing.T) *test.Clients {
clients, err := test.NewClients(pkgTest.Flags.Kubeconfig, pkgTest.Flags.Cluster, namespaceName)
if err != nil {
t.Fatalf("Couldn't initialize clients: %v", err)
}
Expand All @@ -76,10 +83,14 @@ func setup(t *testing.T) *test.Clients {

The `Clients` struct contains initialized clients for accessing:

- Kubernetes objects
- `Kubernetes objects`
- `Services`
- `Routes`
- `Configurations`
- `Revisions`
- `Knative ingress`
- `ServerlessServices`
- `Istio objects`

For example, to create a `Route`:

Expand All @@ -91,9 +102,12 @@ And you can use the client to clean up `Route` and `Configuration` resources
created by your test:

```go
import "knative.dev/serving/test"

func tearDown(clients *test.Clients) {
if clients != nil {
clients.Delete([]string{routeName}, []string{configName})
clients.ServingClient.Routes.Delete(routeName, nil)
clients.ServingClient.Configs.Delete(configName, nil)
}
}
```
Expand All @@ -105,22 +119,22 @@ _See [clients.go](./clients.go)._
After deploying (i.e. creating a `Route` and a `Configuration`) an endpoint will
not be ready to serve requests right away. To poll a deployed endpoint and wait
for it to be in the state you want it to be in (or timeout) use
`WaitForEndpointState`:
`WaitForEndpointState` by importing `knative.dev/pkg/test` with alias `pkgTest`:

```go
err = pkgTest.WaitForEndpointState(
clients.KubeClient,
logger,
updatedRoute.Status.Domain,
pkgTest.EventuallyMatchesBody(expectedText),
"SomeDescription",
test.ServingFlags.ResolvableDomain)
_, err := pkgTest.WaitForEndpointState(
clients.KubeClient,
logger,
updatedRoute.Status.URL.URL(),
pkgTest.EventuallyMatchesBody(expectedText),
"SomeDescription",
test.ServingFlags.ResolvableDomain)
if err != nil {
t.Fatalf("The endpoint for Route %s at domain %s didn't serve the expected text \"%s\": %v", routeName, updatedRoute.Status.Domain, expectedText, err)
}
```

This function makes use of [the environment flag `resolvableDomain`](#use-flags)
This function makes use of [the environment flag `resolvableDomain`](https://github.com/knative/serving/blob/master/test/README.md#using-a-resolvable-domain)
to determine if the ingress should be used or the domain should be used
directly.

Expand Down Expand Up @@ -166,7 +180,7 @@ For example, you can poll a `Configuration` object to find the name of the

```go
var revisionName string
err := test.WaitForConfigurationState(clients.ServingClient, configName, func(c *v1alpha1.Configuration) (bool, error) {
err := v1alpha1testing.WaitForConfigurationState(clients.ServingClient, configName, func(c *v1alpha1.Configuration) (bool, error) {
if c.Status.LatestCreatedRevisionName != "" {
revisionName = c.Status.LatestCreatedRevisionName
return true, nil
Expand All @@ -175,15 +189,17 @@ err := test.WaitForConfigurationState(clients.ServingClient, configName, func(c
}, "ConfigurationUpdatedWithRevision")
```

_[Metrics will be emitted](#emit-metrics) for these `Wait` method tracking how
_v1alpha1testing is alias for package `knative.dev/serving/pkg/testing/v1alpha1`_

_Metrics will be emitted for these `Wait` method tracking how
long test poll for._

We also have `Check*` variants of many of these methods with identical
signatures, same example:

```go
var revisionName string
err := test.CheckConfigurationState(clients.ServingClient, configName, func(c *v1alpha1.Configuration) (bool, error) {
err := v1alpha1testing.CheckConfigurationState(clients.ServingClient, configName, func(c *v1alpha1.Configuration) (bool, error) {
if c.Status.LatestCreatedRevisionName != "" {
revisionName = c.Status.LatestCreatedRevisionName
return true, nil
Expand All @@ -192,38 +208,43 @@ err := test.CheckConfigurationState(clients.ServingClient, configName, func(c *v
})
```

_See [crd_checks.go](./crd_checks.go) and
_v1alpha1testing is alias for package `knative.dev/serving/pkg/testing/v1alpha1`_

_For knative crd state, for example `Config`. You can see the code in [configuration.go](./v1alpha1/configuration.go).
For kubernetes objects see
[kube_checks.go](https://github.com/knative/pkg/blob/master/test/kube_checks.go)._

### Verify resource state transitions

To use the [check functions](#check-knative-serving-resources) you must provide
a function to check the state. Some of the expected transition states (as
defined in [the Knative Serving spec](../docs/spec/spec.md)) are expressed in
functions in [states.go](./states.go).
defined in [the Knative Serving spec](https://github.com/knative/docs/blob/master/docs/serving/spec/knative-api-specification-1.0.md))
, for example `v1alpha1/Revision` state, are expressed in
function in [revision.go](./v1alpha1/revision.go).

For example when a `Revision` has been created, the system will start the
resources required to actually serve it, and then the `Revision` object will be
updated to indicate it is ready. This can be polled with `test.IsRevisionReady`:
updated to indicate it is ready. This can be polled with `v1alpha1testing.IsRevisionReady`:

```go
err := test.WaitForRevisionState(clients.ServingClient, revisionName, test.IsRevisionReady(revisionName))
err := v1alpha1testing.WaitForRevisionState(clients.ServingClient, revisionName, v1alpha1testing.IsRevisionReady(revisionName))
if err != nil {
t.Fatalf("Revision %s did not become ready to serve traffic: %v", revisionName, err)
}
```
_v1alpha1testing is alias for package `knative.dev/serving/pkg/testing/v1alpha1`_

Once the `Revision` is created, all traffic for a `Route` should be routed to
it. This can be polled with `test.AllRouteTrafficAtRevision`:
it. This can be polled with `v1alpha1testing.AllRouteTrafficAtRevision`:

```go
err = test.WaitForRouteState(clients.ServingClient, routeName, test.AllRouteTrafficAtRevision(routeName, revisionName))
err := v1alpha1testing.WaitForRouteState(clients.ServingClient, routeName, v1alpha1testing.AllRouteTrafficAtRevision(routeName, revisionName))
if err != nil {
t.Fatalf("The Route %s was not updated to route traffic to the Revision %s: %v", routeName, revisionName, err)
}
```

_See [states.go](./states.go)._
_See [route.go](./v1alpha1/route.go)._

### Generate boilerplate CRDs

Expand All @@ -247,6 +268,7 @@ func TestSomeAwesomeFeature(t *testing.T) {
// more testing
}
```
_test is package `knative.dev/serving/test`_

Please expand these functions as more use cases are tested.

Expand Down

0 comments on commit e81bbd9

Please sign in to comment.