Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

List inbuilt sources if CRD access is restricted #948

Merged
merged 12 commits into from
Aug 4, 2020
61 changes: 57 additions & 4 deletions pkg/dynamic/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
package dynamic

import (
"errors"
"strings"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/labels"
Expand Down Expand Up @@ -42,12 +45,15 @@ type KnDynamicClient interface {
// ListCRDs returns list of CRDs with their type and name
ListCRDs(options metav1.ListOptions) (*unstructured.UnstructuredList, error)

// ListSourceCRDs returns list of eventing sources CRDs
// ListSourcesTypes returns list of eventing sources CRDs
ListSourcesTypes() (*unstructured.UnstructuredList, error)

// ListSources returns list of available source objects
ListSources(types ...WithType) (*unstructured.UnstructuredList, error)

// ListSourcesUsingGVKs returns list of available source objects using given list of GVKs
ListSourcesUsingGVKs(*[]schema.GroupVersionKind, ...WithType) (*unstructured.UnstructuredList, error)

// RawClient returns the raw dynamic client interface
RawClient() dynamic.Interface
}
Expand Down Expand Up @@ -107,12 +113,17 @@ func (c *knDynamicClient) ListSources(types ...WithType) (*unstructured.Unstruct
var (
sourceList unstructured.UnstructuredList
options metav1.ListOptions
numberOfsourceTypesFound int
numberOfSourceTypesFound int
)
sourceTypes, err := c.ListSourcesTypes()
if err != nil {
return nil, err
}

if sourceTypes == nil || len(sourceTypes.Items) == 0 {
navidshaikh marked this conversation as resolved.
Show resolved Hide resolved
return nil, errors.New("no sources found on the backend, please verify the installation")
}

namespace := c.Namespace()
filters := WithTypes(types).List()
// For each source type available, find out each source types objects
Expand Down Expand Up @@ -141,14 +152,56 @@ func (c *knDynamicClient) ListSources(types ...WithType) (*unstructured.Unstruct

if len(sList.Items) > 0 {
// keep a track if we found source objects of different types
numberOfsourceTypesFound++
numberOfSourceTypesFound++
sourceList.Items = append(sourceList.Items, sList.Items...)
sourceList.SetGroupVersionKind(sList.GetObjectKind().GroupVersionKind())
}
}
// Clear the Group and Version for list if there are multiple types of source objects found
// Keep the source's GVK if there is only one type of source objects found or requested via --type filter
if numberOfSourceTypesFound > 1 {
sourceList.SetGroupVersionKind(schema.GroupVersionKind{Group: "", Version: "", Kind: "List"})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure whether we should set or not-set the GVK based on the result. This might lead to unexpected behaviour depending on the context where you run it. I think its better to just leave out the GVK on the list (or maybe even better, but not sure if this works), just return a slice of Unstructed instead of an UnstructuredList

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done to show proper values if json or yaml format of list is requested and the list contains multiple types of source objects.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd need GVK set for the (list) object here as (json/yaml) printers require it. The question was which version/group to set if there are different types of sources found, so we set cleared group and version values and set List for kind.
If there is only one type of source object found OR user requested single type using --type filter, we keep the GVK intact.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but imagine the situation that by accident you have only one source of one type. If you list those, you the GVK set. Some time later another source of another type is added. Suddenly know the GVK gets removed or changed to an artificial "List". I think as we are dealing with a heterogenous list, that, by accident, can be also homogenous, we should treat it as a heterogenous list always. I would not mind to introduce a new type here, like we did for the Export, with an client.knative.dev group and a v1alpha1 version for now, but then use it all the time.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not mind to introduce a new type here, like we did for the Export, with an client.knative.dev group and a v1alpha1 version for now, but then use it all the time.

yea, we could do that, however for the other point about list of single source type, I think setting exact source type for single-type-of-source list is explicit and we could scope the client custom type for list only if there multiple types of sources.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added #953 to track adding setting this custom client type for list. We can refine in next iteration.

}
return &sourceList, nil
}

// ListSourcesUsingGVKs returns list of available source objects using given list of GVKs
func (c *knDynamicClient) ListSourcesUsingGVKs(gvks *[]schema.GroupVersionKind, types ...WithType) (*unstructured.UnstructuredList, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder what is the best function behavior when gvks is nil: a) to return a nil list or b) to ignore gvks and use types to filter. The current behavior is a).
Yet when types is nil, the current behavior is b) to ignore types and use gvks to filter, not a) to return a nil list. It makes me a little weird.
Of course it doesn't affect the current functions. For future usage, maybe change to the same behavior when either of these two parameters is nil.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use types only to subset the found source types. If gvks list is nil, it returns nil. Does this answer your question ?

if gvks == nil {
return nil, nil
}

var (
sourceList unstructured.UnstructuredList
options metav1.ListOptions
numberOfSourceTypesFound int
)
namespace := c.Namespace()
filters := WithTypes(types).List()

for _, gvk := range *gvks {
if len(filters) > 0 && !util.SliceContainsIgnoreCase(filters, gvk.Kind) {
continue
}

gvr := gvk.GroupVersion().WithResource(strings.ToLower(gvk.Kind) + "s")
navidshaikh marked this conversation as resolved.
Show resolved Hide resolved

// list objects of source type with this GVR
sList, err := c.client.Resource(gvr).Namespace(namespace).List(options)
if err != nil {
return nil, err
}

if len(sList.Items) > 0 {
// keep a track if we found source objects of different types
numberOfSourceTypesFound++
sourceList.Items = append(sourceList.Items, sList.Items...)
sourceList.SetGroupVersionKind(sList.GetObjectKind().GroupVersionKind())
}
}
// Clear the Group and Version for list if there are multiple types of source objects found
// Keep the source's GVK if there is only one type of source objects found or requested via --type filter
if numberOfsourceTypesFound > 1 {
if numberOfSourceTypesFound > 1 {
navidshaikh marked this conversation as resolved.
Show resolved Hide resolved
sourceList.SetGroupVersionKind(schema.GroupVersionKind{Group: "", Version: "", Kind: "List"})
}
return &sourceList, nil
Expand Down
34 changes: 34 additions & 0 deletions pkg/dynamic/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,40 @@ func TestListSources(t *testing.T) {
})
}

func TestListSourcesUsingGVKs(t *testing.T) {
t.Run("No GVKs given", func(t *testing.T) {
client := createFakeKnDynamicClient(testNamespace)
assert.Check(t, client.RawClient() != nil)
s, err := client.ListSourcesUsingGVKs(nil)
assert.NilError(t, err)
assert.Check(t, s == nil)
})

t.Run("source list with given GVKs", func(t *testing.T) {
client := createFakeKnDynamicClient(testNamespace,
newSourceCRDObjWithSpec("pingsources", "sources.knative.dev", "v1alpha1", "PingSource"),
newSourceCRDObjWithSpec("apiserversources", "sources.knative.dev", "v1alpha1", "ApiServerSource"),
newSourceUnstructuredObj("p1", "sources.knative.dev/v1alpha1", "PingSource"),
newSourceUnstructuredObj("a1", "sources.knative.dev/v1alpha1", "ApiServerSource"),
)
assert.Check(t, client.RawClient() != nil)
gv := schema.GroupVersion{"sources.knative.dev", "v1alpha1"}
gvks := []schema.GroupVersionKind{gv.WithKind("ApiServerSource"), gv.WithKind("PingSource")}

s, err := client.ListSourcesUsingGVKs(&gvks)
assert.NilError(t, err)
assert.Check(t, s != nil)
assert.Equal(t, len(s.Items), 2)

// withType
s, err = client.ListSourcesUsingGVKs(&gvks, WithTypeFilter("PingSource"))
assert.NilError(t, err)
assert.Check(t, s != nil)
assert.Equal(t, len(s.Items), 1)
})

}

// createFakeKnDynamicClient gives you a dynamic client for testing containing the given objects.
// See also the one in the fake package. Duplicated here to avoid a dependency loop.
func createFakeKnDynamicClient(testNamespace string, objects ...runtime.Object) KnDynamicClient {
Expand Down
23 changes: 23 additions & 0 deletions pkg/dynamic/lib.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ package dynamic

import (
"fmt"
"strings"

"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
Expand Down Expand Up @@ -123,3 +124,25 @@ func (types WithTypes) List() []string {
}
return stypes
}

// UnstructuredCRDFromGVK constructs an unstructured object using the given GVK
func UnstructuredCRDFromGVK(gvk schema.GroupVersionKind) *unstructured.Unstructured {
navidshaikh marked this conversation as resolved.
Show resolved Hide resolved
name := fmt.Sprintf("%ss.%s", strings.ToLower(gvk.Kind), gvk.Group)
plural := fmt.Sprintf("%ss", strings.ToLower(gvk.Kind))
u := &unstructured.Unstructured{}
u.SetUnstructuredContent(map[string]interface{}{
"metadata": map[string]interface{}{
"name": name,
},
"spec": map[string]interface{}{
"group": gvk.Group,
"version": gvk.Version,
"names": map[string]interface{}{
"kind": gvk.Kind,
"plural": plural,
},
},
})

return u
}
20 changes: 20 additions & 0 deletions pkg/dynamic/lib_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
"testing"

"gotest.tools/assert"
"k8s.io/apimachinery/pkg/runtime/schema"

"knative.dev/client/pkg/util"
)
Expand Down Expand Up @@ -77,3 +78,22 @@ func TestGVRFromUnstructured(t *testing.T) {
assert.Check(t, err != nil)
assert.Check(t, util.ContainsAll(err.Error(), "can't", "find", "version"))
}

func TestUnstructuredCRDFromGVK(t *testing.T) {
u := UnstructuredCRDFromGVK(schema.GroupVersionKind{"sources.knative.dev", "v1alpha2", "ApiServerSource"})
g, err := groupFromUnstructured(u)
assert.NilError(t, err)
assert.Equal(t, g, "sources.knative.dev")

v, err := versionFromUnstructured(u)
assert.NilError(t, err)
assert.Equal(t, v, "v1alpha2")

k, err := kindFromUnstructured(u)
assert.NilError(t, err)
assert.Equal(t, k, "ApiServerSource")

r, err := resourceFromUnstructured(u)
assert.NilError(t, err)
assert.Equal(t, r, "apiserversources")
}
4 changes: 2 additions & 2 deletions pkg/errors/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@ func newInvalidCRD(apiGroup string) *KNError {
return NewKNError(msg)
}

func newNoRouteToHost(errString string) error {
func newNoRouteToHost(errString string) *KNError {
parts := strings.SplitAfter(errString, "dial tcp")
if len(parts) == 2 {
return NewKNError(fmt.Sprintf("error connecting to the cluster, please verify connection at: %s", strings.Trim(parts[1], " ")))
}
return NewKNError(fmt.Sprintf("error connecting to the cluster: %s", errString))
}

func newNoKubeConfig(errString string) error {
func newNoKubeConfig(errString string) *KNError {
return NewKNError("no kubeconfig has been provided, please use a valid configuration to connect to the cluster")
}
9 changes: 9 additions & 0 deletions pkg/errors/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
package errors

import (
"net/http"
"strings"

api_errors "k8s.io/apimachinery/pkg/api/errors"
Expand Down Expand Up @@ -63,3 +64,11 @@ func GetError(err error) error {
return err
}
}

// IsForbiddenError returns true if given error can be converted to API status and of type forbidden access else false
func IsForbiddenError(err error) bool {
if status, ok := err.(api_errors.APIStatus); ok {
return status.Status().Code == int32(http.StatusForbidden)
}
return false
}
31 changes: 31 additions & 0 deletions pkg/errors/factory_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,11 @@ func TestKnErrors(t *testing.T) {
Error: errors.New("no route to host 192.168.1.1"),
ExpectedMsg: "error connecting to the cluster: no route to host 192.168.1.1",
},
{
Name: "foo error which cant be converted to APIStatus",
Error: errors.New("foo error"),
ExpectedMsg: "foo error",
},
}
for _, tc := range cases {
tc := tc
Expand All @@ -130,3 +135,29 @@ func TestKnErrors(t *testing.T) {
})
}
}

func TestIsForbiddenError(t *testing.T) {
cases := []struct {
Name string
Error error
Forbidden bool
}{
{
Name: "forbidden error",
Error: api_errors.NewForbidden(schema.GroupResource{Group: "apiextensions.k8s.io", Resource: "CustomResourceDefinition"}, "", nil),
Forbidden: true,
},
{
Name: "non forbidden error",
Error: errors.New("panic"),
Forbidden: false,
},
}
for _, tc := range cases {
tc := tc
t.Run(tc.Name, func(t *testing.T) {
t.Parallel()
assert.Equal(t, IsForbiddenError(tc.Error), tc.Forbidden)
})
}
}
18 changes: 15 additions & 3 deletions pkg/kn/commands/source/list.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,11 @@ import (
"github.com/spf13/cobra"

"knative.dev/client/pkg/dynamic"
knerrors "knative.dev/client/pkg/errors"
"knative.dev/client/pkg/kn/commands"
"knative.dev/client/pkg/kn/commands/flags"
"knative.dev/client/pkg/kn/commands/source/duck"
sourcesv1alpha2 "knative.dev/client/pkg/sources/v1alpha2"
)

var listExample = `
Expand Down Expand Up @@ -52,15 +54,25 @@ func NewListCommand(p *commands.KnParams) *cobra.Command {
if err != nil {
return err
}

var filters dynamic.WithTypes
for _, filter := range filterFlags.Filters {
filters = append(filters, dynamic.WithTypeFilter(filter))
}

sourceList, err := dynamicClient.ListSources(filters...)
if err != nil {
return err

switch {
case knerrors.IsForbiddenError(err):
gvks := sourcesv1alpha2.BuiltInSourcesGVKs()
if sourceList, err = dynamicClient.ListSourcesUsingGVKs(&gvks, filters...); err != nil {
return knerrors.GetError(err)
}
case err != nil:
navidshaikh marked this conversation as resolved.
Show resolved Hide resolved
return knerrors.GetError(err)
}
if len(sourceList.Items) == 0 {

if sourceList == nil || len(sourceList.Items) == 0 {
fmt.Fprintf(cmd.OutOrStdout(), "No sources found in %s namespace.\n", namespace)
return nil
}
Expand Down
10 changes: 10 additions & 0 deletions pkg/kn/commands/source/list_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ import (
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/runtime"

dynamicfake "k8s.io/client-go/dynamic/fake"
clientdynamic "knative.dev/client/pkg/dynamic"
"knative.dev/client/pkg/kn/commands"
"knative.dev/client/pkg/util"
)
Expand Down Expand Up @@ -71,6 +73,14 @@ func TestSourceListTypesNoHeaders(t *testing.T) {
assert.Check(t, util.ContainsAll(output[0], "PingSource"))
}

func TestListBuiltInSources(t *testing.T) {
navidshaikh marked this conversation as resolved.
Show resolved Hide resolved
fakeDynamic := dynamicfake.NewSimpleDynamicClient(runtime.NewScheme())
sources, err := listBuiltInSourceTypes(clientdynamic.NewKnDynamicClient(fakeDynamic, "current"))
assert.NilError(t, err)
assert.Check(t, sources != nil)
assert.Equal(t, len(sources.Items), 4)
}

func TestSourceList(t *testing.T) {
output, err := sourceFakeCmd([]string{"source", "list"},
newSourceCRDObjWithSpec("pingsources", "sources.knative.dev", "v1alpha1", "PingSource"),
Expand Down
Loading