Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

camel-k-operator forbidden to get kafkachannels.messaging.knative.dev since 1.8.x #2950

Closed
maschmid opened this issue Jan 26, 2022 · 6 comments · Fixed by #2958
Closed

camel-k-operator forbidden to get kafkachannels.messaging.knative.dev since 1.8.x #2950

maschmid opened this issue Jan 26, 2022 · 6 comments · Fixed by #2958
Assignees
Labels
area/knative Related to Knative

Comments

@maschmid
Copy link

With camelk 1.7, I can use the https://github.com/apache/camel-k/tree/main/examples/knative example with a KafkaChannel channels, instead of InMemoryChannel.

The same example fails on 1.8.x with KafkaChannels, due to

error looking up channel words: kafkachannels.messaging.knative.dev "words" is forbidden: User "system:serviceaccount:openshift-operators:camel-k-operator" cannot get resource "kafkachannels" in API group "messaging.knative.dev" in the namespace \"myproject\"

( compare https://github.com/apache/camel-k/blob/release-1.7.x/helm/camel-k/templates/operator-role.yaml#L219-L224 vs https://github.com/apache/camel-k/blob/main/helm/camel-k/templates/operator-role.yaml#L244-L248

it is clear the now the rules are more explicit, which also means it lacks permission to access non-default channel kinds.

(full error from camel-k-operator logs)

{"level":"error","ts":1643215313.6744456,"logger":"controller.integration-controller","msg":"Reconciler error","reconciler group":"camel.apache.org","reconciler kind":"Integration","name":"printer","namespace":"myproject","error":"error during trait customization: error looking up channel words: kafkachannels.messaging.knative.dev \"words\" is forbidden: User \"system:serviceaccount:openshift-operators:camel-k-operator\" cannot get resource \"kafkachannels\" in API group \"messaging.knative.dev\" in the namespace \"myproject\"","errorVerbose":"kafkachannels.messaging.knative.dev \"words\" is forbidden: User \"system:serviceaccount:openshift-operators:camel-k-operator\" cannot get resource \"kafkachannels\" in API group \"messaging.knative.dev\" in the namespace \"myproject\"\nerror looking up channel words\ngithub.com/apache/camel-k/pkg/trait.(*knativeTrait).withServiceDo\n\tgithub.com/apache/camel-k/pkg/trait/knative.go:581\ngithub.com/apache/camel-k/pkg/trait.(*knativeTrait).ifServiceMissingDo\n\tgithub.com/apache/camel-k/pkg/trait/knative.go:552\ngithub.com/apache/camel-k/pkg/trait.(*knativeTrait).configureChannels\n\tgithub.com/apache/camel-k/pkg/trait/knative.go:253\ngithub.com/apache/camel-k/pkg/trait.(*knativeTrait).Apply\n\tgithub.com/apache/camel-k/pkg/trait/knative.go:227\ngithub.com/apache/camel-k/pkg/trait.(*Catalog).apply\n\tgithub.com/apache/camel-k/pkg/trait/trait_catalog.go:113\ngithub.com/apache/camel-k/pkg/trait.Apply\n\tgithub.com/apache/camel-k/pkg/trait/trait.go:47\ngithub.com/apache/camel-k/pkg/controller/integration.(*monitorAction).Handle\n\tgithub.com/apache/camel-k/pkg/controller/integration/monitor.go:118\ngithub.com/apache/camel-k/pkg/controller/integration.(*reconcileIntegration).Reconcile\n\tgithub.com/apache/camel-k/pkg/controller/integration/integration_controller.go:309\ngithub.com/apache/camel-k/pkg/util/monitoring.(*instrumentedReconciler).Reconcile\n\tgithub.com/apache/camel-k/pkg/util/monitoring/controller.go:70\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214\nruntime.goexit\n\truntime/asm_amd64.s:1371\nerror during trait customization\ngithub.com/apache/camel-k/pkg/trait.Apply\n\tgithub.com/apache/camel-k/pkg/trait/trait.go:48\ngithub.com/apache/camel-k/pkg/controller/integration.(*monitorAction).Handle\n\tgithub.com/apache/camel-k/pkg/controller/integration/monitor.go:118\ngithub.com/apache/camel-k/pkg/controller/integration.(*reconcileIntegration).Reconcile\n\tgithub.com/apache/camel-k/pkg/controller/integration/integration_controller.go:309\ngithub.com/apache/camel-k/pkg/util/monitoring.(*instrumentedReconciler).Reconcile\n\tgithub.com/apache/camel-k/pkg/util/monitoring/controller.go:70\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214\nruntime.goexit\n\truntime/asm_amd64.s:1371","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:214"}
@astefanutti
Copy link
Member

Wildcard permissions have been removed with #2807, as unbounded RBAC is not considered a best practice.

A Knative Sink can be any resource that's Addressable, and the Camel K operator do a GET request to retrieve the address from the resource status.address.url field.

We can expand the list of default Addressable resources that the operator is granted read permission on, like KafkaChannel, but that's ultimately the admin responsibility to set these extra permissions in the camel-k-operator Role, similar to what Knative requires with the kafkasinks-addressable-resolver ClusterRole:

https://knative.dev/docs/eventing/sinks/#using-custom-resources-as-sinks

If that is the right approach, we need to document it.

@maschmid
Copy link
Author

Right, I agree it makes sense for the camel-k project to leave the responsibility to the administrator, as camel-k project itself cannot know what the system as a whole will be using.

This of course present a problem for an ecosystem of operators, which should somehow be integrated together and ideally work OOB (like the OpenShift operators), as the role of those operators is to make the administrator's job easier... I don't have answer to that problem. ( I guess some general mechanism would be needed, loosely coupling CRDs and ClusterRoles together, such that some mechanism would grant these rules automatically somehow, but also somehow safely (to not allow just any custom Role to be granted write access to all addressables, etc...?), but of course camel-k is not the proper venue to solve this general problem...

@astefanutti
Copy link
Member

astefanutti commented Jan 27, 2022

Thanks a lot for your feedback.

Would you know if the duck.knative.dev/addressable=true, as in the example taken from the Knative documentation to extend Knative itself with the extra permissions, is a "standard" Knative label:

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: kafkasinks-addressable-resolver
  labels:
    kafka.eventing.knative.dev/release: devel
    duck.knative.dev/addressable: "true"
# Do not use this role directly. These rules will be added to the "addressable-resolver" role.
rules:
  - apiGroups:
      - eventing.knative.dev
    resources:
      - kafkasinks
      - kafkasinks/status
    verbs:
      - get
      - list
      - watch

If that it's the case, it may be possible to create an aggregated role, like:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: camel-k-operator-knative-sinks
aggregationRule:
  clusterRoleSelectors:
  - matchLabels:
      duck.knative.dev/addressable: "true"
rules: []

And have the Kubernetes control plane automatically update it with the aggregated rules.

WDYT?

@pierDipi
Copy link

The aggregated ClusterRole is already created, it's called addressable-resolver.

This is the ClusterRoleBinding to bind a SA to the addressable-resolver ClusterRole

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: camel-k-addressable-resolver
subjects:
  - kind: ServiceAccount
    name: <camel-k-service-account>
    namespace: <camel-k-namespace>
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: addressable-resolver

@astefanutti astefanutti self-assigned this Jan 28, 2022
@astefanutti
Copy link
Member

@pierDipi Thanks!

I infer the addressable-resolver ClusterRole can be assumed to be "publicly" available for other controllers to reference it.

I've researched how to create the corresponding ClusterRoleBinding via OLM, but that does not seem supported, as subjects are not resolved for OLM bundle additional manifests. So it has to be created from the operator at start time.

@astefanutti
Copy link
Member

astefanutti commented Jan 28, 2022

I've open a new PR #2958, that adds bind permission to the addressable-resolver ClusterRole resource, so it's either managed by OLM, or Camel K CLI, while the actual binding is done by the Camel K operator itself, either namespace-scoped (RoleBinding), or globally (ClusterRoleBinding), depending on the deployment mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/knative Related to Knative
Projects
None yet
3 participants