Skip to content

Commit

Permalink
Fully support generation of K8s RBAC resources
Browse files Browse the repository at this point in the history
These changes address a long-time issue in regards of K8s RBAC resources (see related issues).

These changes allow to generate custom Roles, ClusterRoles, ServiceAccount, and RoleBindings.
Plus, it allows the Kubernetes Client and Kubernetes Config extensions to configure the role binding to generate. 

Fix quarkusio#16612
Fix quarkusio#19286
Fix quarkusio#15422
  • Loading branch information
Sgitario committed Mar 15, 2023
1 parent 731756e commit 01ba465
Show file tree
Hide file tree
Showing 52 changed files with 1,844 additions and 137 deletions.
58 changes: 56 additions & 2 deletions docs/src/main/asciidoc/deploying-to-kubernetes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -860,14 +860,68 @@ implementation("io.quarkus:quarkus-kubernetes-client")
----

To access the API server from within a Kubernetes cluster, some RBAC related resources are required (e.g. a ServiceAccount, a RoleBinding).
So, when the `kubernetes-client` extension is present, the `kubernetes` extension is going to create those resources automatically, so that application will be granted the `view` role.
If more roles are required, they will have to be added manually.
So, when the `kubernetes-client` extension is present, by default the `kubernetes` extension is going to create those resources automatically, so that application will be granted the `view` role. You can fully customize the roles and subjects to use using the properties under `quarkus.kubernetes-client.role-binding`.
If more roles are required, they can create them using the properties under `quarkus.kubernetes.rbac.roles`.

[NOTE]
====
You can disable the RBAC resources generation using the property `quarkus.kubernetes-client.generate-rbac=false`.
====

=== Generating RBAC resources

In some scenarios, it's necessary to generate additional https://kubernetes.io/docs/reference/access-authn-authz/rbac/[RBAC] resources that are used by Kubernetes to grant or limit access to other resources. For example, in our use case, we are building https://kubernetes.io/docs/concepts/extend-kubernetes/operator/#operators-in-kubernetes[a Kubernetes operator] that needs to read the list of the installed deployments. To do this, we would need to assign a service account to our operator and link this service account with a role that grants access to the Deployment resources. Let's see how to do this using the `quarkus.kubernetes.rbac` properties:

[source,properties]
----
# Generate the Role resource with name "my-role" <1>
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.api-groups=extensions,apps
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.resources=deployments
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.verbs=list
----

<1> In this example, the role "my-role" will be generated with a policy rule to get the list of deployments.

By default, if one role is configured, a RoleBinding resource will be generated as well to link this role with the ServiceAccount resource.

Moreover, you can have more control over the RBAC resources to be generated:

[source,properties]
----
# Generate Role resource with name "my-role" <1>
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.api-groups=extensions,apps
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.resources=deployments
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.verbs=get,watch,list
# Generate ServiceAccount resource with name "my-service-account" in namespace "my_namespace" <2>
quarkus.kubernetes.rbac.service-accounts.my-service-account.namespace=my_namespace
# Bind Role "my-role" with ServiceAccount "my-service-account" <3>
quarkus.kubernetes.rbac.role-bindings.my-role-binding.subjects.my-service-account.kind=ServiceAccount
quarkus.kubernetes.rbac.role-bindings.my-role-binding.subjects.my-service-account.namespace=my_namespace
quarkus.kubernetes.rbac.role-bindings.my-role-binding.role-name=my-role
----

<1> In this example, the role "my-role" will be generated with the specified policy rules.
<2> Also, the service account "my-service-account" will be generated.
<3> And we can configure the generated RoleBinding resource by selecting the role to be used and the subject.

Finally, we can also generate the cluster wide role resource of "ClusterRole" kind as follows:

[source,properties]
----
# Generate ClusterRole resource with name "my-cluster-role" <1>
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.api-groups=extensions,apps
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.resources=deployments
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.verbs=get,watch,list
# Bind the ClusterRole "my-cluster-role" with the application service account
quarkus.kubernetes.rbac.role-bindings.my-role-binding.role-name=my-cluster-role <2>
----

<1> In this example, the cluster role "my-cluster-role" will be generated with the specified policy rules.
<2> As we have configured only one role, this property is not really necessary.

=== Deploying to Minikube

https://github.com/kubernetes/minikube[Minikube] is quite popular when a Kubernetes cluster is needed for development purposes. To make the deployment to Minikube
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@
import io.quarkus.kubernetes.client.runtime.KubernetesClientBuildConfig;
import io.quarkus.kubernetes.client.runtime.KubernetesClientProducer;
import io.quarkus.kubernetes.client.runtime.KubernetesConfigProducer;
import io.quarkus.kubernetes.client.runtime.RoleBindingConfig;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;
import io.quarkus.maven.dependency.ArtifactKey;

public class KubernetesClientProcessor {
Expand All @@ -69,6 +71,7 @@ public class KubernetesClientProcessor {
private static final DotName KUBE_SCHEMA = DotName.createSimple(KubeSchema.class.getName());
private static final DotName VISITABLE_BUILDER = DotName.createSimple(VisitableBuilder.class.getName());
private static final DotName CUSTOM_RESOURCE = DotName.createSimple(CustomResource.class.getName());
private static final String SERVICE_ACCOUNT = "ServiceAccount";

private static final DotName JSON_FORMAT = DotName.createSimple(JsonFormat.class.getName());
private static final String[] EMPTY_STRINGS_ARRAY = new String[0];
Expand Down Expand Up @@ -106,12 +109,26 @@ public void process(ApplicationIndexBuildItem applicationIndex, CombinedIndexBui
BuildProducer<ReflectiveClassBuildItem> reflectiveClasses,
BuildProducer<ReflectiveHierarchyBuildItem> reflectiveHierarchies,
BuildProducer<IgnoreJsonDeserializeClassBuildItem> ignoredJsonDeserializationClasses,
BuildProducer<KubernetesServiceAccountBuildItem> serviceAccountProducer,
BuildProducer<KubernetesRoleBindingBuildItem> roleBindingProducer,
BuildProducer<ServiceProviderBuildItem> serviceProviderProducer) {

featureProducer.produce(new FeatureBuildItem(Feature.KUBERNETES_CLIENT));
if (kubernetesClientConfig.generateRbac) {
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem("view", true));
RoleBindingConfig rbacConfig = kubernetesClientConfig.roleBinding;
if (SERVICE_ACCOUNT.equals(rbacConfig.subjectKind) && rbacConfig.subjectName.isEmpty()) {
// generate default service account resource
serviceAccountProducer.produce(new KubernetesServiceAccountBuildItem(true));
}

roleBindingProducer.produce(
new KubernetesRoleBindingBuildItem(rbacConfig.name.orElse(null), null, rbacConfig.labels,
new KubernetesRoleBindingBuildItem.RoleRef(rbacConfig.roleName, rbacConfig.clusterWide),
new KubernetesRoleBindingBuildItem.Subject(
rbacConfig.subjectApiGroup.orElse(null),
rbacConfig.subjectKind,
rbacConfig.subjectName.orElse(null),
rbacConfig.subjectNamespace.orElse(null))));
}

// register fully (and not weakly) for reflection watchers, informers and custom resources
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,11 +170,17 @@ public class KubernetesClientBuildConfig {
public Optional<String[]> noProxy;

/**
* Enable the generation of the RBAC manifests.
* Enable the generation of the RBAC manifests. If enabled, it will generate
*/
@ConfigItem(defaultValue = "true")
public boolean generateRbac;

/**
* If generation of RBAC is enabled (see property `quarkus.kubernetes-client.generate-rbac`), then it will use the
* RBAC resources as specified under these properties.
*/
public RoleBindingConfig roleBinding;

/**
* Dev Services
*/
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package io.quarkus.kubernetes.client.runtime;

import java.util.Map;
import java.util.Optional;

import io.quarkus.runtime.annotations.ConfigGroup;
import io.quarkus.runtime.annotations.ConfigItem;

@ConfigGroup
public class RoleBindingConfig {

/**
* Name of the RoleBinding resource to be generated. If not provided, it will use the application name plus the role
* ref name.
*/
@ConfigItem
public Optional<String> name;

/**
* Labels to add into the RoleBinding resource.
*/
@ConfigItem
public Map<String, String> labels;

/**
* The "kind" resource to use by the Subject element in the generated Role Binding resource.
* By default, it's "ServiceAccount" kind.
*/
@ConfigItem(defaultValue = "ServiceAccount")
public String subjectKind;

/**
* The "apiGroup" resource that matches with the "kind" property. By default, it's empty.
*/
@ConfigItem
public Optional<String> subjectApiGroup;

/**
* The "name" resource to use by the Subject element in the generated Role Binding resource.
* By default, it's the application name.
*/
@ConfigItem
public Optional<String> subjectName;

/**
* The "namespace" resource to use by the Subject element in the generated Role Binding resource.
* By default, it will use the same as provided in the generated resources.
*/
@ConfigItem
public Optional<String> subjectNamespace;

/**
* The name of the Role resource to use by the RoleRef element in the generated Role Binding resource.
* By default, it's "view" role name.
*/
@ConfigItem(defaultValue = "view")
public String roleName;

/**
* If the Role sets in the `role-name` property is cluster wide or not.
* By default, it's "true".
*/
@ConfigItem(defaultValue = "true")
public boolean clusterWide;
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package io.quarkus.kubernetes.config.deployment;

import java.util.Collections;
import java.util.List;

import org.jboss.logmanager.Level;
Expand All @@ -15,12 +14,21 @@
import io.quarkus.kubernetes.config.runtime.KubernetesConfigBuildTimeConfig;
import io.quarkus.kubernetes.config.runtime.KubernetesConfigRecorder;
import io.quarkus.kubernetes.config.runtime.KubernetesConfigSourceConfig;
import io.quarkus.kubernetes.config.runtime.SecretsRoleConfig;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.PolicyRule;
import io.quarkus.runtime.TlsConfig;

public class KubernetesConfigProcessor {

private static final String ANY_TARGET = null;
private static final List<PolicyRule> POLICY_RULE_FOR_ROLE = List.of(new PolicyRule(
List.of(""),
List.of("secrets"),
List.of("get")));

@BuildStep
@Record(ExecutionTime.RUNTIME_INIT)
public RunTimeConfigurationSourceValueBuildItem configure(KubernetesConfigRecorder recorder,
Expand All @@ -36,15 +44,26 @@ public RunTimeConfigurationSourceValueBuildItem configure(KubernetesConfigRecord
public void handleAccessToSecrets(KubernetesConfigSourceConfig config,
KubernetesConfigBuildTimeConfig buildTimeConfig,
BuildProducer<KubernetesRoleBuildItem> roleProducer,
BuildProducer<KubernetesClusterRoleBuildItem> clusterRoleProducer,
BuildProducer<KubernetesRoleBindingBuildItem> roleBindingProducer,
KubernetesConfigRecorder recorder) {
if (buildTimeConfig.secretsEnabled) {
roleProducer.produce(new KubernetesRoleBuildItem("view-secrets", Collections.singletonList(
new KubernetesRoleBuildItem.PolicyRule(
Collections.singletonList(""),
Collections.singletonList("secrets"),
List.of("get")))));
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem("view-secrets", false));
SecretsRoleConfig roleConfig = buildTimeConfig.secretsRoleConfig;
String roleName = roleConfig.name;
if (roleConfig.generate) {
if (roleConfig.clusterWide) {
clusterRoleProducer.produce(new KubernetesClusterRoleBuildItem(roleName,
POLICY_RULE_FOR_ROLE,
ANY_TARGET));
} else {
roleProducer.produce(new KubernetesRoleBuildItem(roleName,
roleConfig.namespace.orElse(null),
POLICY_RULE_FOR_ROLE,
ANY_TARGET));
}
}

roleBindingProducer.produce(new KubernetesRoleBindingBuildItem(roleName, roleConfig.clusterWide));
}

recorder.warnAboutSecrets(config, buildTimeConfig);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,9 @@ public class KubernetesConfigBuildTimeConfig {
*/
@ConfigItem(name = "secrets.enabled", defaultValue = "false")
public boolean secretsEnabled;

/**
* Role configuration to generate if the "secrets-enabled" property is true.
*/
public SecretsRoleConfig secretsRoleConfig;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package io.quarkus.kubernetes.config.runtime;

import java.util.Optional;

import io.quarkus.runtime.annotations.ConfigGroup;
import io.quarkus.runtime.annotations.ConfigItem;

@ConfigGroup
public class SecretsRoleConfig {

/**
* The name of the role.
*/
@ConfigItem(defaultValue = "view-secrets")
public String name;

/**
* The namespace of the role.
*/
@ConfigItem
public Optional<String> namespace;

/**
* Whether the role is cluster wide or not. By default, it's not a cluster wide role.
*/
@ConfigItem(defaultValue = "false")
public boolean clusterWide;

/**
* If the current role is meant to be generated or not. If not, it will only be used to generate the role binding resource.
*/
@ConfigItem(defaultValue = "true")
public boolean generate;
}
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
import io.quarkus.kubernetes.spi.CustomProjectRootBuildItem;
import io.quarkus.kubernetes.spi.DecoratorBuildItem;
import io.quarkus.kubernetes.spi.KubernetesAnnotationBuildItem;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesCommandBuildItem;
import io.quarkus.kubernetes.spi.KubernetesDeploymentTargetBuildItem;
import io.quarkus.kubernetes.spi.KubernetesEnvBuildItem;
Expand All @@ -46,6 +47,7 @@
import io.quarkus.kubernetes.spi.KubernetesResourceMetadataBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;

public class KindProcessor {

Expand Down Expand Up @@ -111,6 +113,8 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
Optional<KubernetesHealthReadinessPathBuildItem> readinessPath,
Optional<KubernetesHealthStartupPathBuildItem> startupPath,
List<KubernetesRoleBuildItem> roles,
List<KubernetesClusterRoleBuildItem> clusterRoles,
List<KubernetesServiceAccountBuildItem> serviceAccounts,
List<KubernetesRoleBindingBuildItem> roleBindings,
Optional<CustomProjectRootBuildItem> customProjectRoot) {

Expand All @@ -120,7 +124,7 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
livenessPath,
readinessPath,
startupPath,
roles, roleBindings, customProjectRoot);
roles, clusterRoles, serviceAccounts, roleBindings, customProjectRoot);
}

@BuildStep
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
import io.quarkus.kubernetes.spi.CustomProjectRootBuildItem;
import io.quarkus.kubernetes.spi.DecoratorBuildItem;
import io.quarkus.kubernetes.spi.KubernetesAnnotationBuildItem;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesCommandBuildItem;
import io.quarkus.kubernetes.spi.KubernetesDeploymentTargetBuildItem;
import io.quarkus.kubernetes.spi.KubernetesEnvBuildItem;
Expand All @@ -43,6 +44,7 @@
import io.quarkus.kubernetes.spi.KubernetesResourceMetadataBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;

public class MinikubeProcessor {

Expand Down Expand Up @@ -107,6 +109,8 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
Optional<KubernetesHealthReadinessPathBuildItem> readinessPath,
Optional<KubernetesHealthStartupPathBuildItem> startupPath,
List<KubernetesRoleBuildItem> roles,
List<KubernetesClusterRoleBuildItem> clusterRoles,
List<KubernetesServiceAccountBuildItem> serviceAccounts,
List<KubernetesRoleBindingBuildItem> roleBindings,
Optional<CustomProjectRootBuildItem> customProjectRoot) {

Expand All @@ -116,6 +120,6 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
livenessPath,
readinessPath,
startupPath,
roles, roleBindings, customProjectRoot);
roles, clusterRoles, serviceAccounts, roleBindings, customProjectRoot);
}
}
Loading

0 comments on commit 01ba465

Please sign in to comment.