Skip to content

Commit

Permalink
Fully support generation of K8s RBAC resources
Browse files Browse the repository at this point in the history
These changes address a long-time issue in regards of K8s RBAC resources (see related issues).

These changes allow to generate custom Roles, ClusterRoles, ServiceAccount, and RoleBindings.
Plus, it allows the Kubernetes Client and Kubernetes Config extensions to configure the role binding to generate.

Fix quarkusio#16612
Fix quarkusio#19286
Fix quarkusio#15422
  • Loading branch information
Sgitario committed Mar 16, 2023
1 parent 4e42336 commit 0ad1690
Show file tree
Hide file tree
Showing 52 changed files with 1,734 additions and 155 deletions.
58 changes: 56 additions & 2 deletions docs/src/main/asciidoc/deploying-to-kubernetes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -860,14 +860,68 @@ implementation("io.quarkus:quarkus-kubernetes-client")
----

To access the API server from within a Kubernetes cluster, some RBAC related resources are required (e.g. a ServiceAccount, a RoleBinding).
So, when the `kubernetes-client` extension is present, the `kubernetes` extension is going to create those resources automatically, so that application will be granted the `view` role.
If more roles are required, they will have to be added manually.
So, when the `kubernetes-client` extension is present, by default the `kubernetes` extension is going to create those resources automatically, so that application will be granted the `view` role. You can fully customize the roles and subjects to use using the properties under `quarkus.kubernetes-client.role-binding`.
If more roles are required, they can create them using the properties under `quarkus.kubernetes.rbac.roles`.

[NOTE]
====
You can disable the RBAC resources generation using the property `quarkus.kubernetes-client.generate-rbac=false`.
====

=== Generating RBAC resources

In some scenarios, it's necessary to generate additional https://kubernetes.io/docs/reference/access-authn-authz/rbac/[RBAC] resources that are used by Kubernetes to grant or limit access to other resources. For example, in our use case, we are building https://kubernetes.io/docs/concepts/extend-kubernetes/operator/#operators-in-kubernetes[a Kubernetes operator] that needs to read the list of the installed deployments. To do this, we would need to assign a service account to our operator and link this service account with a role that grants access to the Deployment resources. Let's see how to do this using the `quarkus.kubernetes.rbac` properties:

[source,properties]
----
# Generate the Role resource with name "my-role" <1>
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.api-groups=extensions,apps
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.resources=deployments
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.verbs=list
----

<1> In this example, the role "my-role" will be generated with a policy rule to get the list of deployments.

By default, if one role is configured, a RoleBinding resource will be generated as well to link this role with the ServiceAccount resource.

Moreover, you can have more control over the RBAC resources to be generated:

[source,properties]
----
# Generate Role resource with name "my-role" <1>
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.api-groups=extensions,apps
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.resources=deployments
quarkus.kubernetes.rbac.roles.my-role.policy-rules.0.verbs=get,watch,list
# Generate ServiceAccount resource with name "my-service-account" in namespace "my_namespace" <2>
quarkus.kubernetes.rbac.service-accounts.my-service-account.namespace=my_namespace
# Bind Role "my-role" with ServiceAccount "my-service-account" <3>
quarkus.kubernetes.rbac.role-bindings.my-role-binding.subjects.my-service-account.kind=ServiceAccount
quarkus.kubernetes.rbac.role-bindings.my-role-binding.subjects.my-service-account.namespace=my_namespace
quarkus.kubernetes.rbac.role-bindings.my-role-binding.role-name=my-role
----

<1> In this example, the role "my-role" will be generated with the specified policy rules.
<2> Also, the service account "my-service-account" will be generated.
<3> And we can configure the generated RoleBinding resource by selecting the role to be used and the subject.

Finally, we can also generate the cluster wide role resource of "ClusterRole" kind as follows:

[source,properties]
----
# Generate ClusterRole resource with name "my-cluster-role" <1>
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.api-groups=extensions,apps
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.resources=deployments
quarkus.kubernetes.rbac.cluster-roles.my-cluster-role.policy-rules.0.verbs=get,watch,list
# Bind the ClusterRole "my-cluster-role" with the application service account
quarkus.kubernetes.rbac.role-bindings.my-role-binding.role-name=my-cluster-role <2>
----

<1> In this example, the cluster role "my-cluster-role" will be generated with the specified policy rules.
<2> As we have configured only one role, this property is not really necessary.

=== Deploying to Minikube

https://github.com/kubernetes/minikube[Minikube] is quite popular when a Kubernetes cluster is needed for development purposes. To make the deployment to Minikube
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
import io.quarkus.kubernetes.client.runtime.KubernetesClientBuildConfig;
import io.quarkus.kubernetes.client.runtime.KubernetesClientProducer;
import io.quarkus.kubernetes.client.runtime.KubernetesConfigProducer;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.client.spi.KubernetesClientCapabilityBuildItem;
import io.quarkus.maven.dependency.ArtifactKey;

public class KubernetesClientProcessor {
Expand All @@ -69,6 +69,7 @@ public class KubernetesClientProcessor {
private static final DotName KUBE_SCHEMA = DotName.createSimple(KubeSchema.class.getName());
private static final DotName VISITABLE_BUILDER = DotName.createSimple(VisitableBuilder.class.getName());
private static final DotName CUSTOM_RESOURCE = DotName.createSimple(CustomResource.class.getName());
private static final String SERVICE_ACCOUNT = "ServiceAccount";

private static final DotName JSON_FORMAT = DotName.createSimple(JsonFormat.class.getName());
private static final String[] EMPTY_STRINGS_ARRAY = new String[0];
Expand Down Expand Up @@ -106,13 +107,13 @@ public void process(ApplicationIndexBuildItem applicationIndex, CombinedIndexBui
BuildProducer<ReflectiveClassBuildItem> reflectiveClasses,
BuildProducer<ReflectiveHierarchyBuildItem> reflectiveHierarchies,
BuildProducer<IgnoreJsonDeserializeClassBuildItem> ignoredJsonDeserializationClasses,
BuildProducer<KubernetesRoleBindingBuildItem> roleBindingProducer,
BuildProducer<ServiceProviderBuildItem> serviceProviderProducer) {
BuildProducer<ServiceProviderBuildItem> serviceProviderProducer,
BuildProducer<KubernetesClientCapabilityBuildItem> kubernetesClientCapabilityProducer) {

featureProducer.produce(new FeatureBuildItem(Feature.KUBERNETES_CLIENT));
if (kubernetesClientConfig.generateRbac) {
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem("view", true));
}

kubernetesClientCapabilityProducer
.produce(new KubernetesClientCapabilityBuildItem(kubernetesClientConfig.generateRbac));

// register fully (and not weakly) for reflection watchers, informers and custom resources
final Set<DotName> watchedClasses = new HashSet<>();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,9 @@ public class KubernetesClientBuildConfig {
public Optional<String[]> noProxy;

/**
* Enable the generation of the RBAC manifests.
* Enable the generation of the RBAC manifests. If enabled and no other role binding are provided using the properties
* `quarkus.kubernetes.rbac.`, it will generate a default role binding using the role "view" and the application
* service account.
*/
@ConfigItem(defaultValue = "true")
public boolean generateRbac;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
package io.quarkus.kubernetes.client.spi;

import io.quarkus.builder.item.SimpleBuildItem;

public final class KubernetesClientCapabilityBuildItem extends SimpleBuildItem {

private final boolean generateRbac;

public KubernetesClientCapabilityBuildItem(boolean generateRbac) {
this.generateRbac = generateRbac;
}

public boolean isGenerateRbac() {
return generateRbac;
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package io.quarkus.kubernetes.config.deployment;

import java.util.Collections;
import java.util.List;

import org.jboss.logmanager.Level;
Expand All @@ -15,12 +14,22 @@
import io.quarkus.kubernetes.config.runtime.KubernetesConfigBuildTimeConfig;
import io.quarkus.kubernetes.config.runtime.KubernetesConfigRecorder;
import io.quarkus.kubernetes.config.runtime.KubernetesConfigSourceConfig;
import io.quarkus.kubernetes.config.runtime.SecretsRoleConfig;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;
import io.quarkus.kubernetes.spi.PolicyRule;
import io.quarkus.runtime.TlsConfig;

public class KubernetesConfigProcessor {

private static final String ANY_TARGET = null;
private static final List<PolicyRule> POLICY_RULE_FOR_ROLE = List.of(new PolicyRule(
List.of(""),
List.of("secrets"),
List.of("get")));

@BuildStep
@Record(ExecutionTime.RUNTIME_INIT)
public RunTimeConfigurationSourceValueBuildItem configure(KubernetesConfigRecorder recorder,
Expand All @@ -36,15 +45,28 @@ public RunTimeConfigurationSourceValueBuildItem configure(KubernetesConfigRecord
public void handleAccessToSecrets(KubernetesConfigSourceConfig config,
KubernetesConfigBuildTimeConfig buildTimeConfig,
BuildProducer<KubernetesRoleBuildItem> roleProducer,
BuildProducer<KubernetesClusterRoleBuildItem> clusterRoleProducer,
BuildProducer<KubernetesServiceAccountBuildItem> serviceAccountProducer,
BuildProducer<KubernetesRoleBindingBuildItem> roleBindingProducer,
KubernetesConfigRecorder recorder) {
if (buildTimeConfig.secretsEnabled) {
roleProducer.produce(new KubernetesRoleBuildItem("view-secrets", Collections.singletonList(
new KubernetesRoleBuildItem.PolicyRule(
Collections.singletonList(""),
Collections.singletonList("secrets"),
List.of("get")))));
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem("view-secrets", false));
SecretsRoleConfig roleConfig = buildTimeConfig.secretsRoleConfig;
String roleName = roleConfig.name;
if (roleConfig.generate) {
if (roleConfig.clusterWide) {
clusterRoleProducer.produce(new KubernetesClusterRoleBuildItem(roleName,
POLICY_RULE_FOR_ROLE,
ANY_TARGET));
} else {
roleProducer.produce(new KubernetesRoleBuildItem(roleName,
roleConfig.namespace.orElse(null),
POLICY_RULE_FOR_ROLE,
ANY_TARGET));
}
}

serviceAccountProducer.produce(new KubernetesServiceAccountBuildItem(true));
roleBindingProducer.produce(new KubernetesRoleBindingBuildItem(roleName, roleConfig.clusterWide));
}

recorder.warnAboutSecrets(config, buildTimeConfig);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,9 @@ public class KubernetesConfigBuildTimeConfig {
*/
@ConfigItem(name = "secrets.enabled", defaultValue = "false")
public boolean secretsEnabled;

/**
* Role configuration to generate if the "secrets-enabled" property is true.
*/
public SecretsRoleConfig secretsRoleConfig;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package io.quarkus.kubernetes.config.runtime;

import java.util.Optional;

import io.quarkus.runtime.annotations.ConfigGroup;
import io.quarkus.runtime.annotations.ConfigItem;

@ConfigGroup
public class SecretsRoleConfig {

/**
* The name of the role.
*/
@ConfigItem(defaultValue = "view-secrets")
public String name;

/**
* The namespace of the role.
*/
@ConfigItem
public Optional<String> namespace;

/**
* Whether the role is cluster wide or not. By default, it's not a cluster wide role.
*/
@ConfigItem(defaultValue = "false")
public boolean clusterWide;

/**
* If the current role is meant to be generated or not. If not, it will only be used to generate the role binding resource.
*/
@ConfigItem(defaultValue = "true")
public boolean generate;
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import io.quarkus.deployment.pkg.builditem.ArtifactResultBuildItem;
import io.quarkus.deployment.pkg.builditem.OutputTargetBuildItem;
import io.quarkus.deployment.util.ExecUtil;
import io.quarkus.kubernetes.client.spi.KubernetesClientCapabilityBuildItem;
import io.quarkus.kubernetes.deployment.AddPortToKubernetesConfig;
import io.quarkus.kubernetes.deployment.DevClusterHelper;
import io.quarkus.kubernetes.deployment.KubernetesCommonHelper;
Expand All @@ -32,6 +33,7 @@
import io.quarkus.kubernetes.spi.CustomProjectRootBuildItem;
import io.quarkus.kubernetes.spi.DecoratorBuildItem;
import io.quarkus.kubernetes.spi.KubernetesAnnotationBuildItem;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesCommandBuildItem;
import io.quarkus.kubernetes.spi.KubernetesDeploymentTargetBuildItem;
import io.quarkus.kubernetes.spi.KubernetesEnvBuildItem;
Expand All @@ -46,6 +48,7 @@
import io.quarkus.kubernetes.spi.KubernetesResourceMetadataBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;

public class KindProcessor {

Expand Down Expand Up @@ -97,6 +100,7 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
KubernetesConfig config,
PackageConfig packageConfig,
Optional<MetricsCapabilityBuildItem> metricsConfiguration,
Optional<KubernetesClientCapabilityBuildItem> kubernetesClientConfiguration,
List<KubernetesInitContainerBuildItem> initContainers,
List<KubernetesJobBuildItem> jobs,
List<KubernetesAnnotationBuildItem> annotations,
Expand All @@ -111,16 +115,16 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
Optional<KubernetesHealthReadinessPathBuildItem> readinessPath,
Optional<KubernetesHealthStartupPathBuildItem> startupPath,
List<KubernetesRoleBuildItem> roles,
List<KubernetesClusterRoleBuildItem> clusterRoles,
List<KubernetesServiceAccountBuildItem> serviceAccounts,
List<KubernetesRoleBindingBuildItem> roleBindings,
Optional<CustomProjectRootBuildItem> customProjectRoot) {

return DevClusterHelper.createDecorators(KIND, applicationInfo, outputTarget, config, packageConfig,
metricsConfiguration, initContainers, jobs, annotations, labels, envs, baseImage, image, command, ports,
portName,
livenessPath,
readinessPath,
startupPath,
roles, roleBindings, customProjectRoot);
metricsConfiguration, kubernetesClientConfiguration, initContainers, jobs, annotations, labels, envs,
baseImage, image, command, ports, portName,
livenessPath, readinessPath, startupPath,
roles, clusterRoles, serviceAccounts, roleBindings, customProjectRoot);
}

@BuildStep
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
import io.quarkus.deployment.metrics.MetricsCapabilityBuildItem;
import io.quarkus.deployment.pkg.PackageConfig;
import io.quarkus.deployment.pkg.builditem.OutputTargetBuildItem;
import io.quarkus.kubernetes.client.spi.KubernetesClientCapabilityBuildItem;
import io.quarkus.kubernetes.deployment.AddPortToKubernetesConfig;
import io.quarkus.kubernetes.deployment.DevClusterHelper;
import io.quarkus.kubernetes.deployment.KubernetesCommonHelper;
Expand All @@ -29,6 +30,7 @@
import io.quarkus.kubernetes.spi.CustomProjectRootBuildItem;
import io.quarkus.kubernetes.spi.DecoratorBuildItem;
import io.quarkus.kubernetes.spi.KubernetesAnnotationBuildItem;
import io.quarkus.kubernetes.spi.KubernetesClusterRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesCommandBuildItem;
import io.quarkus.kubernetes.spi.KubernetesDeploymentTargetBuildItem;
import io.quarkus.kubernetes.spi.KubernetesEnvBuildItem;
Expand All @@ -43,6 +45,7 @@
import io.quarkus.kubernetes.spi.KubernetesResourceMetadataBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBindingBuildItem;
import io.quarkus.kubernetes.spi.KubernetesRoleBuildItem;
import io.quarkus.kubernetes.spi.KubernetesServiceAccountBuildItem;

public class MinikubeProcessor {

Expand Down Expand Up @@ -93,6 +96,7 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
KubernetesConfig config,
PackageConfig packageConfig,
Optional<MetricsCapabilityBuildItem> metricsConfiguration,
Optional<KubernetesClientCapabilityBuildItem> kubernetesClientConfiguration,
List<KubernetesInitContainerBuildItem> initContainers,
List<KubernetesJobBuildItem> jobs,
List<KubernetesAnnotationBuildItem> annotations,
Expand All @@ -107,15 +111,15 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
Optional<KubernetesHealthReadinessPathBuildItem> readinessPath,
Optional<KubernetesHealthStartupPathBuildItem> startupPath,
List<KubernetesRoleBuildItem> roles,
List<KubernetesClusterRoleBuildItem> clusterRoles,
List<KubernetesServiceAccountBuildItem> serviceAccounts,
List<KubernetesRoleBindingBuildItem> roleBindings,
Optional<CustomProjectRootBuildItem> customProjectRoot) {

return DevClusterHelper.createDecorators(MINIKUBE, applicationInfo, outputTarget, config, packageConfig,
metricsConfiguration, initContainers, jobs, annotations, labels, envs, baseImage, image, command, ports,
portName,
livenessPath,
readinessPath,
startupPath,
roles, roleBindings, customProjectRoot);
metricsConfiguration, kubernetesClientConfiguration, initContainers, jobs, annotations, labels, envs,
baseImage, image, command, ports, portName,
livenessPath, readinessPath, startupPath,
roles, clusterRoles, serviceAccounts, roleBindings, customProjectRoot);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package io.quarkus.kubernetes.spi;

import java.util.List;

import io.quarkus.builder.item.MultiBuildItem;

/**
* Produce this build item to request the Kubernetes extension to generate
* a Kubernetes {@code ClusterRole} resource.
*/
public final class KubernetesClusterRoleBuildItem extends MultiBuildItem {
/**
* Name of the generated {@code ClusterRole} resource.
*/
private final String name;
/**
* The {@code PolicyRule} resources for this {@code ClusterRole}.
*/
private final List<PolicyRule> rules;

/**
* The target manifest that should include this role.
*/
private final String target;

public KubernetesClusterRoleBuildItem(String name, List<PolicyRule> rules, String target) {
this.name = name;
this.rules = rules;
this.target = target;
}

public String getName() {
return name;
}

public List<PolicyRule> getRules() {
return rules;
}

public String getTarget() {
return target;
}
}
Loading

0 comments on commit 0ad1690

Please sign in to comment.