Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support to prefix for envFrom #39782

Merged
merged 1 commit into from
Jun 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion bom/application/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@
<kotlin.coroutine.version>1.8.1</kotlin.coroutine.version>
<azure.toolkit-lib.version>0.27.0</azure.toolkit-lib.version>
<kotlin-serialization.version>1.7.0</kotlin-serialization.version>
<dekorate.version>4.1.2</dekorate.version> <!-- Please check with Java Operator SDK team before updating -->
<dekorate.version>4.1.3</dekorate.version> <!-- Please check with Java Operator SDK team before updating -->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gsmet, should this change be on separated pull request?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@metacosm I assume the version bump is fine with you?

Copy link
Contributor

@metacosm metacosm Jun 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't know and didn't get a chance to check… That version of dekorate uses an older sundrio version than the one used in the Kubernetes client and I have no idea about compatibility. I would have preferred this upgrade to be isolated from this PR if at all possible and be consulted before merging it, as is mentioned in the POM file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough.

When you get please chance, please check the version and see if everything works as expected.

Thanks!

Copy link
Contributor Author

@mcruzdev mcruzdev Jun 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was necessary because dekorate did not support prefix for env, I had to implement this on the dekorate side.

dekorateio/dekorate#1289

Without this version, this PR breaks.

<maven-invoker.version>3.2.0</maven-invoker.version>
<awaitility.version>4.2.1</awaitility.version>
<jboss-logmanager.version>3.0.6.Final</jboss-logmanager.version>
Expand Down
37 changes: 37 additions & 0 deletions docs/src/main/asciidoc/deploying-to-kubernetes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -467,9 +467,28 @@
optional: false
----

It is also possible to add a prefix when you are generating env from Secret, the following configuration creates environment variable from Secret with key `foo` adding a prefix `BAR`:

Check warning on line 470 in docs/src/main/asciidoc/deploying-to-kubernetes.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.Spelling] Use correct American English spelling. Did you really mean 'env'? Raw Output: {"message": "[Quarkus.Spelling] Use correct American English spelling. Did you really mean 'env'?", "location": {"path": "docs/src/main/asciidoc/deploying-to-kubernetes.adoc", "range": {"start": {"line": 470, "column": 61}}}, "severity": "WARNING"}


[source,properties]
----
quarkus.kubernetes.env.secrets=foo
quarkus.kubernetes.env.using-prefix."BAR".for-secret=foo
----

This would generate the following in the `env` section of your container:
[source,yaml]
----
- env:
envFrom:
- secretRef:
name: foo
prefix: BAR
----

==== Environment variables from ConfigMap

To add all key/value pairs from `ConfigMap` as environment variables just apply the following configuration, separating each

Check warning on line 491 in docs/src/main/asciidoc/deploying-to-kubernetes.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'. Raw Output: {"message": "[Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'.", "location": {"path": "docs/src/main/asciidoc/deploying-to-kubernetes.adoc", "range": {"start": {"line": 491, "column": 34}}}, "severity": "INFO"}
`ConfigMap` to be used as source by a comma (`,`):

[source,properties]
Expand Down Expand Up @@ -512,9 +531,27 @@
optional: false
----

It is also possible to add a prefix when you are generating env from ConfigMap, the following configuration creates environment variable from ConfigMap with key `foo` adding a prefix `BAR`:

Check warning on line 534 in docs/src/main/asciidoc/deploying-to-kubernetes.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.Spelling] Use correct American English spelling. Did you really mean 'env'? Raw Output: {"message": "[Quarkus.Spelling] Use correct American English spelling. Did you really mean 'env'?", "location": {"path": "docs/src/main/asciidoc/deploying-to-kubernetes.adoc", "range": {"start": {"line": 534, "column": 61}}}, "severity": "WARNING"}

[source,properties]
----
quarkus.kubernetes.env.configmaps=foo
quarkus.kubernetes.prefixes."BAR".for-configmap=foo
----

This would generate the following in the `env` section of your container:
[source,yaml]
----
- env:
envFrom:
- configMapRef:
name: foo
prefix: BAR
----

==== Environment variables from fields

It's also possible to use the value from another field to add a new environment variable by specifying the path of the field to be used as a source, as follows:

Check warning on line 554 in docs/src/main/asciidoc/deploying-to-kubernetes.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'. Raw Output: {"message": "[Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'.", "location": {"path": "docs/src/main/asciidoc/deploying-to-kubernetes.adoc", "range": {"start": {"line": 554, "column": 137}}}, "severity": "INFO"}

[source,properties]
----
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,42 +54,45 @@ public boolean mightConflictWith(EnvType type) {
private final EnvType type;
private final String target;
private final boolean oldStyle;
private final String prefix;

public static KubernetesEnvBuildItem createFromField(String name, String targetField, String target,
boolean... oldStyle) {
return create(name, null, null, null, targetField, target, isOldStyle(oldStyle));
return create(name, null, null, null, targetField, target, null, isOldStyle(oldStyle));
}

public static KubernetesEnvBuildItem createFromConfigMap(String configMapName, String target, boolean... oldStyle) {
return create(configMapName, null, null, configMapName, null, target, isOldStyle(oldStyle));
public static KubernetesEnvBuildItem createFromConfigMap(String configMapName, String target, String prefix,
boolean... oldStyle) {
return create(configMapName, null, null, configMapName, null, target, prefix, isOldStyle(oldStyle));
}

public static KubernetesEnvBuildItem createFromSecret(String secretName, String target, boolean... oldStyle) {
return create(secretName, null, secretName, null, null, target, isOldStyle(oldStyle));
public static KubernetesEnvBuildItem createFromSecret(String secretName, String target, String prefix,
boolean... oldStyle) {
return create(secretName, null, secretName, null, null, target, prefix, isOldStyle(oldStyle));
}

public static KubernetesEnvBuildItem createSimpleVar(String name, String value, String target,
boolean... oldStyle) {
return create(name, value, null, null, null, target, isOldStyle(oldStyle));
return create(name, value, null, null, null, target, null, isOldStyle(oldStyle));
}

public static KubernetesEnvBuildItem createFromConfigMapKey(String varName, String key, String configmap, String target,
boolean... oldStyle) {
return create(varName, key, null, configmap, null, target, isOldStyle(oldStyle));
String prefix, boolean... oldStyle) {
return create(varName, key, null, configmap, null, target, prefix, isOldStyle(oldStyle));
}

public static KubernetesEnvBuildItem createFromSecretKey(String varName, String key, String secret, String target,
boolean... oldStyle) {
return create(varName, key, secret, null, null, target, isOldStyle(oldStyle));
String prefix, boolean... oldStyle) {
return create(varName, key, secret, null, null, target, prefix, isOldStyle(oldStyle));
}

public static KubernetesEnvBuildItem createFromResourceKey(String varName, String key, String secret,
String configmap, String target, boolean... oldStyle) {
return create(varName, key, secret, configmap, null, target, isOldStyle(oldStyle));
return create(varName, key, secret, configmap, null, target, null, isOldStyle(oldStyle));
}

public static KubernetesEnvBuildItem create(String name, String value, String secret, String configmap, String field,
String target, boolean... oldStyle) throws IllegalArgumentException {
String target, String prefix, boolean... oldStyle) throws IllegalArgumentException {
final boolean secretPresent = secret != null;
final boolean configmapPresent = configmap != null;
final boolean valuePresent = value != null;
Expand Down Expand Up @@ -134,22 +137,23 @@ public static KubernetesEnvBuildItem create(String name, String value, String se
} else {
type = EnvType.var;
}
return new KubernetesEnvBuildItem(name, value, configmap, secret, field, type, target, isOldStyle(oldStyle));
return new KubernetesEnvBuildItem(name, value, configmap, secret, field, type, target, prefix, isOldStyle(oldStyle));
}

private static boolean isOldStyle(boolean[] oldStyle) {
return oldStyle.length >= 1 && oldStyle[0];
}

KubernetesEnvBuildItem(String name, String value, String configmap, String secret, String field, EnvType type,
String target, boolean oldStyle) {
String target, String prefix, boolean oldStyle) {
this.name = name;
this.value = value;
this.configmap = configmap;
this.secret = secret;
this.field = field;
this.type = type;
this.target = target;
this.prefix = prefix;
this.oldStyle = oldStyle;
}

Expand Down Expand Up @@ -185,9 +189,13 @@ public String getTarget() {
return target;
}

public String getPrefix() {
return prefix;
}

public KubernetesEnvBuildItem newWithTarget(String newTarget) {
return new KubernetesEnvBuildItem(this.name, this.value, this.configmap, this.secret, this.field, this.type, newTarget,
this.oldStyle);
this.prefix, this.oldStyle);
}

public String toString() {
Expand Down Expand Up @@ -228,6 +236,8 @@ public boolean equals(Object o) {
return false;
if (field != null ? !field.equals(that.field) : that.field != null)
return false;
if (prefix != null ? !prefix.equals(that.prefix) : that.prefix != null)
return false;
return type == that.type;
}

Expand All @@ -239,6 +249,7 @@ public int hashCode() {
result = 31 * result + (secret != null ? secret.hashCode() : 0);
result = 31 * result + (field != null ? field.hashCode() : 0);
result = 31 * result + type.hashCode();
result = 31 * result + (prefix != null ? prefix.hashCode() : 0);
return result;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@

import static io.quarkus.kubernetes.spi.KubernetesEnvBuildItem.create;
import static io.quarkus.kubernetes.spi.KubernetesEnvBuildItem.EnvType.configmap;
import static io.quarkus.kubernetes.spi.KubernetesEnvBuildItem.EnvType.secret;
import static io.quarkus.kubernetes.spi.KubernetesEnvBuildItem.EnvType.var;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
Expand All @@ -29,10 +30,11 @@ public class KubernetesEnvBuildItemTest {
private static final String TARGET = "target";
private static final String VALUE = "value";
private static final String NAME = "name";
private static final String PREFIX = "prefix";

@Test
public void testCreateSimpleVarFromEnvConfig() {
final KubernetesEnvBuildItem item = create(NAME, VALUE, null, null, null, TARGET);
final KubernetesEnvBuildItem item = create(NAME, VALUE, null, null, null, TARGET, null);
assertEquals(var, item.getType());
assertEquals(NAME, item.getName());
assertEquals(VALUE, item.getValue());
Expand All @@ -44,12 +46,36 @@ public void testCreateSimpleVarFromEnvConfig() {

@Test
public void testCreateLoadFromConfigMapFromEnvConfig() {
final KubernetesEnvBuildItem item = create(NAME, null, null, VALUE, null, TARGET);
final KubernetesEnvBuildItem item = create(NAME, null, null, VALUE, null, TARGET, null);
assertEquals(configmap, item.getType());
assertEquals(VALUE, item.getName());
assertNull(item.getValue());
assertEquals(VALUE, item.getConfigMap());
assertNull(item.getSecret());
assertNull(item.getField());
}

@Test
public void testCreateConfigMapWithPrefix() {
final KubernetesEnvBuildItem item = create(NAME, null, null, VALUE, null, TARGET, PREFIX);
assertEquals(configmap, item.getType());
assertEquals(VALUE, item.getName());
assertNull(item.getValue());
assertEquals(VALUE, item.getConfigMap());
assertNull(item.getSecret());
assertNull(item.getField());
assertEquals(PREFIX, item.getPrefix());
}

@Test
public void testCreateSecretWithPrefix() {
final KubernetesEnvBuildItem item = create(NAME, null, VALUE, null, null, TARGET, PREFIX);
assertEquals(secret, item.getType());
assertEquals(VALUE, item.getName());
assertNull(item.getValue());
assertEquals(VALUE, item.getSecret());
assertNull(item.getConfigMap());
assertNull(item.getField());
assertEquals(PREFIX, item.getPrefix());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ public String getTargetPlatformName() {
public Collection<Env> convertToEnvs() {
return convertToBuildItems().stream()
.map(kebi -> new Env(EnvConverter.convertName(kebi.getName()), kebi.getValue(), kebi.getSecret(),
kebi.getConfigMap(), kebi.getField(), null))
kebi.getConfigMap(), kebi.getField(), null, kebi.getPrefix()))
.collect(Collectors.toList());
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ public static List<DecoratorBuildItem> createDecorators(String clusterKind,
.withSecret(e.getSecret())
.withConfigmap(e.getConfigMap())
.withField(e.getField())
.withPrefix(e.getPrefix())
.build())));
});

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,14 @@
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;

import io.dekorate.kubernetes.config.Env;
import io.dekorate.kubernetes.config.EnvBuilder;

public class EnvConverter {

public static Env convert(Map.Entry<String, EnvConfig> e) {
return convert(e.getValue()).withName(convertName(e.getKey())).build();
}
Expand All @@ -25,9 +28,16 @@ private static EnvBuilder convert(EnvConfig env) {

public static List<Env> convert(EnvVarsConfig e) {
List<Env> envs = new LinkedList<>();
e.secrets.ifPresent(sl -> sl.forEach(s -> envs.add(new EnvBuilder().withName(convertName(s)).withSecret(s).build())));

Map<String, EnvVarPrefixConfig> prefixMap = collectPrefixes(e);

e.secrets.ifPresent(sl -> sl
.forEach(s -> envs.add(new EnvBuilder().withName(convertName(s)).withSecret(s)
.withPrefix(extractSecretPrefix(s, prefixMap).orElse(null)).build())));
e.configmaps
.ifPresent(cl -> cl.forEach(c -> envs.add(new EnvBuilder().withName(convertName(c)).withConfigmap(c).build())));
.ifPresent(cl -> cl.forEach(c -> envs
.add(new EnvBuilder().withName(convertName(c)).withConfigmap(c)
.withPrefix(extractConfigmapPrefix(c, prefixMap).orElse(null)).build())));
e.vars.forEach((k, v) -> envs.add(new EnvBuilder().withName(convertName(k)).withValue(v.orElse("")).build()));
e.fields.forEach((k, v) -> {
// env vars from fields need to have their name set in addition to their field field :)
Expand All @@ -43,4 +53,22 @@ public static List<Env> convert(EnvVarsConfig e) {
public static String convertName(String name) {
return name != null ? name.toUpperCase().replace('-', '_').replace('.', '_').replace('/', '_') : null;
}

public static Optional<String> extractSecretPrefix(String secret, Map<String, EnvVarPrefixConfig> mappingWithPrefix) {
return mappingWithPrefix.entrySet().stream()
.filter(m -> m.getValue().hasPrefixForSecret(secret))
.findFirst().map(Map.Entry::getKey);
}

public static Optional<String> extractConfigmapPrefix(String configmap,
Map<String, EnvVarPrefixConfig> mappingWithPrefix) {
return mappingWithPrefix.entrySet().stream()
.filter(m -> m.getValue().hasPrefixForConfigmap(configmap))
.findFirst().map(Map.Entry::getKey);
}

public static Map<String, EnvVarPrefixConfig> collectPrefixes(EnvVarsConfig e) {
return e.prefixes.entrySet().stream().filter(p -> p.getValue().anyPresent() && !p.getKey().isBlank())
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,12 @@
package io.quarkus.kubernetes.deployment;

import static io.quarkus.kubernetes.deployment.EnvConverter.collectPrefixes;
import static io.quarkus.kubernetes.deployment.EnvConverter.extractConfigmapPrefix;
import static io.quarkus.kubernetes.deployment.EnvConverter.extractSecretPrefix;

import java.util.Collection;
import java.util.Map;
import java.util.Optional;

import io.quarkus.kubernetes.spi.KubernetesEnvBuildItem;

Expand Down Expand Up @@ -43,16 +48,21 @@ default Collection<KubernetesEnvBuildItem> convertToBuildItems() {
final String target = getTargetPlatformName();
getEnvVars().forEach((key, envConfig) -> {
validator.process(key, envConfig.value, envConfig.secret, envConfig.configmap, envConfig.field, target,
true);
Optional.empty(), true);
});

// override old-style with newer versions if present
final EnvVarsConfig c = getEnv();
Map<String, EnvVarPrefixConfig> prefixMap = collectPrefixes(c);

c.vars.forEach((k, v) -> validator.process(KubernetesEnvBuildItem.createSimpleVar(k, v.orElse(""), target)));
c.fields.forEach((k, v) -> validator.process(KubernetesEnvBuildItem.createFromField(k, v, target)));
c.configmaps
.ifPresent(cl -> cl.forEach(cm -> validator.process(KubernetesEnvBuildItem.createFromConfigMap(cm, target))));
c.secrets.ifPresent(sl -> sl.forEach(s -> validator.process(KubernetesEnvBuildItem.createFromSecret(s, target))));
.ifPresent(
cl -> cl.forEach(cm -> validator.process(KubernetesEnvBuildItem.createFromConfigMap(cm,
target, extractConfigmapPrefix(cm, prefixMap).orElse(null)))));
c.secrets.ifPresent(sl -> sl.forEach(s -> validator.process(KubernetesEnvBuildItem.createFromSecret(s,
target, extractSecretPrefix(s, prefixMap).orElse(null)))));
c.mapping.forEach(
(varName, config) -> validator.process(KubernetesEnvBuildItem.createFromResourceKey(varName, config.withKey,
config.fromSecret.orElse(null), config.fromConfigmap.orElse(null), target)));
Expand Down
Loading
Loading