Unable to obtain configuration for Secret objects from Kubernetes API Server in Funqy apps

[danieloh30]

Describe the bug

(Describe the problem clearly and concisely.)

Funqy application can't refer to the secret in Kubernetes/Openshift cluster via Quarkus Kubernetes Config extension. When I deploy a normal pod(i.e. RestEasy) rather than knative service, the Quarkus app can obtain the secret details properly.

For a workaround, I have to grant admin permission to SA to access the secret manually like below command:

oc policy add-role-to-user admin system:serviceaccount:quarkus-func-sec:default

Or I need to create a custom SA to access the secret. However, the role binding works when I deploy a normal pod.

Expected behavior

(Describe the expected behavior clearly and concisely.)

Obtain the secret resource then Quarkus runs properly

Actual behavior

(Describe the actual behavior clearly and concisely.)

Failed to obtain configuration for Secret objects from Kubernetes API Server

ERROR: Failed to start application (with profile prod)
java.lang.RuntimeException: Unable to obtain configuration for Secret objects from Kubernetes API Server at: https://172.30.0.1:443/
    at io.quarkus.kubernetes.client.runtime.KubernetesConfigSourceProvider.getSecretConfigSources(KubernetesConfigSourceProvider.java:119)
    at io.quarkus.kubernetes.client.runtime.KubernetesConfigSourceProvider.getConfigSources(KubernetesConfigSourceProvider.java:48)
    at io.quarkus.runtime.configuration.ConfigUtils.addSourceProvider(ConfigUtils.java:107)
    at io.quarkus.runtime.configuration.ConfigUtils.addSourceProviders(ConfigUtils.java:121)
    at io.quarkus.runtime.generated.Config.readConfig(Config.zig:1302)
    at io.quarkus.deployment.steps.RuntimeConfigSetup.deploy(RuntimeConfigSetup.zig:60)
    at io.quarkus.runner.ApplicationImpl.doStart(ApplicationImpl.zig:367)
    at io.quarkus.runtime.Application.start(Application.java:90)
    at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:91)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:61)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:38)
    at io.quarkus.runtime.Quarkus.run(Quarkus.java:106)
    at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://172.30.0.1/api/v1/namespaces/quarkus-func-sec/secrets/db-credentials. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. secrets "db-credentials" is forbidden: User "system:serviceaccount:quarkus-func-sec:default" cannot get resource "secrets" in API group "" in the namespace "quarkus-func-sec".
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:589)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:526)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:492)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:451)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:416)
    at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:397)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:890)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:233)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:187)
    at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:79)
    at io.quarkus.kubernetes.client.runtime.KubernetesConfigSourceProvider.getSecretConfigSources(KubernetesConfigSourceProvider.java:105)
    ... 12 more

To Reproduce

Link to a small reproducer (preferably a Maven project if the issue is not Gradle-specific).

Or attach an archive containing the reproducer to the issue.

Steps to reproduce the behavior:

1. Create a secret in OpenShift 4.7 with

kubectl create secret generic db-credentials \
        --from-literal=username=admin \
        --from-literal=password=secret

2. Add a function method:

    @ConfigProperty(name ="username")
    String username;
    
    @ConfigProperty(name="password")
    String password;

    @Funq
    public Map<String, String> secrets() {
        HashMap<String,String> map = new HashMap<>();
        map.put("db.username", username);
        map.put("db.password", password);
        return map;
    }

3. Add the below properties:

quarkus.kubernetes-config.secrets.enabled=true
quarkus.kubernetes-config.secrets=db-credentials

4. Deploy to OpenShift via kn func deploy or mvn package with openshift extension

Configuration

# Add your application.properties here, if applicable.
quarkus.funqy.export=secrets
quarkus.kubernetes-config.secrets=db-credentials
quarkus.kubernetes-config.secrets.enabled=true

Screenshots

(If applicable, add screenshots to help explain your problem.)

Environment (please complete the following information):

Output of uname -a or ver

Output of java -version

GraalVM version (if different from Java)

Quarkus version or git rev

1.11.2.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Additional context

(Add any other context about the problem here.)

[quarkus-bot]

/cc @geoand, @matejvasek, @patriot1burke

[geoand]

So essentially what you are saying is that the Role and RoleBinding are not properly generated when deploying knative applications?

[geoand]

Cc @iocanel

[iocanel]

I will look at it ASAP

[geoand]

Thanks!

[iocanel]

The issue has to be fixed in the kubernetes-client.
I'll raise an issue and take fix it there.

[danieloh30]

@geoand Yeah, I think so because it works when I do RoleBinding manually.