Malformed exception when using binary config map file into an integration

[squakez]

If I try to use a binary file coming from a config map, the runtime fails with following stacktrace:

[1] Exception in thread "main" java.lang.ExceptionInInitializerError
[1] 	at io.quarkus.runner.ApplicationImpl.<clinit>(ApplicationImpl.zig:47)
[1] 	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[1] 	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
[1] 	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
[1] 	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
[1] 	at java.base/java.lang.Class.newInstance(Class.java:584)
[1] 	at io.quarkus.runtime.Quarkus.run(Quarkus.java:61)
[1] 	at io.quarkus.runtime.Quarkus.run(Quarkus.java:38)
[1] 	at io.quarkus.runtime.Quarkus.run(Quarkus.java:104)
[1] 	at io.quarkus.runner.GeneratedMain.main(GeneratedMain.zig:29)
[1] Caused by: java.lang.RuntimeException: java.nio.charset.MalformedInputException: Input length = 1
[1] 	at org.apache.camel.k.support.RuntimeSupport.loadUserProperties(RuntimeSupport.java:379)
[1] 	at org.apache.camel.k.quarkus.ApplicationConfigSourceProvider.getConfigSources(ApplicationConfigSourceProvider.java:31)
[1] 	at io.smallrye.config.SmallRyeConfigBuilder.discoverSources(SmallRyeConfigBuilder.java:97)
[1] 	at io.smallrye.config.SmallRyeConfig.buildConfigSources(SmallRyeConfig.java:79)
[1] 	at io.smallrye.config.SmallRyeConfig.<init>(SmallRyeConfig.java:62)
[1] 	at io.smallrye.config.SmallRyeConfigBuilder.build(SmallRyeConfigBuilder.java:359)
[1] 	at io.quarkus.runtime.generated.Config.<clinit>(Config.zig:287)
[1] 	... 10 more
[1] Caused by: java.nio.charset.MalformedInputException: Input length = 1
[1] 	at java.base/java.lang.StringCoding.throwMalformed(StringCoding.java:685)
[1] 	at java.base/java.lang.StringCoding.decodeUTF8_0(StringCoding.java:872)
[1] 	at java.base/java.lang.StringCoding.newStringNoRepl1(StringCoding.java:1005)
[1] 	at java.base/java.lang.StringCoding.newStringNoRepl(StringCoding.java:990)
[1] 	at java.base/java.lang.System$2.newStringNoRepl(System.java:2197)
[1] 	at java.base/java.nio.file.Files.readString(Files.java:3287)
[1] 	at org.apache.camel.k.support.RuntimeSupport$1.visitFile(RuntimeSupport.java:366)
[1] 	at org.apache.camel.k.support.RuntimeSupport$1.visitFile(RuntimeSupport.java:347)
[1] 	at java.base/java.nio.file.Files.walkFileTree(Files.java:2725)
[1] 	at java.base/java.nio.file.Files.walkFileTree(Files.java:2797)
[1] 	at org.apache.camel.k.support.RuntimeSupport.loadUserProperties(RuntimeSupport.java:377)
[1] 	... 16 more

I think this is because we are expecting the config map to be always a text file.

Steps to reproduce

1. Create a text file with some text inside

printf "hello from test" > data.txt

2. Zip the file in order to get a binary

zip data.zip data.txt

3. Create a configmap from the zipped file

kubectl create configmap testcm --from-file data.zip -n builder-test

4. Create a simple integration to try to use the zipped content integration.groovy

from('timer:resources')
    .routeId('resources')
    .setBody()
        .simple("resource:classpath:data.zip")
    .unmarshal().zipFile()
    .log('file content is: ${body}')

5. Run the integration

kamel run integration.groovy --configmap testcm -d camel-zipfile

[lburgazzoli]

I think the main issue is that we don't have a way to distinguish between configmaps and secrets used to store properties vs resources.

[makandre]

I'm running into this issue as well. For my use case I created a Secret with an application.properties and a truststore.jks in it, and my integration used to be able to mount this Secret via the --secret flag. I think it last worked with camel-k 1.2 and now I get the above exception in 1.3

Update: since I couldn't downgrade my camel-k to 1.2, I tried my scenario again with the RedHat camel-k operator which is 1.2, and I can confirm it work with that but fails with camel-k 1.3

[squakez]

Within the possibility to use binary resources directly into an integration, I think this issue can be closed.

[makandre]

@squakez Do resources retain the file name in the mount path?

Because for sources they are mounted like /etc/camel/sources/i-source-000 from what I can tell. For my use case I need to mount a truststore.jks into the integration and I need to pass that path to the Kafka component.

[squakez]

File name will be the same, not the path, the behavior is the same as for Sources. You can expect the mount point to be /etc/camel/resources/i-resource-xyz, altough this should be transparent for your application. If you're writing the integration in Java, for example, consider that those resource paths will be included in the application classpath.

[lburgazzoli]

@squakez I remember configmaps were mounted on a child folder of $CAMEL_K_MOUNT_PATH_CONFIGMAPS, isn't it ? /etc/camel/resources should be for the resources defined through --resources

[squakez]

@lburgazzoli that's correct, the default is that path. I was thinking that if one has a specific need, he can edit the integration definition by setting a concrete mountPath. For example:

apiVersion: camel.apache.org/v1
kind: Integration
metadata:
  name: resources-route-edited
  namespace: default
spec:
  resources:
  - content: the file body
    name: resources-data.txt
    type: data
    mountPath: /tmp/my-edited-path/
  sources:
  - content: |
      // camel-k: language=groovy
      from('timer:resources')
          .routeId('resources')
          .setBody()
              .simple("resource:classpath:resources-data.txt")
          .log('file content is: ${body}')
    name: resources-route.groovy

kubectl apply -f integration-edited.yaml
integration.camel.apache.org/resources-route-edited created
...
kubectl exec -it resources-route-edited-86dc56d46f-sz4b4 -- cat /tmp/my-edited-path/resources-data.txt
the file body

The same should apply to any binary file. Also here, that path will be included in your application classpath, but it can be useful if a file is expected to be in any particular place. @makandre, hope that helps.

[lburgazzoli]

I don't remember if that options to set the mnount path is still available for config maps and secrets in general,

[makandre]

I like to avoid having to manually editing the integration yaml. So if I don't change the mount path, it defaults to /etc/camel/resources/i-resource-xyz where xyz is an arbitrary number?

Because I need to pass the truststore location to the kafka component like so:

camel.component.kafka.sslTruststoreLocation=path_to_truststore

So if the truststore.jks passed via --resources gets mounted with an arbitrary path, how can I pass it to kafka?

[squakez]

My bad, xyz is the index of the resource starting from 000. You can expect those index to be the same as the --resources you will provide.

[lburgazzoli]

As far as I remember if you use a config map, the path should be:

{{env:CAMEL_K_MOUNT_PATH_CONFIGMAPS}}/config-map-name/key

@squakez maybe we can add an option to the run command, like

kamel run --config-map name[/key][@path]

where

• name is the name of the config map
• key is an optional key
• path is the mount point of the config map

[makandre]

@lburgazzoli yes I'm basically using the method you described to pass my mounted secret's path to Kafka component (in camel-k 1.2)

But since binary file in configmap/secret is broken now in camel-k 1.3, and @squakez was suggesting I can mount my binary file using --resource instead. So my comments and concerns are around this; I just wanted to verify that the mount path of the resource is not arbitrary so it can be passed to Kafka component.

It sounds to me that the mount path of resource does not retain the original file name, however it will be in sequential order (i.e. the first --resource will end with 000, the second ends with 001). Is that correct @squakez?

[squakez]

That's correct. You can rely on that if your resource is always positioned in the same order or edit the integration and provide a mountPath.

[astefanutti]

I wonder if we could explore mounting resources ConfigMap using SubPath. As opposed to mounting them as volumes, it's possible to project keys into an existing directory, which could alleviate the need to generate unique directory paths for each resource.

The only side-effect of using SubPath is that updates to the ConfigMap are not automatically projected into the containers. But I don't think that's an issue for Camel K.

[lburgazzoli]

@astefanutti I remember sub path was not supported by knative thus I had to switch to this "pattern". We should investigate if it is still true and if it is, check with the knative dev if they can enable it

[astefanutti]

@lburgazzoli thanks a lot. It seems it's been fixed in Knative: knative/serving#3926.

Let me create an issue in Camel K to try it out.