java.io.IOException: DerInputStream.getLength(): lengthTag=111, too big.

[droslean]

[2024-03-07 10:17:49,331][main] WARN OpenUnisonOnUndertow - Could not create PKCS12 from /etc/openunison/unisonKeyStore.p12, falling back to JCEKS
java.io.IOException: DerInputStream.getLength(): lengthTag=111, too big.
at java.base/sun.security.util.DerInputStream.getLength(DerInputStream.java:589) ~[?:?]
at java.base/sun.security.util.DerValue.init(DerValue.java:411) ~[?:?]
at java.base/sun.security.util.DerValue.(DerValue.java:352) ~[?:?]
at java.base/sun.security.util.DerValue.(DerValue.java:365) ~[?:?]
at java.base/sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:1949) ~[?:?]
at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:222) ~[?:?]
at java.base/java.security.KeyStore.load(KeyStore.java:1479) ~[?:?]
at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:530) [openunison-on-undertow-1.0.39.jar:?]
at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280) [openunison-on-undertow-1.0.39.jar:?]
Exception in thread "main" java.io.IOException: Invalid keystore format
at java.base/com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:725)
at java.base/java.security.KeyStore.load(KeyStore.java:1479)
at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:536)
at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:280)

We didn't change anything. We didn't update. This just only happened.

Related to the openunison-orchestra

/cc @mlbiam

[droslean]

Forcing the openunison CR to update solved the issue. But this needs to be addressed since its a BUG from Openunison side.

[mlbiam]

[2024-03-07 10:17:49,331][main] WARN OpenUnisonOnUndertow - Could not create PKCS12 from /etc/openunison/unisonKeyStore.p12, falling back to JCEKS

Were there any exceptions from before this in the log? This generally happens because the operator wasn't able to generate the orchestra Secret or there was a problem during its generation. Do you have the logs from the operator pod? Were there any exceptions?

[droslean]

That error is coming from the orchestra pod. I don't see any error on the operator. Note that nothing has changed from our side.

[mlbiam]

That error is coming from the orchestra pod.

Right, because the orchestra pod is reliant on the secret generated by the operator. If the operator didn't generate the secret correctly there is going to be an issue. Do you have the original logs from the orchestra pod that crashed? There should be an exception before the original one posted that gives the root cause as to why the keystore couldn't be loaded as a p12 and openunison tried falling back to JCEKS.

[droslean]

That error is coming from the orchestra pod.

Right, because the orchestra pod is reliant on the secret generated by the operator. If the operator didn't generate the secret correctly there is going to be an issue. Do you have the original logs from the orchestra pod that crashed? There should be an exception before the original one posted that gives the root cause as to why the keystore couldn't be loaded as a p12 and openunison tried falling back to JCEKS.

The log in the description is the error from the original pod.

[droslean]

@mlbiam Crashed again:

[2024-03-12 13:28:45,630][main] WARN  OpenUnisonOnUndertow - Could not create PKCS12 from /etc/openunison/unisonKeyStore.p12, falling back to JCEKS
java.io.IOException: DerInputStream.getLength(): lengthTag=111, too big.
	at java.base/sun.security.util.DerInputStream.getLength(DerInputStream.java:589) ~[?:?]
	at java.base/sun.security.util.DerValue.init(DerValue.java:411) ~[?:?]
	at java.base/sun.security.util.DerValue.<init>(DerValue.java:352) ~[?:?]
	at java.base/sun.security.util.DerValue.<init>(DerValue.java:365) ~[?:?]
	at java.base/sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:1949) ~[?:?]
	at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:222) ~[?:?]
	at java.base/java.security.KeyStore.load(KeyStore.java:1479) ~[?:?]
	at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:534) [openunison-on-undertow-1.0.40.jar:?]
	at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:284) [openunison-on-undertow-1.0.40.jar:?]
Exception in thread "main" java.io.IOException: Invalid keystore format
	at java.base/com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:725)
	at java.base/java.security.KeyStore.load(KeyStore.java:1479)
	at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.setupTlsListener(OpenUnisonOnUndertow.java:540)
	at com.tremolosecurity.openunison.undertow.OpenUnisonOnUndertow.main(OpenUnisonOnUndertow.java:284)

After forcing to update, everything worked fine, but now it crashed again. This is 100% a bug

[mlbiam]

There should be an error before that too in the logs? Also anything in the operator logs?

[droslean]

There should be an error before that too in the logs? Also anything in the operator logs?

No other errors. This is the only error we get, which makes the orchestra pod to crashloop.

[mlbiam]

What versions are the charts and containers?

[droslean]

Operator is ghcr.io/openunison/openunison-kubernetes-operator:1.0.4

All the rest are the latest.

[droslean]

@mlbiam Are there any updates?