Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Config: injection verification during the native image build #36169

Closed
wants to merge 2 commits into from

Conversation

mkouba
Copy link
Contributor

@mkouba mkouba commented Sep 26, 2023

  • verify the current ImageMode when a config object is injected
  • fail if injected during the static initialization phase of a native image build

See also discussion in #36125

The error message looks like:

========================================================================================================================
POSSIBLE CONFIG INJECTION PROBLEM DETECTED
------------------------------------------------------------------------------------------------------------------------
A config object was injected during the static initialization phase of a native image build.
This may result in unexpected errors.
The injected value was obtained at native image build time and cannot be updated at runtime.

Injection point: org.acme.config.Failure#init()
Solutions:
	- If that's intentional then annotate the injected field/parameter with @io.quarkus.arc.config.NativeBuildTime to eliminate the false positive
	- You can leverage the programmatic lookup to delay the injection of a config property; for example '@ConfigProperty(name = "foo") Instance<String> foo'
	- You can try to use a normal CDI scope to initialize the bean lazily; this may help if the is only injected but not directly used during the static initialization phase
========================================================================================================================

@quarkus-bot quarkus-bot bot added the area/arc Issue related to ARC (dependency injection) label Sep 26, 2023
@mkouba mkouba force-pushed the native-config-inject-validation branch 2 times, most recently from 9b28d2a to af8517b Compare September 27, 2023 12:06
@mkouba mkouba changed the title Config: native build verification Config: injection verification during the native image build Sep 27, 2023
@mkouba mkouba force-pushed the native-config-inject-validation branch from af8517b to 07bcc67 Compare September 27, 2023 12:18
@mkouba mkouba marked this pull request as ready for review September 27, 2023 12:18
@quarkus-bot

This comment has been minimized.

@mkouba mkouba requested review from Ladicek and manovotn September 27, 2023 12:33
@mkouba mkouba force-pushed the native-config-inject-validation branch from 07bcc67 to fa88efe Compare September 27, 2023 13:02
@quarkus-bot

This comment has been minimized.

@mkouba mkouba force-pushed the native-config-inject-validation branch from fa88efe to 97e8a56 Compare September 27, 2023 16:49
@quarkus-bot

This comment has been minimized.

Copy link
Member

@yrodiere yrodiere left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! It looks like a nice improvement. I added a few comments below, hopefully not all of them are silly questions :)

@@ -23,6 +24,10 @@ public Object create(SyntheticCreationalContext<Object> context) {
throw new IllegalStateException("No current injection point found");
}

if ((boolean) context.getParams().get("nativeBuild")) {
NativeBuildConfigCheckInterceptor.verifyCurrentImageMode(Set.of());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC the Set is empty here because we assume none of the properties being injected are "build and runtime fixed"... but can't a config mapping be completely "build and runtime fixed"? If so we'd want to skip this check, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, a ConfigRoot defined as a config mapping is something I need to check.

Because we do register a synthetic bean for any BUILD_AND_RUN_TIME_FIXED/RUN_TIME config root here and my guess would be that an injected config root with a config mapping would result in an ambiguous dependency error...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI it's not a problem because a config root implemented as a config mapping is not considered a RootDefinition...

import java.lang.annotation.Target;

/**
* A config property injected during the static initialization phase of a native image build may result in unexpected errors
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* A config property injected during the static initialization phase of a native image build may result in unexpected errors
* Allows a configuration property injection point to retrieve values during static initialization.
* <p>
* A config property injected during the static initialization phase of a native image build may result in unexpected errors

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should improve the wording here, as it's not only about injecting properties.

declaringClassCandidate +
" may lead to unexpected results. To ensure proper results, please " +
"change the type of the field to " +
ParameterizedType.create(DotNames.INSTANCE, new Type[] { type }, null) +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this earlier version of the check took into account the type being injected in the error message (i.e. it would suggest Instance<MyConfig> instead of Instance<String>, IIUC).

Would it make sense to still do that with the new approach, or is it too complex to be worth it?

@radcortez
Copy link
Member

If we remove the JVM mode check, you may still get unexpected results. The issue is that the available Config differs between STATIC_INIT and RUNTIME, even in JVM mode. For instance, sources like Vault, or Database, are not available during STATIC_INIT, meaning that they can override a configuration for components initialized statically.

@yrodiere
Copy link
Member

If we remove the JVM mode check, you may still get unexpected results. The issue is that the available Config differs between STATIC_INIT and RUNTIME, even in JVM mode. For instance, sources like Vault, or Database, are not available during STATIC_INIT, meaning that they can override a configuration for components initialized statically.

So you think the check should happen in JVM mode too, and the annotation should be renamed to something like @StaticInitialization, and the error message rephrased accordingly?

@radcortez
Copy link
Member

So you think the check should happen in JVM mode too, and the annotation should be renamed to something like @StaticInitialization, and the error message rephrased accordingly?

Well, yes if we are trying to prevent this. The issue is that it may or may not be acceptable to the user to get such configuration. We have no way of knowing.

For instance REST Providers are initialized during static init (like Validators), so the available config is the one at STATIC_INIT. This is more evident when building a native image, because phases are separated by building the image and running the image. In JVM these concepts are blurred because they happen in the same runtime context, but the order is still the same.

@yrodiere
Copy link
Member

yrodiere commented Sep 28, 2023

In JVM these concepts are blurred because they happen in the same runtime context, but the order is still the same.

But in JVM mode, static init config is apparently influenced by what developers consider "runtime" config (e.g. env variables or system properties I suppose, or perhaps config files on the filesystem). At least that seems to be what triggered all the fuss in #36125 . If that's not the case, then I don't understand #36125 anymore, because the problem experienced in native mode should have happened in JVM mode as well.

In that light, the fact that Vault or Database config sources don't affect static init config in JVM mode is... confusing? I suppose there are technical reasons (e.g. the DB connection pool is not available during static init), but it's definitely not just "the same".

Now, the fact that env variables or system properties influence static init in JVM mode is good, and we probably don't want to change that.

I'd suggest:

  • Enabling the warning logs in JVM mode too (like warnStaticInitInjectionPoints used to, I think?)
  • Having the check trigger a failure (exception) only if there are problematic config sources.
    1. In native mode, this means all the time. Because there's always the possibility of system properties, for example.
    2. In JVM mode, that's more complicated. If we can identify sources that may be problematic (vault, database, ...), then that's great and we should have the check trigger a failure, but by default it probably shouldn't.

WDYT @mkouba? @radcortez?

@radcortez
Copy link
Member

But in JVM mode, static init config is apparently influenced by what developers consider "runtime" config (e.g. env variables or system properties I suppose, or perhaps config files on the filesystem). At least that seems to be what triggered all the fuss in #36125 . If that's not the case, then I don't understand #36125 anymore, because the problem experienced in native mode should have happened in JVM mode as well.

Correct. Both JVM and Native go through a STATIC_INIT config phase, but each can have different configuration. For instance, JVM mode can add configuration found in config folder, which is not available during the native image build. This does make things super confusing.

In that light, the fact that Vault or Database config sources don't affect static init config in JVM mode is... confusing? I suppose there are technical reasons (e.g. the DB connection pool is not available during static init), but it's definitely not just "the same".

Correct, DB connection pool is not available during STATIC_INIT. I guess it would be possible to have Vault available during JVM STATIC_INIT, but not during native image build (while technically possible, I guess that you don't want to have Vault config participate in that case, because it needs an external URL that may not be available in the build machine). This would require different registration of sources depending on the phase and the target, which would make things even more confusing.

  1. In JVM mode, that's more complicated. If we can identify sources that may be problematic (vault, database, ...), then that's great and we should have the check trigger a failure, but by default it probably shouldn't.

What we can probably do, is record the injected configuration in STATIC_INIT components and match the values after the runtime configuration is available. If you find a mismatch, it means that a runtime ConfigSource made an override to the injected value.

@yrodiere
Copy link
Member

What we can probably do, is record the injected configuration in STATIC_INIT components and match the values after the runtime configuration is available. If you find a mismatch, it means that a runtime ConfigSource made an override to the injected value.

That's interesting... And I think there is such a feature somewhere already, about build time config being overridden later, right?

@mkouba WDYT?

@radcortez
Copy link
Member

That's interesting... And I think there is such a feature somewhere already, about build time config being overridden later, right?

Yes, we do it here:

public void handleConfigChange(Map<String, ConfigValue> buildTimeRuntimeValues) {
SmallRyeConfig config = ConfigProvider.getConfig().unwrap(SmallRyeConfig.class);
// Disable the BuildTime RunTime Fixed (has the highest ordinal), because a lookup will get the expected value,
// and we have no idea if the user tried to override it in another source.
Optional<ConfigSource> builtTimeRunTimeFixedConfigSource = config.getConfigSource("BuildTime RunTime Fixed");
if (builtTimeRunTimeFixedConfigSource.isPresent()) {
ConfigSource configSource = builtTimeRunTimeFixedConfigSource.get();
if (configSource instanceof DisableableConfigSource) {
((DisableableConfigSource) configSource).disable();
}
}
List<String> mismatches = new ArrayList<>();
for (Map.Entry<String, ConfigValue> entry : buildTimeRuntimeValues.entrySet()) {
ConfigValue currentValue = config.getConfigValue(entry.getKey());
// Check for changes. Also, we only have a change if the source ordinal is higher
if (currentValue.getValue() != null && !entry.getValue().getValue().equals(currentValue.getValue())
&& entry.getValue().getSourceOrdinal() < currentValue.getSourceOrdinal()) {
mismatches.add(
" - " + entry.getKey() + " is set to '" + currentValue.getValue()
+ "' but it is build time fixed to '"
+ entry.getValue().getValue() + "'. Did you change the property " + entry.getKey()
+ " after building the application?");
}
}
// Enable the BuildTime RunTime Fixed. It should be fine doing these operations, because this is on startup
if (builtTimeRunTimeFixedConfigSource.isPresent()) {
ConfigSource configSource = builtTimeRunTimeFixedConfigSource.get();
if (configSource instanceof DisableableConfigSource) {
((DisableableConfigSource) configSource).enable();
}
}
if (!mismatches.isEmpty()) {
final String msg = "Build time property cannot be changed at runtime:\n" + String.join("\n", mismatches);
switch (configurationConfig.buildTimeMismatchAtRuntime) {
case fail:
throw new IllegalStateException(msg);
case warn:
log.warn(msg);
break;
default:
throw new IllegalStateException("Unexpected " + BuildTimeMismatchAtRuntime.class.getName() + ": "
+ configurationConfig.buildTimeMismatchAtRuntime);
}
}
}
, but I believe that for this discussed case it would be easier to implement.

For build time, we generate a MAX ordinal source to prevent users to override the values, but we have to discard it to search for values that may have been overridden in other sources by matching the recorded build time time properties against the runtime properties.

mkouba added 2 commits October 2, 2023 11:52
- verify the current ImageMode when a config object is injected
- fail if injected  during the static initialization phase of a native image build
- deprecate ConfigInjectionStaticInitBuildItem
@mkouba
Copy link
Contributor Author

mkouba commented Oct 2, 2023

What we can probably do, is record the injected configuration in STATIC_INIT components and match the values after the runtime configuration is available. If you find a mismatch, it means that a runtime ConfigSource made an override to the injected value.

That's interesting... And I think there is such a feature somewhere already, about build time config being overridden later, right?

@mkouba WDYT?

Hm, but this would mean that we would eventually fail at RUNTIME_INIT when the app starts... which might be a bit too late?

Also it's probably easy to record injected properties, but I'm not sure about config mappings...

@mkouba mkouba force-pushed the native-config-inject-validation branch from 02b4703 to db78fb2 Compare October 2, 2023 10:21
@quarkus-bot
Copy link

quarkus-bot bot commented Oct 2, 2023

Failing Jobs - Building db78fb2

Status Name Step Failures Logs Raw logs Build scan
JVM Tests - JDK 11 Build Failures Logs Raw logs
JVM Tests - JDK 17 Build Failures Logs Raw logs
JVM Tests - JDK 17 Windows Build Failures Logs Raw logs
JVM Tests - JDK 20 Build Failures Logs Raw logs
Native Tests - HTTP Build Failures Logs Raw logs
Native Tests - Messaging1 Build Failures Logs Raw logs
Native Tests - Messaging2 Build Failures Logs Raw logs
Native Tests - Misc3 Build Failures Logs Raw logs
Native Tests - Virtual Thread - Messaging Build ⚠️ Check → Logs Raw logs

Full information is available in the Build summary check run.

Failures

⚙️ JVM Tests - JDK 11 #

- Failing: extensions/undertow/deployment 
! Skipped: devtools/cli extensions/agroal/deployment extensions/avro/deployment and 346 more

📦 extensions/undertow/deployment

io.quarkus.undertow.test.config.WebFilterConfigInjectionWarningsTest. line 42 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <1> but was: <0>
	at io.quarkus.undertow.test.config.WebFilterConfigInjectionWarningsTest.lambda$static$2(WebFilterConfigInjectionWarningsTest.java:42)
	at io.quarkus.test.QuarkusUnitTest.afterAll(QuarkusUnitTest.java:769)

⚙️ JVM Tests - JDK 17 #

- Failing: extensions/undertow/deployment 
! Skipped: devtools/cli extensions/agroal/deployment extensions/avro/deployment and 347 more

📦 extensions/undertow/deployment

io.quarkus.undertow.test.config.WebFilterConfigInjectionWarningsTest. line 42 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <1> but was: <0>
	at io.quarkus.undertow.test.config.WebFilterConfigInjectionWarningsTest.lambda$static$2(WebFilterConfigInjectionWarningsTest.java:42)
	at io.quarkus.test.QuarkusUnitTest.afterAll(QuarkusUnitTest.java:769)

⚙️ JVM Tests - JDK 17 Windows #

- Failing: extensions/arc/deployment 
! Skipped: devtools/cli extensions/agroal/deployment extensions/amazon-lambda-http/deployment and 469 more

📦 extensions/arc/deployment

io.quarkus.arc.test.config.nativebuild.NativeBuildConfigInjectionOkTest. - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: java.lang.RuntimeException: Failed to get GraalVM version

⚙️ JVM Tests - JDK 20 #

- Failing: extensions/undertow/deployment 
! Skipped: devtools/cli extensions/agroal/deployment extensions/avro/deployment and 347 more

📦 extensions/undertow/deployment

io.quarkus.undertow.test.config.WebFilterConfigInjectionWarningsTest. line 42 - More details - Source on GitHub

org.opentest4j.AssertionFailedError: expected: <1> but was: <0>
	at io.quarkus.undertow.test.config.WebFilterConfigInjectionWarningsTest.lambda$static$2(WebFilterConfigInjectionWarningsTest.java:42)
	at io.quarkus.test.QuarkusUnitTest.afterAll(QuarkusUnitTest.java:769)

⚙️ Native Tests - HTTP #

- Failing: integration-tests/resteasy-reactive-kotlin/standard 

📦 integration-tests/resteasy-reactive-kotlin/standard

Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-resteasy-reactive-kotlin-standard: Failed to build quarkus application


⚙️ Native Tests - Messaging1 #

- Failing: integration-tests/kafka-oauth-keycloak integration-tests/reactive-messaging-kafka 

📦 integration-tests/kafka-oauth-keycloak

Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-kafka-oauth-keycloak: Failed to build quarkus application

📦 integration-tests/reactive-messaging-kafka

Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-reactive-messaging-kafka: Failed to build quarkus application


⚙️ Native Tests - Messaging2 #

- Failing: integration-tests/reactive-messaging-amqp integration-tests/reactive-messaging-pulsar integration-tests/reactive-messaging-rabbitmq and 1 more

📦 integration-tests/reactive-messaging-amqp

Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-amqp: Failed to build quarkus application

📦 integration-tests/reactive-messaging-pulsar

Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-reactive-messaging-pulsar: Failed to build quarkus application

📦 integration-tests/reactive-messaging-rabbitmq

Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-reactive-messaging-rabbitmq: Failed to build quarkus application

📦 integration-tests/reactive-messaging-rabbitmq-dyn

Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-reactive-messaging-rabbitmq-dyn: Failed to build quarkus application


⚙️ Native Tests - Misc3 #

- Failing: integration-tests/smallrye-config 

📦 integration-tests/smallrye-config

Failed to execute goal io.quarkus:quarkus-maven-plugin:999-SNAPSHOT:build (default) on project quarkus-integration-test-smallrye-config: Failed to build quarkus application

@mkouba
Copy link
Contributor Author

mkouba commented Oct 2, 2023

That's interesting... And I think there is such a feature somewhere already, about build time config being overridden later, right?

Yes, we do it here: ...

By the way, this results in a relatively large generated StartupTask class as it contains all build-time and build-runtime-fixed values, ~ 25KB for the config quickstart.

In theory, we could record the values for all @ConfigProperty injection points and check those values during RUNTIME_INIT (and fail if a mismatch occurs), and handle the config mapping injection points differently... not sure how though :-).

@radcortez
Copy link
Member

By the way, this results in a relatively large generated StartupTask class as it contains all build-time and build-runtime-fixed values, ~ 25KB for the config quickstart.

We shouldn't need this. Only the properties that were injected in the static init classes. If we do need them, they are already recorded in a ConfigSource set with Integer.MAX, so querying the Config system will always return the recorded expected value

In theory, we could record the values for all @ConfigProperty injection points and check those values during RUNTIME_INIT (and fail if a mismatch occurs), and handle the config mapping injection points differently... not sure how though :-).

Config mappings are not registered by default for STATIC_INIT, you need to explicitly annotate the mapping with @StaticInitSafe. I think we can consider that if you add that annotation you are on your own :)

@mkouba
Copy link
Contributor Author

mkouba commented Oct 3, 2023

In theory, we could record the values for all @ConfigProperty injection points and check those values during RUNTIME_INIT (and fail if a mismatch occurs), and handle the config mapping injection points differently... not sure how though :-).

Config mappings are not registered by default for STATIC_INIT, you need to explicitly annotate the mapping with @StaticInitSafe. I think we can consider that if you add that annotation you are on your own :)

Yes, that's actually good news. So injecting a config mapping that is not annotated with @StaticInitSafe results in a failure during STATIC_INIT. The same applies to RUN_TIME config roots. And BUILD_AND_RUN_TIME_FIXED config roots shouldn't change. So if we ignore the programmatic API for now (which is IMO ok) we could just record the properties injected during STATIC_INIT, compare the runtime values and fail the RUNTIME_INIT if a mismatch is found.

In other words, if a CDI bean that is created during STATIC_INIT injects a config property we will record the injected value and then during RUNTIME_INIT we'll compare the injected values with the "final" values. If a mismatch is found then we will fail with an actionable error.

Does it make sense? @radcortez @yrodiere

@yrodiere
Copy link
Member

yrodiere commented Oct 3, 2023

Does it make sense? @radcortez @yrodiere

It does to me.

The one downside compared to your current solution is that application developers might end up in a situation where their config gets injected without error, and they end up using it during static init, triggering other, less explicit errors down the line because the property has no value during static init.

But I suppose it has more to do with other libraries (e.g. Hibernate Validator) at this point, and at least in such a scenario the developer will get a stacktrace showing them where the problem happens,

So if we ignore the programmatic API for now (which is IMO ok)

By "programmatic API", do you mean the use of Instance (e.g. @ConfigProperty(...) Instance<String>), or retrieval of config properties via Arc.container().instance(...) (is that even possible?), or retrieval of config properties through Smallrye's ConfigProvider.getConfig()? I'm not sure I'd consider any of those negligible...

@mkouba
Copy link
Contributor Author

mkouba commented Oct 3, 2023

The one downside compared to your current solution is that application developers might end up in a situation where their config gets injected without error, and they end up using it during static init, triggering other, less explicit errors down the line because the property has no value during static init.

Yes, on the other hand we would not fail if the injected value did not change which is good ;-)

So if we ignore the programmatic API for now (which is IMO ok)

By "programmatic API", do you mean the use of Instance (e.g. @ConfigProperty(...) Instance<String>), or retrieval of config properties via Arc.container().instance(...) (is that even possible?), or retrieval of config properties through Smallrye's ConfigProvider.getConfig()? I'm not sure I'd consider any of those negligible...

I mean the "retrieval of config properties through Smallrye's ConfigProvider.getConfig()". @ConfigProperty(...) Instance<String> and Arc.container().instance(...) would be handled (since the CDI beans are obtained under the hood).

@yrodiere
Copy link
Member

yrodiere commented Oct 3, 2023

I mean the "retrieval of config properties through Smallrye's ConfigProvider.getConfig()". @ConfigProperty(...) Instance<String> and Arc.container().instance(...) would be handled (since the CDI beans are obtained under the hood).

Nice! Well I don't even know if Smallrye's ConfigProvider.getConfig() can be handled, so 🤷 @radcortez will know.

@radcortez
Copy link
Member

I mean the "retrieval of config properties through Smallrye's ConfigProvider.getConfig()". @ConfigProperty(...) Instance<String> and Arc.container().instance(...) would be handled (since the CDI beans are obtained under the hood).

Correct.

I mean the "retrieval of config properties through Smallrye's ConfigProvider.getConfig()". @ConfigProperty(...) Instance<String> and Arc.container().instance(...) would be handled (since the CDI beans are obtained under the hood).

ConfigProvider.getConfig will just give you the Config instance that is available at each phase. We register a Config instance for STATIC_INIT and that is the one you get during that phase. When we move to runtime, we release the old instance and register a new instance with the components now available for runtime too.

@mkouba
Copy link
Contributor Author

mkouba commented Oct 5, 2023

This PR was superseded by #36281.

@mkouba mkouba closed this Oct 5, 2023
@quarkus-bot quarkus-bot bot added the triage/invalid This doesn't seem right label Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants