2.13.0.Final: Flyway Migration checksum mismatch #28301

andreas-eberle · 2022-09-29T14:48:48Z

Describe the bug

I just updated my service from Quarkus 1.12.3.Final to 1.13.0.Final and now it crashes (no other changes) during startup due to the following Flyway error: Validate failed: Migrations have failed validation.

2022-09-29 16:42:35,209 ERROR [io.qua.run.Application] (Quarkus Main Thread) Failed to start application (with profile dev): org.flywaydb.core.api.exception.FlywayValidateException: Validate failed: Migrations have failed validation
Migration checksum mismatch for migration version 2.2.0
-> Applied to database : 489677976
-> Resolved locally    : -1831618313
Either revert the changes to the migration, or run repair to update the schema history.
Need more flexibility with validation rules? Learn more: https://rd.gt/3AbJUZE
	at org.flywaydb.core.Flyway.lambda$migrate$0(Flyway.java:134)
	at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:204)
	at org.flywaydb.core.Flyway.migrate(Flyway.java:128)
	at io.quarkus.flyway.runtime.FlywayRecorder.doStartActions(FlywayRecorder.java:79)
	at io.quarkus.deployment.steps.FlywayProcessor$startActions1770701860.deploy_0(Unknown Source)
	at io.quarkus.deployment.steps.FlywayProcessor$startActions1770701860.deploy(Unknown Source)
	at io.quarkus.runner.ApplicationImpl.doStart(Unknown Source)
	at io.quarkus.runtime.Application.start(Application.java:101)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:110)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:69)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:122)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:103)
	at java.base/java.lang.Thread.run(Thread.java:833)

2022-09-29 16:42:35,213 INFO  [io.qua.dep.dev.IsolatedDevModeMain] (main) Attempting to start live reload endpoint to recover from previous Quarkus startup failure
2022-09-29 16:42:35,215 ERROR [io.qua.run.boo.StartupActionImpl] (Quarkus Main Thread) Error running Quarkus: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:103)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NoClassDefFoundError: org/apache/maven/shared/utils/cli/CommandLineException
	at io.quarkus.deployment.dev.IsolatedDevModeMain$1.accept(IsolatedDevModeMain.java:117)
	at io.quarkus.deployment.dev.IsolatedDevModeMain$1.accept(IsolatedDevModeMain.java:93)
	at io.quarkus.runtime.ApplicationLifecycleManager.lambda$setDefaultExitCodeHandler$0(ApplicationLifecycleManager.java:364)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:208)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:69)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:122)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	... 6 more
Caused by: java.lang.ClassNotFoundException: org.apache.maven.shared.utils.cli.CommandLineException
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:490)
	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:440)
	... 14 more

2022-09-29 16:42:35,215 ERROR [io.qua.run.boo.StartupActionImpl] (Quarkus Main Thread) Error running Quarkus: java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at io.quarkus.runner.bootstrap.StartupActionImpl$1.run(StartupActionImpl.java:103)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NoClassDefFoundError: org/apache/maven/shared/utils/cli/CommandLineException
	at io.quarkus.deployment.dev.IsolatedDevModeMain$1.accept(IsolatedDevModeMain.java:117)
	at io.quarkus.deployment.dev.IsolatedDevModeMain$1.accept(IsolatedDevModeMain.java:93)
	at io.quarkus.runtime.ApplicationLifecycleManager.lambda$setDefaultExitCodeHandler$0(ApplicationLifecycleManager.java:364)
	at io.quarkus.runtime.ApplicationLifecycleManager.run(ApplicationLifecycleManager.java:208)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:69)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:42)
	at io.quarkus.runtime.Quarkus.run(Quarkus.java:122)
	at io.quarkus.runner.GeneratedMain.main(Unknown Source)
	... 6 more
Caused by: java.lang.ClassNotFoundException: org.apache.maven.shared.utils.cli.CommandLineException
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:490)
	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:440)
	... 14 more

To be clear, no migration should be triggered. The database is already at the newest migration and the exact same service works with 1.12.3.Final.

Furthermore, I wonder if these ClassNotFoundExceptions are "normal" or "expected". They certainly look wrong.

I'll try to create a reproducer later.

Expected behavior

Flyway should still work with the newer Quarkus version.

Actual behavior

No response

How to Reproduce?

No response

Output of `uname -a` or `ver`

No response

Output of `java -version`

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

No response

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

No response

Additional information

All my dependencies in the project:


dependencies {
    implementation enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
    implementation 'io.quarkus:quarkus-arc'

    implementation 'io.quarkus:quarkus-resteasy-reactive'
    implementation 'io.quarkus:quarkus-resteasy-reactive-kotlin'
    implementation 'io.quarkus:quarkus-resteasy-reactive-jackson'

    implementation 'io.quarkus:quarkus-kotlin'
    implementation 'org.jetbrains.kotlin:kotlin-stdlib-jdk8'
    implementation 'com.fasterxml.jackson.module:jackson-module-kotlin'

    implementation 'io.quarkus:quarkus-hibernate-orm'
    implementation 'io.quarkus:quarkus-jdbc-postgresql'
    implementation 'io.quarkus:quarkus-flyway'
}

The text was updated successfully, but these errors were encountered:

quarkus-bot · 2022-09-29T14:48:51Z

/cc @cristhiank, @evanchooly, @gastaldi, @geoand, @gsmet

andreas-eberle · 2022-09-29T15:22:51Z

When I run migrations from the start again, I actually get a flyway schema history that is valid and works with 1.13.0.Final and thus I'm not able to create my broken database again (=> cannot provide a reproducer :( ). However, it looks like the schema of the "broken" and the "correct" database are the same.

Anyways, I think Flyway is actually ok. However, I wonder why it didn't notice the problem before? And the ClassNotFoundException are maybe also something to look into?

gsmet · 2022-09-29T15:42:41Z

@aloubyansky I wonder if the issue could be related to us removing the bootstrap transitive dependencies? Could it be that we haven't totally restored the existing behavior in Gradle?

gsmet · 2022-09-29T15:43:48Z

Ah wait. We have a Maven exception around here which doesn't look like something we want (or we would have to also add the Maven model to Gradle).

aloubyansky · 2022-09-29T15:51:07Z

@andreas-eberle did you clean the project before launching quarkusDev with the new Quarkus version?

gsmet · 2022-09-29T16:17:05Z

@aloubyansky I think it is a valid issue because of this:

https://github.com/quarkusio/quarkus/blob/main/core/deployment/src/main/java/io/quarkus/deployment/dev/IsolatedDevModeMain.java#L25-L26

We actually have Maven classes around in there. I think we should get rid of them and have a copy of this specific CommandLineUtils.translateCommandline() method in Quarkus. WDYT?

rmanibus · 2022-09-29T19:14:31Z

I noticed the same thing on my project, flyway started to validate checksums in quarkus 2.13.
enabling quarkus.flyway.repair-at-start align all the checksums in the history table and allow the application to start again.
I think this only happen if a migration has been modified after it has been applied.

gsmet · 2022-09-30T09:00:01Z

@rmanibus any chance you could write something that I could include in the migration guide?

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301

gsmet · 2022-09-30T09:34:02Z

#28318 should fix the ClassNotFoundException. Thanks for the report!

rmanibus · 2022-09-30T16:30:11Z

@gsmet of course, my analysis is that after this commit: a936ad0 quarkus.flyway.validate-on-migrate is now activated by default.

So I would write the following in the migration guide:

quarkus.flyway.validate-on-migrate is now enabled by default, which will cause the deployment to fail if you have modified migrations after they have been applied.
You can either disable it, or enable quarkus.flyway.repair-at-start to align checksums in the history table with their current values.

rmanibus · 2022-09-30T16:38:40Z

@gsmet also, I have a regression on 2.13.0 on my application, it is something related to request context.
I have a request scoped bean that apparently is reset during a request scoped session (at least I don't get the same instance in two different application scoped beans). It is a request context that I create myself with @ActivateRequestContext.

do you know if something changed in this release ?
It is working fine with 2.12.3.

I want to create an issue for it but I'm not sure what the problem is. I did not manage to reproduce this in a blank project.

gsmet · 2022-09-30T16:44:19Z

@rmanibus yes, there was a lot of work done on the contexts. I think we will really need a reproducer to understand what's going on. But first, maybe create a specific issue and explain what's going wrong.

gsmet · 2022-09-30T16:45:39Z

I added the entry in the migration guide, thanks @rmanibus .

I also fixed the CNFE for 2.13.1.Final so all should be good. Closing.

rmanibus · 2022-09-30T16:45:57Z

@gsmet ok good to know, I will create the issue thanks !

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301 (cherry picked from commit 49b2352)

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301

andreas-eberle added the kind/bug Something isn't working label Sep 29, 2022

quarkus-bot bot added area/container-image area/flyway area/kotlin labels Sep 29, 2022

gsmet changed the title ~~1.13.0.Final: Flyway Migration checksum mismatch~~ 2.13.0.Final: Flyway Migration checksum mismatch Sep 29, 2022

gsmet added a commit to gsmet/quarkus that referenced this issue Sep 30, 2022

Avoid Maven imports in core/deployment

49b2352

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301

gsmet mentioned this issue Sep 30, 2022

Avoid Maven imports in core/deployment #28318

Merged

gsmet closed this as completed Sep 30, 2022

igorregis pushed a commit to igorregis/quarkus that referenced this issue Oct 16, 2022

Avoid Maven imports in core/deployment

096906b

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301

igorregis pushed a commit to igorregis/quarkus that referenced this issue Oct 16, 2022

Avoid Maven imports in core/deployment

c9ac5c8

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301

igorregis pushed a commit to igorregis/quarkus that referenced this issue Oct 16, 2022

Avoid Maven imports in core/deployment

7d91bb8

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301

igorregis pushed a commit to igorregis/quarkus that referenced this issue Oct 17, 2022

Avoid Maven imports in core/deployment

de7602b

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301

tmihalac pushed a commit to tmihalac/quarkus that referenced this issue Oct 27, 2022

Avoid Maven imports in core/deployment

c7200d5

The Maven classes won't be available in a Gradle environment. Also install forbiddenapis to make sure they don't come back. Relates to quarkusio#28301

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2.13.0.Final: Flyway Migration checksum mismatch #28301

2.13.0.Final: Flyway Migration checksum mismatch #28301

andreas-eberle commented Sep 29, 2022 •

edited

Loading

quarkus-bot bot commented Sep 29, 2022

andreas-eberle commented Sep 29, 2022 •

edited

Loading

gsmet commented Sep 29, 2022

gsmet commented Sep 29, 2022

aloubyansky commented Sep 29, 2022

gsmet commented Sep 29, 2022

rmanibus commented Sep 29, 2022 •

edited

Loading

gsmet commented Sep 30, 2022

gsmet commented Sep 30, 2022

rmanibus commented Sep 30, 2022

rmanibus commented Sep 30, 2022

gsmet commented Sep 30, 2022

gsmet commented Sep 30, 2022

rmanibus commented Sep 30, 2022

2.13.0.Final: Flyway Migration checksum mismatch #28301

2.13.0.Final: Flyway Migration checksum mismatch #28301

Comments

andreas-eberle commented Sep 29, 2022 • edited Loading

Describe the bug

Expected behavior

Actual behavior

How to Reproduce?

Output of uname -a or ver

Output of java -version

GraalVM version (if different from Java)

Quarkus version or git rev

Build tool (ie. output of mvnw --version or gradlew --version)

Additional information

quarkus-bot bot commented Sep 29, 2022

andreas-eberle commented Sep 29, 2022 • edited Loading

gsmet commented Sep 29, 2022

gsmet commented Sep 29, 2022

aloubyansky commented Sep 29, 2022

gsmet commented Sep 29, 2022

rmanibus commented Sep 29, 2022 • edited Loading

gsmet commented Sep 30, 2022

gsmet commented Sep 30, 2022

rmanibus commented Sep 30, 2022

rmanibus commented Sep 30, 2022

gsmet commented Sep 30, 2022

gsmet commented Sep 30, 2022

rmanibus commented Sep 30, 2022

andreas-eberle commented Sep 29, 2022 •

edited

Loading

Output of `uname -a` or `ver`

Output of `java -version`

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

andreas-eberle commented Sep 29, 2022 •

edited

Loading

rmanibus commented Sep 29, 2022 •

edited

Loading