Endless crashloop after upgrading to latest

[accforgithubtest]

Bug Description

n8n is in an endless crash-loop after upgrading to latest.
This has been running fine for more than year. A standard n8n set-up with docker compose with mysql database.
Only change in a long time has been upgrading to latest version.

User settings loaded from: /home/node/.n8n/config
Last session crashed
Migrations in progress, please do NOT stop the process.
Migration "AddActivatedAtUserSetting1717498465931" failed, error: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'JSON))
			WHERE settings IS NOT NULL AND JSON_EXTRACT(settings, '$.userActiva...' at line 2
Error: There was an error running database migrations
QueryFailedError: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'JSON))
			WHERE settings IS NOT NULL AND JSON_EXTRACT(settings, '$.userActiva...' at line 2
User settings loaded from: /home/node/.n8n/config
Last session crashed

To Reproduce

A standard n8n set-up with docker compose with mysql database.
Only change in a long time has been upgrading to latest version.

Issue is with the latest version. Apologies since I cannot remember the previous version, but I had upgraded to the latest docker version in approx early June 2024.

Expected behavior

Dont be in crash loop.

Operating System

Ubuntu 22.04 LTS Server

n8n Version

latest docker tag

Node.js Version

not sure

Database

MySQL

Execution mode

main (default)

[Joffcom]

Hey @accforgithubtest

It looks like one of the database migrations has an issue.

I would recommend going back to the previous version you were on and restoring a database backup to get you up and running.

Ideally the next step would be to use the cli to export all of the workflows and credentials then importing them into a Postgres database as we are removing mysql support in the future.

I will still get a dev ticket created to fix this but maybe the above will get you back up and running.

[accforgithubtest]

thanks @Joffcom

use the cli to export all of the workflows and credentials then importing them into a Postgres database

Is there any guides / scripts for this mysql -> postgres migration please ?

[Joffcom]

Hey @accforgithubtest,

No specific guides or scripts at the moment sadly but it is a similar process to moving from sqlite to mysql if you didn't deploy that way from the start.

First step is to run the cli to export workflows and credentials

n8n export:workflow --all --output=workflows.json
n8n export:credentials --all --output=credentials.json

Then update your env options to point to the new database, start up n8n and create the owner account again (as we don't yet have a way to export users). Then import the workflows and credentials with:

n8n import:workflow --input=workflows.json
n8n import:credentials --input=credentials.json

Then delete the credentials.json and workflows.json and you should be good to go unless you are currently using enterprise features which will be a bit trickier.

The dev ticket to get this resolved is N8N-7470

[accforgithubtest]

Thanks @Joffcom .

I tried restoring the database from back up (successful as I can see the data in the db), and switched to a previous version of n8n 1.47.1 - still no luck and I am seeing the same issue.

Any other suggestions for me to try ?

[accforgithubtest]

Even going back as far as 1.46.0 doesnt help, still seeing the same issue.

So this an irrecoverable situation ? what options do I have to proceed from here ?

I cannot run cli export commands mentioned above, as the container is a crash-loop and doesnt have an opportunity to run these commands.

[accforgithubtest]

@netroy / @Joffcom - has a fix been released since this issue is closed ?

[netroy]

the issue got auto-closed by github because the linked PR was merged.
The fix will be released in a new release tomorrow, and we'll also back-port the fix into a patch release for the current latest.

[accforgithubtest]

ok thanks, looking forward to the fix since n8n has been down since the upgrade !

[janober]

Fix got released with [email protected]

[accforgithubtest]

Thank you all ! The issue is fixed now. Cheers!