-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Forms: Duplicate form key stored in database #402
Comments
To fix the exception, I've manually copied the data from the |
After encountering the same issue, I've compared the data in the The first row (with ID 6) contained: "Id":0,
"Key":"7e1365db-406c-44d6-95ca-4b966167879b",
"CreateDate":"2020-07-23T14:33:08.29Z",
"UpdateDate":"2020-09-18T14:31:43.2475711+02:00" And the dulicate row: "Id":6,
"Key":"7e1365db-406c-44d6-95ca-4b966167879b",
"CreateDate":"2020-07-23T14:33:08.29Z",
"UpdateDate":"2020-09-18T14:31:43.2085948+02:00" So it looks like both rows are updated, but the duplicate is saved first (looking at the |
@ronaldbarendse I experienced this exact same issue in Umbraco Cloud. I also couldn't delete the row from the SQL database as I didn't have significant permissions (or it was kicking off about something). The kind fellas at Umbraco Cloud Support removed the entry for me but my experience was EXACTLY the same. I was editing a workflow to change an email value, I hit save, cta changed to a cross (Sidenote, I feel this UX needs to be clearer as it's not that obvious unless you're paying attention that something hasn't worked) - then reloading the forms page just broke entirely as there were duplicate entries. Might not be useful, but this was the error in the repeater when a duplicate entry was found in the DB |
I have also faced this issue on Umbraco Cloud. Support was able to delete the duplicate for me but it's only a temporary fix. Whenever I update a workflow, the entire form is duplicated and thus becomes unusable. I wonder if checking for a duplicate ID when the form is saved or updated in the database would help address this..? |
@ronaldbarendse I only just now saw that the problem Definition here is showing a very slight but interesting difference: #402 (comment) So the update time is merely 4 milliseconds different. Could it be that there was an app pool recycle when saving the Form? I'm thinking it might be related to the fixes with did with unsubscribing from event handlers: umbraco/Umbraco-CMS#8815 I notice you're on 8.6.4, can @seekingcode18 and @lukehook confirm they were also on 8.6.4 when this happened? I'm thinking that an overlapping application pool might be the cause here, so both app pools still processed the Save action which is what caused a duplicate to be added? Although.. hmm, well then the int identifiers wouldn't be so far apart I guess. |
@nul800sebastiaan I'm on Umbraco 8.8.0 and Forms 8.5.3 |
@nul800sebastiaan The chances of an application recycle and saving the form at the same time are astronomically small. Because I've already encountered this multiple times myself and others have reported it too, this would also make that very unlikely. The main issue is that it's currently technically possible to store the same form in different rows, so either the |
It's really not that small since we apparently have for a long time not been shutting down the old app pool in a timely manner, the overlap between app pools might actually be quite long. Yes, I know what the technical problem is, but that doesn't give me a cause of the issue, of course we need to prevent duplicate keys from being able to exist. I would really like to repro this issue while we work on a fix and so far it's not been possible to repro. @seekingcode18 could you show what your database looks like in the |
@nul800sebastiaan unfortunately I don't have the site or db on my machine as I've been working with it in Umbraco Cloud and Kudu. Would it be helpful if I add you to the project instead? I can try and get the db working locally but it may take a couple of days as I haven't done that with Umbraco before. Steps to reproduce for me are: open a form in the backend, edit a workflow, and save a workflow. When I try to access any forms again, it throws that error. |
@seekingcode18 Ah, that would work for me too, can you invite me on that email address then I can have a look to see what is stored at least. Thanks! |
@nul800sebastiaan will do, thank you for looking into it! |
@nul800sebastiaan apologies I can't quite remember the version it was when I received the error. Forms 8.6.3 maybe?! It was an Umbraco Cloud project so has autograded since. If it makes a difference, the errors occurred immediately after activating Forms in the Database. It was the first time I had edited a workflow (the razor email one if it helps to be specific) since 'migrating' or activating it. I no longer work on the solution but I think it's been fine since. I don't know if this correlates to anyone elses experience but thought it worth mentioning in case it's something that occurs as a consequence of them being added to the database (whatever is involved in that) |
@seekingcode18 Good news/bad news! I can now reproduce the problem 100% on your database. That's also the bad news. 🙈 But this helps a lot, now we can look into what the cause is, thanks so far! Can you please delete me from your project again, I have what I need now! 👍 |
@ronaldbarendse Did you also happen to have a duplicate workflow in your DB when you encountered this, I can repro by having a workflow that has the same key as another workflow. |
@nul800sebastiaan I haven't encountered the same issue with workflows, but the issue is probably similar, as it also uses an auto-incremented integer primary key. |
I would argue the different IDs are more interesting, as I already noted in that same comment:
In any case, it's strange to have both ID and Key as columns and also be part of the JSON stored in the definition. |
Because having a duplicate form key completely breaks both the back-office and front-end and this can be caused by an editor simply saving a form (e.g. after adding a workflow), I've manually added the following unique indexes to prevent this from happening: CREATE UNIQUE INDEX IX_UFDataSource_Key ON UFDataSource([Key]);
CREATE UNIQUE INDEX IX_UFForms_Key ON UFForms([Key]);
CREATE UNIQUE INDEX IX_UFPrevalueSource_Key ON UFPrevalueSource([Key]);
CREATE UNIQUE INDEX IX_UFWorkflows_Key ON UFWorkflows([Key]); When Forms tries to save the duplicate key, the database will now throw an exception. The changes are already saved to the original row (in the case I tested), so this can be used as a workaround 🎉 No need to wait for the release that fixes this and no more manually removing the duplicate rows! Looking at the stack-trace of this exception, you can see the
|
I'm experiencing the same issue with latest Umbraco Forms and UmbracoCMS (8.9.0 and 8.5.4) on a new Cloud project using database storage for the forms. EDIT: Forgot to mention - this is in my local dev environment for the cloud project, the code hasn't been pushed to Cloud yet. |
Hello, I was wondering if there are any updates about when this might be close to being fixed? |
I've just had a project on Umbraco Cloud (Umbraco 8.9.1, Forms 8.5.5, Deploy 3.5.1) that had more than 15.000 rows in the I've removed all the duplicate rows using the following SQL query (only keeping the first/lowest integer ID): SELECT 0 -- Set rowcount to 1
WHILE (@@ROWCOUNT > 0)
BEGIN
DELETE FROM UFForms WHERE Id IN (SELECT MAX(Id) FROM UFForms GROUP BY [Key] HAVING COUNT([Key]) > 1);
END |
And to check whether you have any duplicate rows, you can run the following SQL query: SELECT [Key], COUNT([Key]) FROM UFDataSource GROUP BY [Key];
SELECT [Key], COUNT([Key]) FROM UFForms GROUP BY [Key];
SELECT [Key], COUNT([Key]) FROM UFPrevalueSource GROUP BY [Key];
SELECT [Key], COUNT([Key]) FROM UFWorkflows GROUP BY [Key]; |
The large amount of duplicate
I've removed all rows in the |
I'm experiencing a similar issue. When I make any change to a form, it saves a second duplicated record and I'm unable to open any forms or entries in the back-office until I manually delete one of the duplicates in the database. Umbraco Cloud |
Similar issue here Umbraco 8.9.1 |
This has been fixed by this PR - https://github.com/umbraco/Umbraco-Deploy/pull/460 and was due to a combination of Forms and Deploy together. This fix is done in the Deploy codebase and is fixed in the DLL |
@warrenbuckley Thanks!!!! Any ETA on that Deploy release? |
I can't tell you when the next Deploy release is due @hfloyd but perhaps my lovely colleague @filipbech may be able to give you some insight. However as an alternative here is a CI build of this PR, the fix is contained in |
@warrenbuckley / @filipbech would it be safe for me to use this file and push it to Cloud? Or will that screw up the automated Update process once the official release is ready? |
We don't have a date set yet - best guess would be next week! |
Thanks, @filipbech . Would it be safe for me to use this file in the meantime? |
I don't know... sorry |
@filipbech & @nul800sebastiaan I am still seeing this issue on a live Cloud site: Saving the Form via the back-office UI is adding duplicated records to the table, and duplicating the workflows on the form. |
@umbrabot reopen |
@nul800sebastiaan @filipbech @warrenbuckley Guys, This is KILLING me! Still a problem on a live site. I am caught in an evil UI loop - I have duplicated workflows sending dupe emails (Client not loving this...), when I attempt to delete the extra workflows, then I have to SAVE the form - which then adds the extra form record to the damn db, which then kills the UI, and if I run the manual SQL to kill the extra form record, the extra workflows are present again. I was finally able to delete the extra workflows via SQL Server, but I am concerned that re-saving a form will bring them back again. REOPEN THIS ISSUE, PLEASE. IT IS NOT FIXED Umbraco 8.11.1 |
That sounds awful, sorry to hear @hfloyd! Could you please contact Cloud support with the project alias and give them access to it so we can have a look. I suspect the underlying issue might be fixed, but we've caused the data to be incorrect so that might need fixing and then it should be fine for you, not sure. Make sure to refer support to this issue and ask them to escalate it to second level support, then we can take it from there! |
Thanks, @nul800sebastiaan So, for anyone coming across this problem - If you had a previous version of forms installed, and the update didn't fix the issue, check the [UFWorkflows] tables for existing duplicates and delete them from that table directly. |
I get it, super frustrating! 😬 Let's leave this open for a bit longer to make sure the issues have actually been fixed and don't re-occur. The offer is still open to reach out and let support access your site/db for an extra pair of eyes if you run into more problems! |
Thanks, @nul800sebastiaan . Will do :-) |
Will close this now as having implemented the data integrity keys and constraints in 8.7 we should have resolved the root of the issue - that the database allowed duplicates. It now doesn't and will avoid an installation getting into the state that causes the problems described in this issue. |
Apparently it's possible to store form data multiple times using the same key in the
UFForms
table:Reproduction
Bug summary
After adding a new workflow to a form and trying to save it, I got a validator error (forgot to enter an email subject), I fixed this and saved the form again. The save button changed from the progress/loading animation to a cross and none of the forms loaded anymore, because an exception.
Specifics
Umbraco Forms 8.5.3
Umbraco 8.6.4
Expected result
I would expect the key column to be unique, so this can't happen in the first place. As an additional bonus, making it an unique index will speed up the lookup (especially if you have a lot of forms). Besides that, it should always check whether the form already exists in the database!
This item has been added to our backlog AB#8286
The text was updated successfully, but these errors were encountered: