Renaming folder actually deletes most of its contents

[RobinJ1995]

I was on a holiday for about a week. I had no internet connection on my laptop, but added some pictures into my Foto's folder. I just connected to the internet, and renamed the folder in question to Pictures. The data I added to the folder is still there, everything that was in there before is gone, both on the server and on the client. That's ~10GB of pictures that just went poof.

Please look into this. Much appreciated.

[guruz]

Ouch, this sounds bad.

@PVince81 @icewind1991 @DeepDiver1975 Can you help with this? From the screenshots it looks like server 8.0.2

@RobinJ1995 btw we also have https://raw.githubusercontent.com/owncloud/client/master/issue_template.md

[PVince81]

Big folder rename, likely to be: owncloud/core#13391

If it is a shared folder that you renamed as recipient, and renamed it back to the original sharer name, then it's: owncloud/core#15340

[RobinJ1995]

The folder in question was my own folder, and was not shared with anyone.

Regarding owncloud/core#13391;
I renamed the folder on my laptop using my file manager, not though the web client. I also didn't notice anything strange in the web client like a page that took a long time to load. The data was just gone.

I have access to the server in question, so if you tell me which log files (if any) you would like to see, I can provide them.

[PVince81]

This could also happen with your own folder if it contains many files and subfolders.
In the case of shared folders it would be even slower and increase the probability of the problem happening.

[RobinJ1995]

So is there any clue as to why this is happening? Is it perhaps checking stuff in the wrong order, or doing them at the same time?

[PVince81]

Yes, this happens because the index needs to be updated for a lot of entries. There is a window of time where the files on the server disk and index do not match. If during this time another client tries to access this folder, it will access outdated data and be confused. This can also happen if the time needed to update the database takes longer than a sync cycle, which means that while the PHP process is still running the next sync run will kick in.

There is a workaround here owncloud/core#13391 (comment) to make the DB update faster, but it won't eliminate the probability of concurrency.

In the future there will be high-level file locking owncloud/core#11804 to make sure that such concurrent operations cannot cause trouble.

[PVince81]

with "confused" I mean: the second process will find that what's on disk doesn't match the cache, then it will ALSO try to update the index/cache with what's on disk. Basically there will be two processes contradicting themselves about what needs to be written in the index/cache.

[RobinJ1995]

Ah. So some kind of locking mechanism should be implemented :)
Thanks for the explanation.

My desktop computer still had the files, so I unplugged the network cable before booting, copied my ownCloud folder, and reconnected to the network. Then I proceeded to upload all my files again (which took a while). So at least (almost) no data was permanently lost for me, but others may not be as lucky.
I hope this issue gets resolved quickly.

lol @ this (half-)fix, though :P
owncloud/core@6c00521