rename torture test causes internal error and potential data loss

[jnweiger]

testpilotclient 1.7.1 on openSUSE against:
a) daily.owncloud.com/stable7/owncloud
b) 8.0.0 on CentOS_7 docker image with sqlite
c) 8.0.0 on openSUSE_13.1 eval appliance with mysql

The appended rename_torture.sh shell script runs ca 2 minutes and performs tests on a folder Foo with 20 randomly created files.

This reliably triggers several internal errors in the server, and the client sees several 'no such file or directory' conditions with b) and c), not yet seen with a). After the run the number of files often differs and/or the directory hierarch differs. E.g. having subfolders Foo/Foo that were never created by the script.

#! /bin/bash
# Reproducer for renaming issues.
# Probably related to https://github.com/owncloud/core/issues/7046
#
#2015-0-26 [email protected]

rm -rf Foo Bar

if [ ! -f .owncloudsync.log ]; then
  echo Could not find .owncloudsync.log here.
  echo Please run $0 this script inside an active owncloud sync folder.
  exit 1
fi

mkdir Foo
for i in 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 ; do
  echo $RANDOM $RANDOM $RANDOM > Foo/file$RANDOM.txt
done

set -x
ls -la Foo/*
du -a Foo | wc -l
set +x

echo "Renaming Foo <-> Bar back and forth for two minutes."

for i in 1 2 3 4 5 6 7 8 9 0  1 2 3 4 5 6 7 8 9 0  1 2 3 4 5 6 7 8 9 0  1 2 3 4 5 6 7 8 9 0  1 2 3 4 5 6 7 8 9 0  1 2 3 4 5 6 7 8 9 0 ; do
  mv Foo Bar
  sleep 1
  mv Bar Foo
  sleep 1
  echo -n .
done

echo
set -x
ls -la Foo/*
du -a Foo | wc -l
test -d Bar && du -a Bar

echo Waiting 10 seconds for any pending sync to complete...
sleep 10
du -a Foo | wc -l
test -d Bar && du -a Bar

@DeepDiver1975 @dragotin

[dragotin]

Probably another symptom of #10566

[jnweiger]

Note that #10566 is an issue with sqlite. I see this issue with both mysql and sqlite.

[nickvergessen]

Can you attach owncloud.log and potentially server log?

[jnweiger]

Folder with logs and script: https://s3.owncloud.com/owncloud/public.php?service=files&t=b7515cbafd17945ffbfca34ec69c23b7
reproduced with owncloud-8.0.0-5-debian_7.0

[PVince81]

Possibly related to #11795 too

[jnweiger]

@ogoffart @karlitschek
reproduced today with server 8.0.2 on Ubuntu 14.04 and client 1.8.0
new logfiles series added to https://s3.owncloud.com/owncloud/public.php?service=files&t=b7515cbafd17945ffbfca34ec69c23b7

Result of this run: 12 files remained in folder Foo from the original 21 files.

[karlitschek]

@MorrisJobke @felixboehm @DeepDiver1975 possible that this is the way to reproduce the issue we discussed yesterday.

[MorrisJobke]

I just talked to @jnfrmarks to include this into smashbox. There is already a first simple test with less files in the subfolder. We should extend the test case there to have this covered by an automatic test rather than this bash script. owncloud/smashbox#41

[nickvergessen]

Let's close this one in favor of the smashbox issue ( owncloud/smashbox#41 ). Once we have the smashbox test we can have a look to make it pass reliable