Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Windows, Linux, and macOS clients freezing when bulk uploading #4106

Closed
appliedbarrel opened this issue Dec 22, 2021 · 115 comments
Closed

Windows, Linux, and macOS clients freezing when bulk uploading #4106

appliedbarrel opened this issue Dec 22, 2021 · 115 comments

Comments

@appliedbarrel
Copy link

How to use GitHub

  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.

Expected behaviour

Client should upload files to server and notify the user of conflicts, server issues, or other errors with the upload.

Actual behaviour

Client freezes when trying to upload a file.
Folder sync connections which don't have changes work fine. Downloads from the server also work.
Pausing folders which have pending uploads prevents the client from freezing, but the client still freezes when resuming.
Cannot reproduce this problem on version 3.3.6 of the client.

Steps to reproduce

  1. Start client version 3.4.1
  2. Resume sync on one or more folders which have upload jobs
  3. Observe client beginning upload and progress bar moving
  4. Client freezes at the end of the upload
  5. Client program is completely unresponsive, has high CPU usage, and must be killed

Client configuration

Client version: 3.4.1

Operating system: Kubuntu 21.10

OS language: en-US

Qt version used by client package: ubuntu-5.13.0-22-generic

Client package: Nextcloud-3.4.1-x86_64.AppImage from this Github repo

Installation path of client: /home/[username] /Applications/Nextcloud-3.4.1-x86_64_5cbe3362585cfaace861289d8773450b.AppImage

Server configuration

Nextcloud version: 23

Storage backend (external storage): No external storage, Filesystem: ext4

Logs

Please use Gist (https://gist.github.com/) or a similar code paster for longer
logs.

  1. Client logfile:
    I won't upload a debug archive as it does not obfuscate sensitive information.

Here is the final 10 lines of the log produced by the client (GIthub Gist).

The final line of the logs is usually the same type of message:

[ info nextcloud.sync.networkjob /home/user/src/libsync/abstractnetworkjob.cpp:361 ]: OCC::PutMultiFileJob created for "https://[mywebsite.url]" + "" "OCC::BulkPropagatorJob"

Judging by other log messages, it looks like the empty string above "" is where a path should go. I don't know enough to know whether this is useful information or not.

There was one instance (could not reproduce) where the above line was followed by:

[ info nextcloud.sync.propagator.root.directory /home/user/src/libsync/owncloudpropagator.cpp:1214 ]: scheduleSelfOrChild OCC::PropagatorJob::Running pending uploads 0 subjobs state OCC::PropagatorJob::Running

  1. Web server error log (formatted)
    It spits this error out each time:

[webdav] Fatal: Unknown error while seeking content
POST /remote.php/dav/bulk
from [IP ADDRESS] by [nextcloud username] at 2021-12-22T16:13:18+00:00

  1. Server logfile: nextcloud log (data/nextcloud.log):
    I can possibly provide this in the future if necessary.
@fefrei
Copy link

fefrei commented Dec 23, 2021

I can reproduce this on Windows, too: 3.4.1 is affected, 3.3.6 is fine. I'm also seeing the Unknown error while seeking content log message on the server, but it does not appear every time when the client freezes, so it might be unrelated.

@allexzander
Copy link
Contributor

allexzander commented Dec 24, 2021

@mgallien Did you have anything similar on Linux?

UPDATE: Might be something related to Bulk Upload.
Last log line: 2021-12-22 16:03:12:037 [ info nextcloud.sync.networkjob /home/user/src/libsync/abstractnetworkjob.cpp:361 ]: OCC::PutMultiFileJob created for "https://cloud.[mywebsite.url]" + "" "OCC::BulkPropagatorJob"

@appliedbarrel appliedbarrel changed the title Linux Client appimage 3.4.1 Freezing when uploading files Linux Client AppImage 3.4.1 Freezing When Uploading Files Dec 25, 2021
@allexzander
Copy link
Contributor

Desktop logs might help solve this. You can upload them here https://cloud.nextcloud.com/s/DNQPFm7NXtQZtdS in case you prefer not to make them publicly available.

@allexzander
Copy link
Contributor

I can reproduce this on Windows, too: 3.4.1 is affected, 3.3.6 is fine. I'm also seeing the Unknown error while seeking content log message on the server, but it does not appear every time when the client freezes, so it might be unrelated.

Looks very similar to #4077

@lars-becker
Copy link

lars-becker commented Dec 28, 2021

I have the same problem with the build from Archlinux. Same message in the log file and not indicators what the problem might be. I uploaded the last few lines of the log file. Those with the unchanged files before, which made no problem and the transaction when the client freezes.

@lars-becker
Copy link

lars-becker commented Dec 28, 2021

@allexzander I found a corresponding log in the php-fpm journal:

Dez 28 17:09:43 jarvis Nextcloud[21403]: {"reqId":"0hgEj8EYAmtZMK2RzNhl","level":4,"time":"2021-12-28T17:09:43+01:00","remoteAddr":"IP_ADDRESS_REDACTED","user":"lars","app":"webdav","method":"POST","url":"/remote.php/dav/bulk","message":"{\"Exce
ption\":\"Sabre\\\\DAV\\\\Exception\",\"Message\":\"Unknown error while seeking content\",\"Code\":500,\"Trace\":[{\"file\":\"/var/www/nextcloud/apps/dav/lib/BulkUpload/MultipartRequestParser.php\",\"line\":129,\"function\":\"isAt\",\"cl
ass\":\"OCA\\\\DAV\\\\BulkUpload\\\\MultipartRequestParser\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/apps/dav/lib/BulkUpload/BulkUploadPlugin.php\",\"line\":69,\"function\":\"isAtLastBoundary\",\"class\":\"OCA\\\\DAV\\\\BulkUploa
d\\\\MultipartRequestParser\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php\",\"line\":89,\"function\":\"httpPost\",\"class\":\"OCA\\\\DAV\\\\BulkUpload\\\\BulkUploadPlugin\",\"type\":\
"->\"},{\"file\":\"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php\",\"line\":472,\"function\":\"emit\",\"class\":\"Sabre\\\\DAV\\\\Server\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php\"
,\"line\":253,\"function\":\"invokeMethod\",\"class\":\"Sabre\\\\DAV\\\\Server\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php\",\"line\":321,\"function\":\"start\",\"class\":\"Sabre\\\\DAV\\\\Serv
er\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/apps/dav/lib/Server.php\",\"line\":339,\"function\":\"exec\",\"class\":\"Sabre\\\\DAV\\\\Server\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php\",\"lin
e\":35,\"function\":\"exec\",\"class\":\"OCA\\\\DAV\\\\Server\",\"type\":\"->\"},{\"file\":\"/var/www/nextcloud/remote.php\",\"line\":166,\"args\":[\"/var/www/nextcloud/apps/dav/appinfo/v2/remote.php\"],\"function\":\"require_once\"}],\"
File\":\"/var/www/nextcloud/apps/dav/lib/BulkUpload/MultipartRequestParser.php\",\"Line\":111,\"CustomMessage\":\"--\"}","userAgent":"Mozilla/5.0 (Linux) mirall/3.4.1git (Nextcloud, arch-5.15.11-arch2-1 ClientArchitecture: x86_64 OsArchi
tecture: x86_64)","version":"23.0.0.10"}

Looks like a bug on the server side and missing error handling for this situation on the client side.

@fefrei
Copy link

fefrei commented Dec 28, 2021

I managed to capture a client log for such a freeze. I uploaded a redacted version of the log to link provided by @allexzander (20211228_1711_owncloud.redacted.log.zip), the last few lines of which I reproduced here:

2021-12-28 17:13:15:291 [ info nextcloud.sync.propagator.root.directory C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:1214 ]:	scheduleSelfOrChild OCC::PropagatorJob::Running pending uploads 2 subjobs state OCC::PropagatorJob::Running
2021-12-28 17:13:15:291 [ info nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.h:212 ]:	Starting CSyncEnums::CSYNC_INSTRUCTION_IGNORE propagation of "■■■■■■■■■■" by OCC::PropagateIgnoreJob(0x1ad7a6d7d00)
2021-12-28 17:13:15:291 [ warning nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:281 ]:	Could not complete propagation of "■■■■■■■■■■" by OCC::PropagateIgnoreJob(0x1ad7a6d7d00) with status OCC::SyncFileItem::FileIgnored and error: "File is listed on the ignore list."
2021-12-28 17:13:15:291 [ debug nextcloud.sync.statustracker C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\syncfilestatustracker.cpp:274 ]	[ OCC::SyncFileStatusTracker::slotItemCompleted ]:	Item completed "■■■■■■■■■■" OCC::SyncFileItem::FileIgnored CSyncEnums::CSYNC_INSTRUCTION_IGNORE
2021-12-28 17:13:15:291 [ warning nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\usermodel.cpp:559 ]:	Item  "■■■■■■■■■■"  retrieved resulted in  "File is listed on the ignore list."
2021-12-28 17:13:15:291 [ warning nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\usermodel.cpp:536 ]:	Item  "■■■■■■■■■■"  retrieved resulted in error  "File is listed on the ignore list."
2021-12-28 17:13:15:291 [ info nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\activitylistmodel.cpp:371 ]:	First checking for duplicates then add file to the notification list of ignored files:  "■■■■■■■■■■"
2021-12-28 17:13:15:295 [ info nextcloud.sync.propagator.root.directory C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:1214 ]:	scheduleSelfOrChild OCC::PropagatorJob::Running pending uploads 2 subjobs state OCC::PropagatorJob::Running
2021-12-28 17:13:15:295 [ info nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.h:212 ]:	Starting CSyncEnums::CSYNC_INSTRUCTION_IGNORE propagation of "■■■■■■■■■■" by OCC::PropagateIgnoreJob(0x1ad7a6d82a0)
2021-12-28 17:13:15:295 [ warning nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:281 ]:	Could not complete propagation of "■■■■■■■■■■" by OCC::PropagateIgnoreJob(0x1ad7a6d82a0) with status OCC::SyncFileItem::FileIgnored and error: "File is listed on the ignore list."
2021-12-28 17:13:15:295 [ debug nextcloud.sync.statustracker C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\syncfilestatustracker.cpp:274 ]	[ OCC::SyncFileStatusTracker::slotItemCompleted ]:	Item completed "■■■■■■■■■■" OCC::SyncFileItem::FileIgnored CSyncEnums::CSYNC_INSTRUCTION_IGNORE
2021-12-28 17:13:15:295 [ warning nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\usermodel.cpp:559 ]:	Item  "■■■■■■■■■■"  retrieved resulted in  "File is listed on the ignore list."
2021-12-28 17:13:15:295 [ warning nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\usermodel.cpp:536 ]:	Item  "■■■■■■■■■■"  retrieved resulted in error  "File is listed on the ignore list."
2021-12-28 17:13:15:295 [ info nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\activitylistmodel.cpp:371 ]:	First checking for duplicates then add file to the notification list of ignored files:  "■■■■■■■■■■"
2021-12-28 17:13:15:298 [ info nextcloud.sync.propagator.root.directory C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:1214 ]:	scheduleSelfOrChild OCC::PropagatorJob::Running pending uploads 2 subjobs state OCC::PropagatorJob::Running
2021-12-28 17:13:15:298 [ info nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.h:212 ]:	Starting CSyncEnums::CSYNC_INSTRUCTION_IGNORE propagation of "■■■■■■■■■■" by OCC::PropagateIgnoreJob(0x1ad7a6d8890)
2021-12-28 17:13:15:298 [ warning nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:281 ]:	Could not complete propagation of "■■■■■■■■■■" by OCC::PropagateIgnoreJob(0x1ad7a6d8890) with status OCC::SyncFileItem::FileIgnored and error: "File is listed on the ignore list."
2021-12-28 17:13:15:298 [ debug nextcloud.sync.statustracker C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\syncfilestatustracker.cpp:274 ]	[ OCC::SyncFileStatusTracker::slotItemCompleted ]:	Item completed "■■■■■■■■■■" OCC::SyncFileItem::FileIgnored CSyncEnums::CSYNC_INSTRUCTION_IGNORE
2021-12-28 17:13:15:298 [ warning nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\usermodel.cpp:559 ]:	Item  "■■■■■■■■■■"  retrieved resulted in  "File is listed on the ignore list."
2021-12-28 17:13:15:298 [ warning nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\usermodel.cpp:536 ]:	Item  "■■■■■■■■■■"  retrieved resulted in error  "File is listed on the ignore list."
2021-12-28 17:13:15:298 [ info nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\activitylistmodel.cpp:371 ]:	First checking for duplicates then add file to the notification list of ignored files:  "■■■■■■■■■■"
2021-12-28 17:13:15:301 [ info nextcloud.sync.propagator.root.directory C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:1214 ]:	scheduleSelfOrChild OCC::PropagatorJob::Running pending uploads 2 subjobs state OCC::PropagatorJob::Running
2021-12-28 17:13:15:301 [ info nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.h:212 ]:	Starting CSyncEnums::CSYNC_INSTRUCTION_IGNORE propagation of "■■■■■■■■■■" by OCC::PropagateIgnoreJob(0x1ad7a6d8ac0)
2021-12-28 17:13:15:301 [ warning nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:281 ]:	Could not complete propagation of "■■■■■■■■■■" by OCC::PropagateIgnoreJob(0x1ad7a6d8ac0) with status OCC::SyncFileItem::FileIgnored and error: "File is listed on the ignore list."
2021-12-28 17:13:15:301 [ debug nextcloud.sync.statustracker C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\syncfilestatustracker.cpp:274 ]	[ OCC::SyncFileStatusTracker::slotItemCompleted ]:	Item completed "■■■■■■■■■■" OCC::SyncFileItem::FileIgnored CSyncEnums::CSYNC_INSTRUCTION_IGNORE
2021-12-28 17:13:15:301 [ warning nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\usermodel.cpp:559 ]:	Item  "■■■■■■■■■■"  retrieved resulted in  "File is listed on the ignore list."
2021-12-28 17:13:15:301 [ warning nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\usermodel.cpp:536 ]:	Item  "■■■■■■■■■■"  retrieved resulted in error  "File is listed on the ignore list."
2021-12-28 17:13:15:301 [ info nextcloud.gui.activity C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\gui\tray\activitylistmodel.cpp:371 ]:	First checking for duplicates then add file to the notification list of ignored files:  "■■■■■■■■■■"
2021-12-28 17:13:15:301 [ info nextcloud.sync.propagator.root.directory C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:1238 ]:	OCC::SyncFileItem::Success slotSubJobsFinished OCC::PropagatorJob::Running pending uploads 2 subjobs state OCC::PropagatorJob::Finished
2021-12-28 17:13:15:301 [ info nextcloud.sync.propagator C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\owncloudpropagator.cpp:1270 ]:	PropagateRootDirectory::scheduleDelayedJobs
2021-12-28 17:13:15:301 [ info nextcloud.sync.checksums C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\checksums.cpp:230 ]:	Computing "MD5" checksum of "■■■■■■■■■■" in a thread
2021-12-28 17:13:15:301 [ info nextcloud.sync.checksums C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\checksums.cpp:230 ]:	Computing "MD5" checksum of "■■■■■■■■■■" in a thread
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 "■■■■■■■■■■"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 2 0
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 3 0
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 4 0
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 5 49156
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 6 1616699454
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:151 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 7 "MD5:50cf6b1f0de5932e99cdea807c24177f"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "INSERT OR REPLACE INTO uploadinfo (path, chunk, transferid, errorcount, size, modtime, contentChecksum) VALUES ( ?1 , ?2, ?3 , ?4 ,  ?5, ?6 , ?7 )"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\ownsql.cpp:328 ]	[ OCC::SqlQuery::exec ]:	Last exec affected 1 rows.
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\syncjournaldb.cpp:2439 ]	[ OCC::SyncJournalDb::commitInternal ]:	Transaction commit "Upload info" and starting new transaction
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:151 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 "■■■■■■■■■■"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT baseFileId, baseModtime, baseEtag, basePath FROM conflicts WHERE path=?1;"
2021-12-28 17:13:15:303 [ info nextcloud.sync.propagator.bulkupload C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\bulkpropagatorjob.cpp:183 ]:	"■■■■■■■■■■" transmission checksum "50cf6b1f0de5932e99cdea807c24177f" "■■■■■■■■■■"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 "■■■■■■■■■■"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 2 0
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 3 0
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 4 0
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 5 24413
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 6 1616699725
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:151 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 7 "MD5:a72fc90754d894d131eaa4c930c09593"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "INSERT OR REPLACE INTO uploadinfo (path, chunk, transferid, errorcount, size, modtime, contentChecksum) VALUES ( ?1 , ?2, ?3 , ?4 ,  ?5, ?6 , ?7 )"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\ownsql.cpp:328 ]	[ OCC::SqlQuery::exec ]:	Last exec affected 1 rows.
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\syncjournaldb.cpp:2439 ]	[ OCC::SyncJournalDb::commitInternal ]:	Transaction commit "Upload info" and starting new transaction
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common/ownsql.h:151 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 "■■■■■■■■■■"
2021-12-28 17:13:15:303 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT baseFileId, baseModtime, baseEtag, basePath FROM conflicts WHERE path=?1;"
2021-12-28 17:13:15:303 [ info nextcloud.sync.propagator.bulkupload C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\bulkpropagatorjob.cpp:183 ]:	"■■■■■■■■■■" transmission checksum "a72fc90754d894d131eaa4c930c09593" "■■■■■■■■■■"
2021-12-28 17:13:15:304 [ info nextcloud.sync.accessmanager C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\accessmanager.cpp:78 ]:	4 "" "https://cloud.example.invalid/remote.php/dav/bulk" has X-Request-ID "■■■■■■■■■■"
2021-12-28 17:13:15:304 [ debug nextcloud.sync.cookiejar C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\cookiejar.cpp:90 ]	[ OCC::CookieJar::cookiesForUrl ]:	QUrl("https://cloud.example.invalid/remote.php/dav/bulk") requests: (QNetworkCookie("__Host-nc_sameSiteCookielax=true; secure; HttpOnly; expires=Fri, 31-Dec-2100 23:59:59 GMT; domain=cloud.example.invalid; path=/"), QNetworkCookie("__Host-nc_sameSiteCookiestrict=true; secure; HttpOnly; expires=Fri, 31-Dec-2100 23:59:59 GMT; domain=cloud.example.invalid; path=/"), QNetworkCookie("oc_sessionPassphrase=■■■■■■■■■■; secure; HttpOnly; domain=cloud.example.invalid; path=/"), QNetworkCookie("■■■■■■■■■■; secure; HttpOnly; domain=cloud.example.invalid; path=/"))
2021-12-28 17:13:15:304 [ info nextcloud.sync.networkjob C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\abstractnetworkjob.cpp:361 ]:	OCC::PutMultiFileJob created for "https://cloud.example.invalid" + "" "OCC::BulkPropagatorJob"

What happened before is that I upgraded the client, started it, and it started to sync the folder in question. With no freeze occurring within about 30 minutes, I paused and restarted the sync, which almost immediately resulted in the freeze.

@Lord-Nightmare
Copy link

Lord-Nightmare commented Dec 29, 2021

This also happens to me with 3.4.1 on windows. Rolling back to 3.3.6 fixes it.
Same symptoms as above: the last client log file line before it hangs is:
2021-12-29 12:59:43:090 [ info nextcloud.sync.networkjob C:\Users\sysadmin\AppData\Local\Temp\2\windows-8634\client-building\desktop\src\libsync\abstractnetworkjob.cpp:361 ]: OCC::PutMultiFileJob created for "https://my.nextcloud.example.com" + "" "OCC::BulkPropagatorJob"

And the server logs show "Unknown error while seeking content"

@Quix0r
Copy link

Quix0r commented Dec 30, 2021

Confirmed with Linux version 5.10.0-8-amd64 ([email protected]) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.46-3 (2021-07-28)

Client 3.3.6 does not show these symptoms.

@mael2
Copy link

mael2 commented Jan 2, 2022

same thing happened to me on Win10 with 3.4.1
downgrading to 3.3.6 64bit resolved it apparently.

@bananer
Copy link

bananer commented Jan 7, 2022

I'm also affected here. Server is 23.0.0, using Windows client 3.4.1. Client freezes a few seconds after starting, waited 30+ minutes and did not resume. Tried deleting .sync.db files, but the error came back after sync resumed.

Downgrading client to 3.3.6 makes the problem go away.

Entry in Server logs:

[webdav] Fatal: Unknown error while seeking content

POST /remote.php/dav/bulk
from 172.19.0.11 by philip at 2022-01-07T14:46:05+00:00

{"reqId":"WKHg0G6RNgiuTb0lKM3O","level":4,"time":"2022-01-07T14:46:05+00:00","remoteAddr":"172.19.0.11","user":"philip","app":"webdav","method":"POST","url":"/remote.php/dav/bulk","message":"Unknown error while seeking content","userAgent":"Mozilla/5.0 (Windows) mirall/3.4.1stable-Win64 (build 20211221) (Nextcloud, windows-10.0.19044 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"23.0.0.10","exception":{"Exception":"Sabre\\DAV\\Exception","Message":"Unknown error while seeking content","Code":500,"Trace":[{"file":"/var/www/html/apps/dav/lib/BulkUpload/MultipartRequestParser.php","line":129,"function":"isAt","class":"OCA\\DAV\\BulkUpload\\MultipartRequestParser","type":"->","args":["--boundary_.oOo._Q1JHJPZzZXyeHgVSSeAKtc8EVuC12zVL--\r\n"]},{"file":"/var/www/html/apps/dav/lib/BulkUpload/BulkUploadPlugin.php","line":69,"function":"isAtLastBoundary","class":"OCA\\DAV\\BulkUpload\\MultipartRequestParser","type":"->","args":[]},{"file":"/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPost","class":"OCA\\DAV\\BulkUpload\\BulkUploadPlugin","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:POST",[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/apps/dav/lib/Server.php","line":339,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/remote.php","line":166,"args":["/var/www/html/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/var/www/html/apps/dav/lib/BulkUpload/MultipartRequestParser.php","Line":111,"CustomMessage":"--"},"id":"61d8522d3155f"}

@Derkades
Copy link

Derkades commented Jan 8, 2022

It doesn't freeze permanently for me (hangs for a few seconds at a time). 100% CPU usage on a single core during bulk upload. It manages to upload 100 files every 30s-1m, which is slower than 3.3.6 was with one request per file.

@appliedbarrel
Copy link
Author

I have also had the issue on Windows 10, and downgrading to 3.3.6 fixed it on there too.

It doesn't freeze permanently for me (hangs for a few seconds at a time). 100% CPU usage on a single core during bulk upload. It manages to upload 100 files every 30s-1m, which is slower than 3.3.6 was with one request per file.

I've also noticed that it uses 100% CPU on a single core on both Windows and my Linux system. It never actually completed on either system (after leaving it in background for over an hour), but I was only uploading one or two file changes at a time.

@appliedbarrel appliedbarrel changed the title Linux Client AppImage 3.4.1 Freezing When Uploading Files Windows and Linux (AppImage) clients 3.4.1 Freezing When Uploading Files Jan 8, 2022
@willvousden
Copy link

This is happening to me with the macOS client as well (version 3.4.1). File is not successfully uploaded; client hangs and must be killed.

@allexzander
Copy link
Contributor

Could be related to #4141

@sigreer
Copy link

sigreer commented Jan 11, 2022

Confirmed here (Linux Manjaro Gnome). Also log files generated in ~/.config/Nextcloud/logs many, many GB in size.

Downgrading to the previous version (had to use AppImage) fixed the problem.

@landryb
Copy link

landryb commented Jan 20, 2022

same error log since the update to nextcloud 23.0.0, worked fine with nextcloud 21.0.6, using client 3.4.1. both clients and servers are on OpenBSD, so unrelated to appimage.

{
  "reqId": "7qEGZTcQeMdpw3QOctgu",
  "level": 4,
  "time": "2022-01-20T09:53:44+00:00",
  "remoteAddr": "10.246.200.35",
  "user": "landry",
  "app": "webdav",
  "method": "POST",
  "url": "/remote.php/dav/bulk",
  "message": "Unknown error while seeking content",
  "userAgent": "Mozilla/5.0 (OpenBSD) mirall/3.4.1git (Nextcloud, unknown-7.0 ClientArchitecture: x86_64 OsArchitecture: x86_64)",
  "version": "23.0.0.10",
  "exception": {
    "Exception": "Sabre\\DAV\\Exception",
    "Message": "Unknown error while seeking content",
    "Code": 500,
    "Trace": [
      {
        "file": "/nextcloud/apps/dav/lib/BulkUpload/MultipartRequestParser.php",
        "line": 129,
        "function": "isAt",
        "class": "OCA\\DAV\\BulkUpload\\MultipartRequestParser",
        "type": "->"
      },
      {
        "file": "/nextcloud/apps/dav/lib/BulkUpload/BulkUploadPlugin.php",
        "line": 69,
        "function": "isAtLastBoundary",
        "class": "OCA\\DAV\\BulkUpload\\MultipartRequestParser",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
        "line": 89,
        "function": "httpPost",
        "class": "OCA\\DAV\\BulkUpload\\BulkUploadPlugin",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 472,
        "function": "emit",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 253,
        "function": "invokeMethod",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 321,
        "function": "start",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/nextcloud/apps/dav/lib/Server.php",
        "line": 339,
        "function": "exec",
        "class": "Sabre\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/nextcloud/apps/dav/appinfov2/remote.php",
        "line": 35,
        "function": "exec",
        "class": "OCA\\DAV\\Server",
        "type": "->"
      },
      {
        "file": "/nextcloud/remote.php",
        "line": 166,
        "args": [
          "/nextcloud/apps/dav/appinfo/v2/remote.php"
        ],
        "function": "require_once"
      }
    ],
    "File": "/nextcloud/apps/dav/lib/BulkUpload/MultipartRequestParser.php",
    "Line": 111,
    "CustomMessage": "--"
  }
}

@aduarte9
Copy link

aduarte9 commented Jan 2, 2023

I've "fixed" it by disabling circles app

@BtbN
Copy link

BtbN commented Jan 2, 2023

I never had the Circles App enabled in the first place and still had it happen.
The only reliable workaround for me has been either disabling bulk uploads, or not setting a speed limit on the client.

@aduarte9
Copy link

aduarte9 commented Jan 3, 2023

I never had the Circles App enabled in the first place and still had it happen. The only reliable workaround for me has been either disabling bulk uploads, or not setting a speed limit on the client.

Well, actually I've started by disabling bulk uploads and speed limit on the client but the issue continued until I've disabled Circles APP so, propably all this three were causing the issue

@Macrum
Copy link

Macrum commented Jan 5, 2023

I confirm the problem with newest client and NC 25. Disabling speed limit in Windows client resolved situation.

I confirm the same "solution" for the linux client. When disabling the speed limit it works.

@theinvisible
Copy link

Same here, disabling upload limit solved the problem of hanging client on file uploads. Using Nextcloud Server 25.0.2 and Desktop Client 3.6.4 on Windows 11

@justinsangoi
Copy link

Same here, disabling upload limit solved the problem of hanging client on file uploads. Using Nextcloud Server 25.0.2 and Desktop Client 3.6.4 on Windows 11

This solved my freezing issue as well. It was set to automatic limit. I'm on Desktop client 3.6.4 and server 25.0.1.

This has been frustrating me for days. Behaviour was as described. The client would attempt to sync (I'm using virtual files) and it would seize up and prevent the context menu in Windows from being able to be opened until I killed the Nextcloud cleint process. It would use CPU endlessly. Enabling logs on the client recorded no errors. Server did not record relevant errors.

The only way I could interact with the client was to start it and immediately disable the sync before it locked up.

@tallzabby
Copy link

Happens to me even without the speed limit.

@EliottAndrieuINOVEFA
Copy link

Disabling bulk upload and speed limit doesn't fix the problem ; had to remove thousand of files in folder to fix it

@labor4
Copy link

labor4 commented Jan 16, 2023

maybe related
nextcloud/server#29985

this made bulk upload work first time for me, as per the apache request logs.

@SunnyPaprika
Copy link

Same here with client 3.6.5 and Server 25.0.3 under Windows 10

Uploading large amounts of files (31k Files, 50GB) at the same time causes the desktop client to consume large amounts of ram (~28G), becoming unresponsive and making right click on a file freeze (if the client is in the context menu).

The Server Log shows:

{"reqId":"Ko73h5uiNcnTOsVYCHdY","level":3,"time":"2023-01-19T12:28:45+00:00","remoteAddr":"XXXXXX","user":"XXXXXX","app":"no app in context","method":"POST","url":"/remote.php/dav/bulk","message":"Computed md5 hash is incorrect.","userAgent":"Mozilla/5.0 (Windows) mirall/3.6.4stable-Win64 (build 20221208) (Nextcloud, windows-10.0.19044 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.3.2","data":[]}

And

{"reqId":"onXbPyAQqtM1wR59qP1w","level":3,"time":"2023-01-20T08:17:21+00:00","remoteAddr":"XXXXXX","user":"XXXXXX","app":"webdav","method":"POST","url":"/remote.php/dav/bulk","message":"Unknown error while seeking content","userAgent":"Mozilla/5.0 (Windows) mirall/3.6.5stable-Win64 (build 20230118) (Nextcloud, windows-10.0.19044 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"25.0.3.2","exception":{"Exception":"Sabre\\DAV\\Exception","Message":"Unknown error while seeking content","Code":500,"Trace":[{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/BulkUpload/MultipartRequestParser.php","line":129,"function":"isAt","class":"OCA\\DAV\\BulkUpload\\MultipartRequestParser","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/BulkUpload/BulkUploadPlugin.php","line":71,"function":"isAtLastBoundary","class":"OCA\\DAV\\BulkUpload\\MultipartRequestParser","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpPost","class":"OCA\\DAV\\BulkUpload\\BulkUploadPlugin","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/usr/share/webapps/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/lib/Server.php","line":360,"function":"exec","class":"Sabre\\DAV\\Server","type":"->"},{"file":"/usr/share/webapps/nextcloud/apps/dav/appinfo/v2/remote.php","line":35,"function":"exec","class":"OCA\\DAV\\Server","type":"->"},{"file":"/usr/share/webapps/nextcloud/remote.php","line":171,"args":["/usr/share/webapps/nextcloud/apps/dav/appinfo/v2/remote.php"],"function":"require_once"}],"File":"/usr/share/webapps/nextcloud/apps/dav/lib/BulkUpload/MultipartRequestParser.php","Line":111,"message":"Unknown error while seeking content","exception":[],"CustomMessage":"Unknown error while seeking content"},"id":"63ca549e3a6f5"}

PostgreSQL log shows:

2023-01-19 13:28:44.965 CET [2228472] ERROR:  duplicate key value violates unique constraint "oc_file_metadata_pkey"
2023-01-19 13:28:44.965 CET [2228472] DETAIL:  Key (id, group_name)=(33821, size) already exists.
2023-01-19 13:28:44.965 CET [2228472] STATEMENT:  INSERT INTO "oc_file_metadata" ("group_name", "id", "metadata") VALUES($1, $2, $3)

The Desktop client log shows:

2023-01-20 08:57:45:405 [ debug nextcloud.sync.statustracker C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\libsync\syncfilestatustracker.cpp:229 ]	[ OCC::SyncFileStatusTracker::slotAboutToPropagate ]:	Investigating "share/zeichnung/Canvas-Logo.zip" OCC::SyncFileItem::NoStatus CSyncEnums::CSYNC_INSTRUCTION_NEW OCC::SyncFileItem::Up
2023-01-20 08:57:45:405 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 -4227333617020141538
2023-01-20 08:57:45:405 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted,   lock, lockOwnerDisplayName, lockOwnerId, lockType, lockOwnerEditor, lockTime, lockTimeout  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
2023-01-20 08:57:45:405 [ debug nextcloud.sync.statustracker C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\libsync\syncfilestatustracker.cpp:229 ]	[ OCC::SyncFileStatusTracker::slotAboutToPropagate ]:	Investigating "share/zeichnung/Character-Sheet.pdf" OCC::SyncFileItem::NoStatus CSyncEnums::CSYNC_INSTRUCTION_NEW OCC::SyncFileItem::Up
2023-01-20 08:57:45:407 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 -2885302577579614120
2023-01-20 08:57:45:407 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted,   lock, lockOwnerDisplayName, lockOwnerId, lockType, lockOwnerEditor, lockTime, lockTimeout  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
2023-01-20 08:57:45:407 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 -1972842117078875464

[...]

2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 -1972842117078875464
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted,   lock, lockOwnerDisplayName, lockOwnerId, lockType, lockOwnerEditor, lockTime, lockTimeout  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 6838229285097880166
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted,   lock, lockOwnerDisplayName, lockOwnerId, lockType, lockOwnerEditor, lockTime, lockTimeout  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 902098477930242311
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted,   lock, lockOwnerDisplayName, lockOwnerId, lockType, lockOwnerEditor, lockTime, lockTimeout  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 -5183247471363569001
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted,   lock, lockOwnerDisplayName, lockOwnerId, lockType, lockOwnerEditor, lockTime, lockTimeout  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 -2064615103904087547
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted,   lock, lockOwnerDisplayName, lockOwnerId, lockType, lockOwnerEditor, lockTime, lockTimeout  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common/ownsql.h:145 ]	[ OCC::SqlQuery::bindValue ]:	SQL bind 1 -1560858303716470105
2023-01-20 08:57:46:342 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path, inode, modtime, type, md5, fileid, remotePerm, filesize,  ignoredChildrenRemote, contentchecksumtype.name || ':' || contentChecksum, e2eMangledName, isE2eEncrypted,   lock, lockOwnerDisplayName, lockOwnerId, lockType, lockOwnerEditor, lockTime, lockTimeout  FROM metadata  LEFT JOIN checksumtype as contentchecksumtype ON metadata.contentChecksumTypeId == contentchecksumtype.id WHERE phash=?1"
2023-01-20 08:57:46:356 [ info nextcloud.sync.engine C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\libsync\syncengine.cpp:755 ]:	#### Reconcile (aboutToPropagate OK) ####################################################  1569807 ms
2023-01-20 08:57:46:357 [ debug nextcloud.sync.database C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\syncjournaldb.cpp:2418 ]	[ OCC::SyncJournalDb::commitInternal ]:	Transaction commit "post treewalk" and starting new transaction
2023-01-20 08:57:46:360 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT tmpfile, etag, errorcount, path FROM downloadinfo"
2023-01-20 08:57:46:366 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path,transferid FROM uploadinfo"
2023-01-20 08:57:46:385 [ debug nextcloud.sync.database.sql C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\ownsql.cpp:295 ]	[ OCC::SqlQuery::exec ]:	SQL exec "SELECT path FROM blacklist"
2023-01-20 08:57:46:385 [ debug nextcloud.sync.database C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\common\syncjournaldb.cpp:2418 ]	[ OCC::SyncJournalDb::commitInternal ]:	Transaction commit "post stale entry removal" and starting new transaction
2023-01-20 08:57:46:391 [ info nextcloud.sync.engine C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\libsync\syncengine.cpp:809 ]:	#### Post-Reconcile end ####################################################  1569843 ms
2023-01-20 08:57:46:393 [ info nextcloud.gui.folder.manager C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\gui\folderman.cpp:896 ]:	Etag poll timer timeout
2023-01-20 08:57:46:393 [ info nextcloud.gui.folder.manager C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\gui\folderman.cpp:900 ]:	Folders to sync: 1
2023-01-20 08:57:46:393 [ info nextcloud.gui.folder.manager C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\gui\folderman.cpp:910 ]:	Number of folders that don't use push notifications: 1
2023-01-20 08:57:46:394 [ info nextcloud.gui.folder.manager C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\gui\folderman.cpp:927 ]:	Run etag job on folder OCC::Folder(0x206519f7f90)
2023-01-20 08:57:46:394 [ info nextcloud.gui.folder.manager C:\Users\sysadmin\AppData\Local\Temp\2\windows-13461\client-building\desktop\src\gui\folderman.cpp:933 ]:	Can not run etag job: Sync is running

Consumed memory:
image

@Hightmar
Copy link

Hightmar commented Feb 3, 2023

#5094

It turned out that the bulk upload feature is in error. As a workaround, I turned off bulk upload in the server’s config.php file by adding the following line:

'bulkupload.enabled' => false,

Try to add 'bulkupload.enabled' => false,' in the config file

@MiniMaleN
Copy link

Can confirm that this is still an issue with the newest version of the app on my machine as well.
Downgrading client to 3.3.6 makes the problem go away.
I couldn't even get into the app, it just hanged itself and used 25-28% CPU all the time.

@cdnvn
Copy link

cdnvn commented Feb 10, 2023

I had a freezing issue syncing to my v25.03 Windows/WSL2 server after my desktop client upgraded to v3.7.X. Downgraded client back to v3.6.6 and the sync is now working as it was before the upgrade.

@awigen
Copy link

awigen commented Feb 11, 2023

I ran into the 100% CPU issue and infinite hang when I enabled upload bandwidth limiting.
Once I edited ~/.config/Nextcloud/nextcloud.cfg and turned useUploadLimit=0 the issue is gone.

@raychani
Copy link

I had a freezing also with my Nextcloud Hub 3(25.0.3) docker server and Nextcloud-3.7.3-x64.msi, Nextcloud-3.6.6-x64.msi on Win 11. Using 10% CPU of my intel 12700. There was no logs for this problem. I never used the Circles app. I didn't try the bulkupload disabled.
3.3.6 works good.

@mgallien mgallien added the approved bug approved by the team label Feb 23, 2023
@alerque
Copy link

alerque commented Apr 13, 2023

BTW the title of this issue says "Windows and Linux clients", but this affects macOS clients too. I forgot to disable bulk uploads an my server when bumping to Nextcloud 26 recently and several macOS clients froze up. Same deal as I've seen on Linux.

@appliedbarrel appliedbarrel changed the title Windows and Linux clients freezing when bulk uploading Windows, Linux, and macOS clients freezing when bulk uploading Apr 13, 2023
@riweckr
Copy link

riweckr commented May 4, 2023

I ran into the 100% CPU issue and infinite hang when I enabled upload bandwidth limiting. Once I edited ~/.config/Nextcloud/nextcloud.cfg and turned useUploadLimit=0 the issue is gone.

Running a 25.0.6 server and a 3.8.1 Win 10 client, setting useUploadLimit=0 and useDownloadLimit=0 fixed it.

@mickare
Copy link

mickare commented May 24, 2023

Hey, I discovered that the Nextcloud desktop client is freezing when a end-to-end encrypted folder is selected for sync.
A freshly configured client (rm ~/.config/Nextcloud) is working fine.
As soon as I add the E2E folder for the sync the client freezes immediately.

Maybe someone can confirm?
Or is this another issue?

@alerque
Copy link

alerque commented May 24, 2023

@mickare That is going to be a different issue, please file a new tracking issue. This one is about bulk upload and is repeatable with no E2E in sight.

@mickare
Copy link

mickare commented May 24, 2023

@mickare That is going to be a different issue, please file a new tracking issue. This one is about bulk upload and is repeatable with no E2E in sight.

Thank you! Yes, I also just found #5657 in the release notes of 3.8.2.
I'll try it.

@appliedbarrel
Copy link
Author

Fixed by pull #5680 alongside issue #5094

Tested with server setting:
'bulkupload.enabled' => true,

  1. Paste files into syncing directory
  2. Start Desktop client or resume sync
    Client settings:
    Upload bandwidth: No limit -> Client uploads files as expected
    Upload bandwidth: Limit automatically -> Client uploads files as expected

@alerque
Copy link

alerque commented Jul 4, 2023

Wow after 2 years and 26 versions and almost no acknowledgement I did not have this turning up fixed on my bingo card today, but I'll take it.

@TMUniversal
Copy link

This may have resurfaced (#5680 (comment)).
I also experience a freeze when uploading 100+ files in bulk, with upload bandwith limit set to auto.
The problem is not present with no limit.

Client version: 3.14.3

@alerque
Copy link

alerque commented Nov 10, 2024

Yes this was not actually fixed. There are open issues tracking it. See e.g. #6990.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests