Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Command-line clients fail to download files after upgrade to Nextcloud 13 #8367

Closed
BastiTee opened this issue Feb 15, 2018 · 2 comments
Closed

Command-line clients fail to download files after upgrade to Nextcloud 13 #8367

BastiTee opened this issue Feb 15, 2018 · 2 comments

Comments

@BastiTee
Copy link

Steps to reproduce

Prerequisite: I have a self-hosted nextcloud instance running, that is accessible via files.mydomain.com and anotherdomain.com. I was using the command-line calls as described below for many weeks without problems. After upgrading from NC 11 to NC 13 via NC 12, the command line clients don't download any files anymore.
The GUI clients still work, including an Ubuntu-based Owncloud client as well as my Android client.

Steps:

  • Create a folder "Test" inside nextcloud root-folder for user
  • Create a file named "hello-world.txt" inside the "Test" folder
  • Double check that file is accessible via nextcloud gui client etc.

nextcloud
nextcloud2

  • Run the following script:
#!/bin/bash

function run_test() {
    
    domain=$1
    client=$2

    rm -rf /tmp/oc-test
    mkdir -p /tmp/oc-test

    $client \
        -u username \
        -p 'password' \
        /tmp/oc-test \
        https://$domain/remote.php/webdav/Test/

    [ -f /tmp/oc-test/hello-world.txt ] && result="success" || result="failure"

    echo "test for domain '$domain' with client '$client': $result"
}

run_test "files.mydomain.com" "owncloudcmd" 
run_test "files.mydomain.com" "nextcloudcmd" 
run_test "anotherdomain.com" "owncloudcmd" 
run_test "anotherdomain.com" "nextcloudcmd"

Expected behaviour

All four tests should not fail, i.e., the client should download the hello-world.txt file.

Actual behaviour

Files are not downloaded. See log files:

02-15 10:02:23:518 [ info sync.accessmanager ]:	2 "" "https://files.mydomain.com/ocs/v1.php/cloud/capabilities?format=json" has X-Request-ID "2c674aea-ec8d-4613-b7eb-9828805cd81d"
02-15 10:02:23:519 [ info sync.networkjob ]:	OCC::JsonApiJob created for "https://files.mydomain.com" + "ocs/v1.php/cloud/capabilities" ""
02-15 10:02:24:543 [ info sync.networkjob.jsonapi ]:	JsonApiJob of QUrl("https://files.mydomain.com/ocs/v1.php/cloud/capabilities?format=json") FINISHED WITH STATUS QNetworkReply::NetworkError(NoError) ""
02-15 10:02:24:544 [ info sync.clientproxy ]:	Set proxy configuration to use system configuration
02-15 10:02:24:544 [ info sync.engine ]:	There are 13137354752 bytes available at "/tmp/oc-test/"
02-15 10:02:24:544 [ info sync.engine ]:	New sync (no sync journal exists)
02-15 10:02:24:544 [ info sync.engine ]:	"Using Qt 5.6.2 SSL library OpenSSL 1.0.2g  1 Mar 2016 on Ubuntu 17.10"
02-15 10:02:24:545 [ info sync.database ]:	sqlite3 version "3.19.3"
02-15 10:02:24:545 [ info sync.database ]:	sqlite3 journal_mode= "wal"
02-15 10:02:24:560 [ info sync.database ]:	possibleUpgradeFromMirall_1_5 detected!
02-15 10:02:24:567 [ info sync.database ]:	Forcing remote re-discovery by deleting folder Etags
02-15 10:02:24:568 [ info sync.engine ]:	NOT Using Selective Sync
02-15 10:02:24:568 [ info sync.engine ]:	#### Discovery start ####################################################
02-15 10:02:24:568 [ info sync.engine ]:	Server "" 
02-15 10:02:24:569 [ info sync.accessmanager ]:	6 "PROPFIND" "https://files.mydomain.com/remote.php/dav/files/username/Test" has X-Request-ID "2a507f38-6e41-44af-a691-ec0db7b4876b"
02-15 10:02:24:569 [ info sync.networkjob ]:	OCC::LsColJob created for "https://files.mydomain.com" + "/Test" "OCC::DiscoverySingleDirectoryJob"
02-15 10:02:25:360 [ warning sync.networkjob ]:	QNetworkReply::NetworkError(ContentNotFoundError) "Server replied \"404 Not Found\" to \"PROPFIND https://files.mydomain.com/remote.php/dav/files/username/Test\"" QVariant(int, 404)
02-15 10:02:25:360 [ info sync.networkjob.lscol ]:	LSCOL of QUrl("https://files.mydomain.com/remote.php/dav/files/username/Test") FINISHED WITH STATUS QNetworkReply::NetworkError(ContentNotFoundError) "Server replied \"404 Not Found\" to \"PROPFIND https://files.mydomain.com/remote.php/dav/files/username/Test\""
02-15 10:02:25:360 [ warning sync.discovery ]:	LSCOL job error "Error transferring https://files.mydomain.com/remote.php/dav/files/username/Test - server replied: Not Found" 404 QNetworkReply::NetworkError(ContentNotFoundError)
02-15 10:02:25:360 [ warning sync.engine ]:	ERROR during  csync_update :  "CSync failed to access "
02-15 10:02:25:361 [ info sync.database ]:	Closing DB "/tmp/oc-test/._sync_3e02db10375e.db"
02-15 10:02:25:376 [ info sync.engine ]:	CSync run took  807 ms
02-15 10:02:25:376 [ info sync.database ]:	Closing DB "/tmp/oc-test/._sync_3e02db10375e.db"

/tmp/oc-test
/tmp/oc-test/._sync_3e02db10375e.db

test for domain 'files.mydomain.com' with client 'owncloudcmd': failure

[9] csync_exclude_load  Adding entry: *~
[9] csync_exclude_load  Adding entry: ~$*
[9] csync_exclude_load  Adding entry: .~lock.*
[9] csync_exclude_load  Adding entry: ~*.tmp
[9] csync_exclude_load  Adding entry: ]*.~*
*9] csync_exclude_load  Adding entry: ]Icon
[9] csync_exclude_load  Adding entry: ].DS_Store
[9] csync_exclude_load  Adding entry: ].ds_store
[9] csync_exclude_load  Adding entry: ._*
[9] csync_exclude_load  Adding entry: ]Thumbs.db
[9] csync_exclude_load  Adding entry: desktop.ini
[9] csync_exclude_load  Adding entry: System Volume Information
[9] csync_exclude_load  Adding entry: .*.sw?
[9] csync_exclude_load  Adding entry: .*.*sw?
[9] csync_exclude_load  Adding entry: ].TemporaryItems
[9] csync_exclude_load  Adding entry: ].Trashes
[9] csync_exclude_load  Adding entry: ].DocumentRevisions-V100
[9] csync_exclude_load  Adding entry: ].Trash-*
[9] csync_exclude_load  Adding entry: .fseventd
[9] csync_exclude_load  Adding entry: .apdisk
[9] csync_exclude_load  Adding entry: .htaccess
[9] csync_exclude_load  Adding entry: .directory
[9] csync_exclude_load  Adding entry: *.part
[9] csync_exclude_load  Adding entry: *.filepart
[9] csync_exclude_load  Adding entry: *.crdownload
[9] csync_exclude_load  Adding entry: *.kate-swp
[9] csync_exclude_load  Adding entry: *.gnucash.tmp-*
[9] csync_exclude_load  Adding entry: .synkron.*
[9] csync_exclude_load  Adding entry: .sync.ffs_db
[9] csync_exclude_load  Adding entry: .symform
[9] csync_exclude_load  Adding entry: .symform-store
[9] csync_exclude_load  Adding entry: .fuse_hidden*
[9] csync_exclude_load  Adding entry: *.unison
[9] csync_exclude_load  Adding entry: .nfs*
[9] csync_exclude_load  Adding entry: My Saved Places.
[6] csync_statedb_load  sqlite3 version "3.19.3"
[6] csync_statedb_load  Success
[7] csync_memstat_check  Memory: 571992K total size, 22448K resident, 20260K shared
[9] csync_walker  file: /tmp/oc-test/._sync_3e02db10375e.db-shm [inode=1707082 size=32768]
[9] _csync_detect_update  ._sync_3e02db10375e.db-shm excluded  (1)
[9] csync_walker  file: /tmp/oc-test/._sync_3e02db10375e.db-wal [inode=1707076 size=127752]
[9] _csync_detect_update  ._sync_3e02db10375e.db-wal excluded  (1)
[9] csync_walker  file: /tmp/oc-test/._sync_3e02db10375e.db [inode=1707059 size=4096]
[9] _csync_detect_update  ._sync_3e02db10375e.db excluded  (1)
[9] csync_ftw   <= Closing walk for /tmp/oc-test with read_from_db 0
[8] csync_update  Update detection for local replica took 0,00 seconds walking 0 files.
[7] csync_memstat_check  Memory: 572020K total size, 22448K resident, 20260K shared
[6] csync_statedb_close  sqlite3_close=0

/tmp/oc-test
/tmp/oc-test/._sync_3e02db10375e.db

test for domain 'files.mydomain.com' with client 'nextcloudcmd': failure

02-15 10:02:26:712 [ info sync.accessmanager ]:	2 "" "https://anotherdomain.com/ocs/v1.php/cloud/capabilities?format=json" has X-Request-ID "a38ca35f-f7a5-49bd-8a3c-24e63aa498e7"
02-15 10:02:26:713 [ info sync.networkjob ]:	OCC::JsonApiJob created for "https://anotherdomain.com" + "ocs/v1.php/cloud/capabilities" ""
02-15 10:02:27:407 [ info sync.networkjob.jsonapi ]:	JsonApiJob of QUrl("https://anotherdomain.com/ocs/v1.php/cloud/capabilities?format=json") FINISHED WITH STATUS QNetworkReply::NetworkError(NoError) ""
02-15 10:02:27:408 [ info sync.clientproxy ]:	Set proxy configuration to use system configuration
02-15 10:02:27:408 [ info sync.engine ]:	There are 13137354752 bytes available at "/tmp/oc-test/"
02-15 10:02:27:408 [ info sync.engine ]:	New sync (no sync journal exists)
02-15 10:02:27:408 [ info sync.engine ]:	"Using Qt 5.6.2 SSL library OpenSSL 1.0.2g  1 Mar 2016 on Ubuntu 17.10"
02-15 10:02:27:409 [ info sync.database ]:	sqlite3 version "3.19.3"
02-15 10:02:27:409 [ info sync.database ]:	sqlite3 journal_mode= "wal"
02-15 10:02:27:423 [ info sync.database ]:	possibleUpgradeFromMirall_1_5 detected!
02-15 10:02:27:429 [ info sync.database ]:	Forcing remote re-discovery by deleting folder Etags
02-15 10:02:27:429 [ info sync.engine ]:	NOT Using Selective Sync
02-15 10:02:27:429 [ info sync.engine ]:	#### Discovery start ####################################################
02-15 10:02:27:429 [ info sync.engine ]:	Server "" 
02-15 10:02:27:430 [ info sync.accessmanager ]:	6 "PROPFIND" "https://anotherdomain.com/remote.php/dav/files/username/Test" has X-Request-ID "8eada83e-958d-421a-99f7-f5d5a43cfe82"
02-15 10:02:27:431 [ info sync.networkjob ]:	OCC::LsColJob created for "https://anotherdomain.com" + "/Test" "OCC::DiscoverySingleDirectoryJob"
02-15 10:02:28:025 [ warning sync.networkjob ]:	QNetworkReply::NetworkError(ContentNotFoundError) "Server replied \"404 Not Found\" to \"PROPFIND https://anotherdomain.com/remote.php/dav/files/username/Test\"" QVariant(int, 404)
02-15 10:02:28:025 [ info sync.networkjob.lscol ]:	LSCOL of QUrl("https://anotherdomain.com/remote.php/dav/files/username/Test") FINISHED WITH STATUS QNetworkReply::NetworkError(ContentNotFoundError) "Server replied \"404 Not Found\" to \"PROPFIND https://anotherdomain.com/remote.php/dav/files/username/Test\""
02-15 10:02:28:025 [ warning sync.discovery ]:	LSCOL job error "Error transferring https://anotherdomain.com/remote.php/dav/files/username/Test - server replied: Not Found" 404 QNetworkReply::NetworkError(ContentNotFoundError)
02-15 10:02:28:025 [ warning sync.engine ]:	ERROR during  csync_update :  "CSync failed to access "
02-15 10:02:28:025 [ info sync.database ]:	Closing DB "/tmp/oc-test/._sync_c74d5f875239.db"
02-15 10:02:28:038 [ info sync.engine ]:	CSync run took  609 ms
02-15 10:02:28:039 [ info sync.database ]:	Closing DB "/tmp/oc-test/._sync_c74d5f875239.db"

/tmp/oc-test
/tmp/oc-test/._sync_c74d5f875239.db

test for domain 'anotherdomain.com' with client 'owncloudcmd': failure

[9] csync_exclude_load  Adding entry: *~
[9] csync_exclude_load  Adding entry: ~$*
[9] csync_exclude_load  Adding entry: .~lock.*
[9] csync_exclude_load  Adding entry: ~*.tmp
[9] csync_exclude_load  Adding entry: ]*.~*
*9] csync_exclude_load  Adding entry: ]Icon
[9] csync_exclude_load  Adding entry: ].DS_Store
[9] csync_exclude_load  Adding entry: ].ds_store
[9] csync_exclude_load  Adding entry: ._*
[9] csync_exclude_load  Adding entry: ]Thumbs.db
[9] csync_exclude_load  Adding entry: desktop.ini
[9] csync_exclude_load  Adding entry: System Volume Information
[9] csync_exclude_load  Adding entry: .*.sw?
[9] csync_exclude_load  Adding entry: .*.*sw?
[9] csync_exclude_load  Adding entry: ].TemporaryItems
[9] csync_exclude_load  Adding entry: ].Trashes
[9] csync_exclude_load  Adding entry: ].DocumentRevisions-V100
[9] csync_exclude_load  Adding entry: ].Trash-*
[9] csync_exclude_load  Adding entry: .fseventd
[9] csync_exclude_load  Adding entry: .apdisk
[9] csync_exclude_load  Adding entry: .htaccess
[9] csync_exclude_load  Adding entry: .directory
[9] csync_exclude_load  Adding entry: *.part
[9] csync_exclude_load  Adding entry: *.filepart
[9] csync_exclude_load  Adding entry: *.crdownload
[9] csync_exclude_load  Adding entry: *.kate-swp
[9] csync_exclude_load  Adding entry: *.gnucash.tmp-*
[9] csync_exclude_load  Adding entry: .synkron.*
[9] csync_exclude_load  Adding entry: .sync.ffs_db
[9] csync_exclude_load  Adding entry: .symform
[9] csync_exclude_load  Adding entry: .symform-store
[9] csync_exclude_load  Adding entry: .fuse_hidden*
[9] csync_exclude_load  Adding entry: *.unison
[9] csync_exclude_load  Adding entry: .nfs*
[9] csync_exclude_load  Adding entry: My Saved Places.
[6] csync_statedb_load  sqlite3 version "3.19.3"
[6] csync_statedb_load  Success
[7] csync_memstat_check  Memory: 571992K total size, 25144K resident, 20920K shared
[9] csync_walker  file: /tmp/oc-test/._sync_c74d5f875239.db-wal [inode=1707076 size=127752]
[9] _csync_detect_update  ._sync_c74d5f875239.db-wal excluded  (1)
[9] csync_walker  file: /tmp/oc-test/._sync_c74d5f875239.db-shm [inode=1707082 size=32768]
[9] _csync_detect_update  ._sync_c74d5f875239.db-shm excluded  (1)
[9] csync_walker  file: /tmp/oc-test/._sync_c74d5f875239.db [inode=1707059 size=4096]
[9] _csync_detect_update  ._sync_c74d5f875239.db excluded  (1)
[9] csync_ftw   <= Closing walk for /tmp/oc-test with read_from_db 0
[8] csync_update  Update detection for local replica took 0,00 seconds walking 0 files.
[7] csync_memstat_check  Memory: 572020K total size, 25144K resident, 20920K shared
[6] csync_statedb_close  sqlite3_close=0

/tmp/oc-test
/tmp/oc-test/._sync_c74d5f875239.db

test for domain 'anotherdomain.com' with client 'nextcloudcmd': failure

Server configuration

Operating system: Ubuntu 17.10

Web server: apache2 2.4.18

Database: postgresql 9.5

PHP version: php 7.0

Nextcloud version: 13.0.0.14

Updated from an older Nextcloud/ownCloud or fresh install:
Updated from 11 via 12

Where did you install Nextcloud from: https://nextcloud.com/install/ manually installed

Signing status:

Signing status 
No errors have been found.

List of activated apps:

App list 
Enabled:
  - activity: 2.6.1
  - bruteforcesettings: 1.0.3
  - comments: 1.3.0
  - dav: 1.4.6
  - federatedfilesharing: 1.3.1
  - federation: 1.3.0
  - files: 1.8.0
  - files_pdfviewer: 1.2.0
  - files_sharing: 1.5.0
  - files_texteditor: 2.5.1
  - files_trashbin: 1.3.0
  - files_versions: 1.6.0
  - files_videoplayer: 1.2.0
  - firstrunwizard: 2.2.1
  - gallery: 18.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.1.0
  - nextcloud_announcements: 1.2.0
  - notifications: 2.1.2
  - oauth2: 1.1.0
  - password_policy: 1.3.0
  - provisioning_api: 1.3.0
  - serverinfo: 1.3.0
  - sharebymail: 1.3.0
  - survey_client: 1.1.0
  - systemtags: 1.3.0
  - theming: 1.4.1
  - twofactor_backupcodes: 1.2.3
  - updatenotification: 1.3.0
  - workflowengine: 1.3.0
Disabled:
  - admin_audit
  - encryption
  - files_external
  - user_external
  - user_ldap

Nextcloud configuration:

Config report 
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "files.mydomain.com",
            "anotherdomain.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "http:\/\/anotherdomain.com",
        "dbtype": "pgsql",
        "version": "13.0.0.14",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "updater.secret": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "loglevel": 2,
        "maintenance": false
    }
}

Are you using external storage, if yes which one: local

Are you using encryption: no

Are you using an external user-backend, if yes which one: no
@BastiTee
Copy link
Author

BastiTee commented Feb 15, 2018 •
edited
Loading

Stuff I tried after initially filing issue without any change to the problem described:

  • Activated files_external app.
  • Tried with different folders from nextcloud.
  • Tried with root webdav address https://anotherdomain.com/remote.php/webdav/.
  • Rescanned files via sudo -u www-data php occ files:scan --all.

@BastiTee
Copy link
Author

BastiTee commented Feb 15, 2018 •
edited
Loading

I found the reason.
Apparently the username-part used in the WebDAV path of the request is now case-sensitive. On the file system my account's subfolder was /files/Username/ even though I login with 'username' all-lowercase all the time. The casing was not relevant for the client-connection in NX11.

@nextcloud-bot nextcloud-bot mentioned this issue Aug 13, 2018
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@BastiTee