Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WHM/cPanel WebDAV fails in NextCloud (but not in OwnCloud) #11810

Closed
ridexbuilder opened this issue Oct 13, 2018 · 14 comments
Closed

WHM/cPanel WebDAV fails in NextCloud (but not in OwnCloud) #11810

ridexbuilder opened this issue Oct 13, 2018 · 14 comments
Labels
bug

Comments

@ridexbuilder
Copy link

Steps to reproduce

Create a WebDAV backup configuration in WHM.
Try to validate the entered credentials

Expected behaviour

Success: The destination “owncloud” validated.

Actual behaviour

Error: Validation for transport “nextcloud” failed: Could not list files in destination: Server response: Multistatus

A Ticket was raised with cPanel and Support responded with this:
"Yes, NextCloud seems give different dav responses than OwnCloud does, and the underlying perl module that we use to connect to webdav servers (HTTP::DAV) doesn't deal well with responses that NextCloud uses, because it considers any error response within a multiresponse to propfind to be an overall failure.

The case I mentioned earlier, CPANEL-23378, should address this, but I do not have a workaround for it or now, or an ETA on when a fix for this will be published. I should also note that it is possible it may be published in our changelogs under CPANEL-17033 as well."

Server configuration detail

Operating system: Linux 4.15.18-5-pve #1 SMP PVE 4.15.18-24 (Thu, 13 Sep 2018 09:15:10 +0200) x86_64

Webserver: Apache (apache2handler)

Database: mysql 10.1.26

PHP version:

7.0.30-0+deb9u1
Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, Reflection, SPL, session, standard, apache2handler, mysqlnd, PDO, xml, apcu, calendar, ctype, curl, dom, mbstring, fileinfo, ftp, gd, gettext, gmp, iconv, igbinary, imap, intl, json, ldap, exif, mcrypt, mysqli, pdo_mysql, Phar, posix, readline, redis, shmop, SimpleXML, sockets, sysvmsg, sysvsem, sysvshm, tokenizer, wddx, xmlreader, xmlwriter, xsl, zip, Zend OPcache

Nextcloud version: 13.0.4 - 13.0.4.0

Updated from an older Nextcloud/ownCloud or fresh install: Fresh

Where did you install Nextcloud from: Turnkey NextCloud - manual install has same issue

Signing status

Array
(
)

List of activated apps 
Enabled:
 - activity: 2.6.1
 - admin_audit: 1.3.0
 - comments: 1.3.0
 - dav: 1.4.7
 - encryption: 2.0.0
 - federatedfilesharing: 1.3.1
 - federation: 1.3.0
 - files: 1.8.0
 - files_automatedtagging: 1.3.0
 - files_retention: 1.2.0
 - files_rightclick: 0.8.4
 - files_sharing: 1.5.0
 - files_texteditor: 2.5.1
 - files_trashbin: 1.3.0
 - files_versions: 1.6.0
 - files_videoplayer: 1.2.0
 - firstrunwizard: 2.2.1
 - gallery: 18.0.0
 - issuetemplate: 0.4.0
 - logreader: 2.0.0
 - lookup_server_connector: 1.1.0
 - nextcloud_announcements: 1.2.0
 - notifications: 2.1.2
 - oauth2: 1.1.1
 - password_policy: 1.3.0
 - provisioning_api: 1.3.0
 - quota_warning: 1.2.0
 - serverinfo: 1.3.0
 - sharebymail: 1.3.0
 - survey_client: 1.1.0
 - systemtags: 1.3.0
 - theming: 1.4.5
 - twofactor_backupcodes: 1.2.3
 - updatenotification: 1.3.0
 - workflowengine: 1.3.0
Disabled:
 - files_external
 - files_pdfviewer
 - user_external
 - user_ldap
Prior to adding any apps/encryption, issue was the same.
Configuration (config/config.php) 
{
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "localhost",
        "nc.xxxxservers.com"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "overwrite.cli.url": "http:\/\/localhost",
    "dbtype": "mysql",
    "version": "13.0.4.0",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbport": "",
    "dbtableprefix": "oc_",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "filelocking.enabled": true,
    "memcache.locking": "\\OC\\Memcache\\Redis",
    "memcache.local": "\\OC\\Memcache\\APCu",
    "memcache.distributed": "\\OC\\Memcache\\Redis",
    "redis": {
        "host": "***REMOVED SENSITIVE VALUE***",
        "port": 6379,
        "timeout": 0,
        "password": "***REMOVED SENSITIVE VALUE***"
    },
    "memcached_servers": [
        [
            "localhost",
            11211
        ]
    ]
}

Are you using external storage, if yes which one: no

Are you using encryption: true

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: WHM/cPanel API 74.0.9

Operating system: Centos 7.5 WHM 74.09

Logs

Browser log 
From cPanel Support:

Here is an example of what your webdav server returns during the backup validation process:

=====================
<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Thu, 11 Oct 2018 01:19:23 GMT</d:getlastmodified>
<d:resourcetype>
<d:collection/>
</d:resourcetype>
<d:quota-used-bytes>669</d:quota-used-bytes>
<d:quota-available-bytes>36084080402</d:quota-available-bytes>
<d:getetag>"5bbea51b74add"</d:getetag>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-15659-1539220626.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Thu, 11 Oct 2018 01:17:33 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"aab732f13c5d78c329575b47c2bedf26"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-15778-1539220736.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Thu, 11 Oct 2018 01:19:23 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"108b450fbfdd031ad4c4be262a71f698"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-1609-1539172298.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 11:52:02 GMT</d:getlastmodified>
<d:getcontentlength>83</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"26aa3010702fcc3dfff4340dd1f598bf"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-1630-1539172392.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 11:53:38 GMT</d:getlastmodified>
<d:getcontentlength>83</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"c4d625d045fbd6aff33419bdc64ade9a"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-16708-1539180163.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 14:03:10 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"38c35bf00d6a5c869b7d437691e1738c"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-17185-1539180520.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 14:09:07 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"c5ede3f4527f85415099637244af027a"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-22346-1539200325.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 19:39:06 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"c388b5c811be69e826bc9c5e72467202"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-4591-1539173966.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 12:19:52 GMT</d:getlastmodified>
<d:getcontentlength>83</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"aa637182997496401a7e4fddae58d799"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
</d:multistatus>
========================

The various 404 responses within the multi response aren't handled properly by HTTP::DAV, which is what causes the validation to fail.
Nextcloud log
Browser log

From cPanel Support:

Here is an example of what your webdav server returns during the backup validation process:

=====================

<d:multistatus xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Thu, 11 Oct 2018 01:19:23 GMT</d:getlastmodified>
<d:resourcetype>
<d:collection/>
</d:resourcetype>
<d:quota-used-bytes>669</d:quota-used-bytes>
<d:quota-available-bytes>36084080402</d:quota-available-bytes>
<d:getetag>"5bbea51b74add"</d:getetag>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-15659-1539220626.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Thu, 11 Oct 2018 01:17:33 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"aab732f13c5d78c329575b47c2bedf26"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-15778-1539220736.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Thu, 11 Oct 2018 01:19:23 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"108b450fbfdd031ad4c4be262a71f698"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-1609-1539172298.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 11:52:02 GMT</d:getlastmodified>
<d:getcontentlength>83</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"26aa3010702fcc3dfff4340dd1f598bf"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-1630-1539172392.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 11:53:38 GMT</d:getlastmodified>
<d:getcontentlength>83</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"c4d625d045fbd6aff33419bdc64ade9a"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-16708-1539180163.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 14:03:10 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"38c35bf00d6a5c869b7d437691e1738c"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-17185-1539180520.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 14:09:07 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"c5ede3f4527f85415099637244af027a"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-22346-1539200325.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 19:39:06 GMT</d:getlastmodified>
<d:getcontentlength>84</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"c388b5c811be69e826bc9c5e72467202"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/nextcloud/remote.php/webdav/whm/validate.tmp-4591-1539173966.txt</d:href>
<d:propstat>
<d:prop>
<d:getlastmodified>Wed, 10 Oct 2018 12:19:52 GMT</d:getlastmodified>
<d:getcontentlength>83</d:getcontentlength>
<d:resourcetype/>
<d:getetag>"aa637182997496401a7e4fddae58d799"</d:getetag>
<d:getcontenttype>text/plain</d:getcontenttype>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
<d:propstat>
<d:prop>
<d:quota-used-bytes/>
<d:quota-available-bytes/>
</d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status>
</d:propstat>
</d:response>
</d:multistatus>

The various 404 responses within the multi response aren't handled properly by HTTP::DAV, which is what causes the validation to fail.
@ridexbuilder
Copy link
Author

Further info from cPanel Support:

As an additional heads up for another issue you would run across if you manage to prevent/fix the quota 404s, NextCloud seems to return 404 responses when trying to describe the size of a folder, which also causes HTTP::DAV to consider any PROPFIND request to be a failure if you are listing the contents of a folder that contains other folders inside of it.

Since our backups do use separate folders for each day, on NextCloud destination servers this prevents pruning (cPanel deleting older backups) from functioning once you get the transport working -- but this would be addressed by the same case that is causing the problems with validation.

@nextcloud-bot
Copy link
Member

GitMate.io thinks possibly related issues are #3738 (Webdav), #7062 (Nextcloud webDAV not supporting SSO?), #104 (NextCloud / OwnCloud), and #5947 (webDAV PROPFIND).

@parvez
Copy link

parvez commented Jan 6, 2019

I had filed Support Issue for this as well earlier, sadly with no remedy. Any progress?

@ridexbuilder
Copy link
Author

WHM/cPanel have released an update for v78 that tries to address the issue.
It doesn't explain why the underlying webdav code differs from that of OwnCloud, which doesn't have this problem.

@kesselb
Copy link
Contributor

kesselb commented Jan 16, 2019

Does it work for you? If so we could close this issue.

@ridexbuilder
Copy link
Author

ridexbuilder commented Jan 16, 2019 •
edited
Loading

I only just got notified an hour ago. I don't use Release Candidate v78 of WHM (I use v76 GA), so unable to test at the present time on live servers.
I'm currently updating a secondary server to v78, to test. I have also requested the cPanel patch to be backported to v76.

@parvez
Copy link

parvez commented Jan 24, 2019

Awesome! I just verified on v78.0.3 and it worked like a charm. You can close this issue now.

@ridexbuilder
Copy link
Author

Well, you can thank me for raising and pursuing this, with cPanel. You can thank cPanel more, for actively diagnosing the the issue, specific to NextCloud and providing the patch.
Zero kudos to NextCloud team though, as the difference in handling webdav is still there, for whatever reason.

@parvez
Copy link

parvez commented Jan 24, 2019

Thanks a million @ridexbuilder and cPanel. Without the backup, felt like living without insurance, hoping nothing crashed upon me! 😅

@ridexbuilder
Copy link
Author

@parvez Please mention the version of NextCloud that you've got working.
(The test server that I was using is unavailable due to that hosting provider going out of business, so it may be some time before I can purpose another, for testing.)

@parvez
Copy link

parvez commented Jan 24, 2019

My NextCloud version is 14.0.4

@ridexbuilder
Copy link
Author

ridexbuilder commented Jan 24, 2019 •
edited
Loading

Thanks, at least I know I can upgrade from v13 without that bit getting screwed up ;-)
In my eyes, the issue remains unaddressed in NextCloud, so someone else can close, if they feel the need.
Also, cPanel have stated they don't have the resources to backport their 'fix'/kludge to the current recommended v76 tier.

@parvez
Copy link

parvez commented Jan 24, 2019

I just upgraded NextCloud to v15.0.2 and it works too! Thanks again @CpanelInc Team and @ridexbuilder!

@kesselb
Copy link
Contributor

kesselb commented Jan 24, 2019

Thank you for opening the issue and reporting back 👍

As reference the patch for HTTP::DAV: cosimo/perl5-http-dav@ee405c9

A multistatus as response to a propfind request looks right to me. Don't know why owncloud does something different. https://www.greenbytes.de/tech/webdav/rfc2518.html#n-example---retrieving-named-properties (a multistatus response with another statuscode for propstats looks fine according to the rfc).

@kesselb kesselb closed this as completed Jan 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@parvez @kesselb @ridexbuilder @nextcloud-bot