Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SMB External storage webdav path adds extra string "Dc" and causes share to be not accessible #29003

Closed
uniuuu opened this issue Sep 30, 2021 · 12 comments · Fixed by #32632
Closed
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap bug needs info

Comments

@uniuuu
Copy link

uniuuu commented Sep 30, 2021

How to use GitHub

  • Please use the 👍 reaction to show that you are affected by the same issue.
  • Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
  • Subscribe to receive notifications on status change and new comments.

Steps to reproduce

  1. To have installation with external storage using SMB share.
  2. To upgrade from 21.0.4 to 22.1.1
  3. Go to external storage mounted folder and see nested folder names changed adding "Dc" at the end

Expected behaviour

External storage will nested folders names not changed

Actual behaviour

External storage nested folders names have changed adding "Dc" at the end

Server configuration

Operating system:
Debian (in docker)

Web server:
Apache

Database:
MySQL

PHP version:
8.0.11

Nextcloud version: (see Nextcloud admin page)
22.1.1

Updated from an older Nextcloud/ownCloud or fresh install:
21.0.4

Where did you install Nextcloud from:
docker

Signing status:

Signing status
No errors have been found.

List of activated apps:

App list
Enabled:
  - activity: 2.15.0
  - admin_audit: 1.11.0
  - cloud_federation_api: 1.4.0
  - dav: 1.18.0
  - federatedfilesharing: 1.11.0
  - files: 1.16.0
  - files_external: 1.12.1
  - files_sharing: 1.13.2
  - files_trashbin: 1.11.0
  - impersonate: 1.9.0
  - logreader: 2.7.0
  - lookup_server_connector: 1.9.0
  - oauth2: 1.9.0
  - password_policy: 1.12.0
  - provisioning_api: 1.11.0
  - settings: 1.3.0
  - twofactor_admin: 3.1.0
  - twofactor_backupcodes: 1.10.1
  - twofactor_totp: 6.1.0
  - user_ldap: 1.11.0
  - viewer: 1.6.0
  - workflowengine: 2.3.1


Disabled:
  - accessibility
  - analytics
  - announcementcenter
  - appointments
  - auto_groups
  - bookmarks
  - calendar
  - carnet
  - checksum
  - circles
  - cms_pico
  - comments
  - contacts
  - contactsinteraction
  - cospend
  - dashboard
  - data_request
  - deck
  - documentserver_community
  - drawio
  - duplicatefinder
  - encryption
  - event_update_notification
  - external
  - extract
  - facerecognition
  - federation
  - files_accesscontrol
  - files_antivirus
  - files_automatedtagging
  - files_fulltextsearch
  - files_mindmap
  - files_pdfviewer
  - files_photospheres
  - files_rightclick
  - files_trackdownloads
  - files_versions
  - files_videoplayer
  - firstrunwizard
  - forms
  - fulltextsearch
  - groupfolders
  - health
  - integration_suitecrm
  - integration_twitter
  - keeweb
  - ldap_write_support
  - login_notes
  - mail
  - maps
  - metadata
  - nextcloud_announcements
  - notes
  - notifications
  - ocdownloader
  - onlyoffice
  - passman
  - passwords
  - phonetrack
  - photos
  - polls
  - privacy
  - quicknotes
  - rainloop
  - ransomware_protection
  - recommendations
  - registration
  - richdocuments
  - richdocumentscode
  - riotchat
  - sendent
  - serverinfo
  - sharebymail
  - sharelisting
  - sharingpath
  - smb_test
  - spreed
  - support
  - survey_client
  - suspicious_login
  - systemtags
  - tasks
  - text
  - theming
  - theming_customcss
  - timemanager
  - timetracker
  - unsplash
  - updatenotification
  - user_status
  - video_converter
  - weather_status
  - workflow_ocr
  - workflow_pdf_converter
  - workflow_script

Nextcloud configuration:

Config report
{
    "system": {
        "htaccess.RewriteBase": "\/",
        "skeletondirectory": "",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***",
            "172.25.200.128",
            "10.218.99.7"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "default_phone_region": "SG",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "22.1.1.2",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "logtimezone": "Asia\/Singapore",
        "log_type": "owncloud",
        "logfile": "\/var\/log\/nextcloud.log",
        "loglevel": 2,
        "log_rotate_size": "104857600",
        "twofactor_enforced": "true",
        "twofactor_enforced_groups": [],
        "twofactor_enforced_excluded_groups": [
            "Test"
        ],
        "maintenance": false,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpauth": 1,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "app.mail.verify-tls-peer": false,
        "mail_smtpdebug": false,
        "defaultapp": "files",
        "app_install_overwrite": [
            "spreedme",
            "twofactor_admin"
        ],
        "theme": "",
        "has_rebuilt_cache": true,
        "debug": false,
        "overwriteprotocol": "https",
        "mail_smtpsecure": "ssl",
        "session_lifetime": 2400,
        "remember_login_cookie_lifetime": 3600,
        "updater.secret": "***REMOVED SENSITIVE VALUE***"
    }
}

Are you using external storage, if yes which one: SMB

Are you using encryption: no

Are you using an external user-backend, if yes which one: LDAP

LDAP configuration (delete this part if not used)

LDAP config
+-------------------------------+---------------------------------------------------------------------------------------------------+
| Configuration                 | s01                                                                                               |
+-------------------------------+---------------------------------------------------------------------------------------------------+
| hasMemberOfFilterSupport      |                                                                                                   |
| homeFolderNamingRule          |                                                                                                   |
| lastJpegPhotoLookup           | 0                                                                                                 |
| ldapAgentName                 | cn=admin,dc=***REMOVED SENSITIVE VALUE***                                                          |
| ldapAgentPassword             | ***                                                                                               |
| ldapAttributesForGroupSearch  |                                                                                                   |
| ldapAttributesForUserSearch   |                                                                                                   |
| ldapBackupHost                |                                                                                                   |
| ldapBackupPort                |                                                                                                   |
| ldapBase                      | dc=fscloud,dc=***REMOVED SENSITIVE VALUE***                                                          |
| ldapBaseGroups                | dc=fscloud,dc=***REMOVED SENSITIVE VALUE***                                                                  |
| ldapBaseUsers                 | dc=fscloud,dc=***REMOVED SENSITIVE VALUE***                                                                  |
| ldapCacheTTL                  | 600                                                                                               |
| ldapConfigurationActive       | 1                                                                                                 |
| ldapDefaultPPolicyDN          |                                                                                                   |
| ldapDynamicGroupMemberURL     |                                                                                                   |
| ldapEmailAttribute            | mail                                                                                              |
| ldapExperiencedAdmin          | 0                                                                                                 |
| ldapExpertUUIDGroupAttr       |                                                                                                   |
| ldapExpertUUIDUserAttr        |                                                                                                   |
| ldapExpertUsernameAttr        | uid                                                                                               |
| ldapExtStorageHomeAttribute   |                                                                                                   |
| ldapGidNumber                 | gidNumber                                                                                         |
| ldapGroupDisplayName          | cn                                                                                                |
| ldapGroupFilter               | (&(|(objectclass=posixGroup)))                                                                    |
| ldapGroupFilterGroups         |                                                                                                   |
| ldapGroupFilterMode           | 0                                                                                                 |
| ldapGroupFilterObjectclass    | posixGroup                                                                                        |
| ldapGroupMemberAssocAttr      | memberUid                                                                                         |
| ldapHost                      | openldap                                                                                          |
| ldapIgnoreNamingRules         |                                                                                                   |
| ldapLoginFilter               | (&(|(objectclass=posixAccount))(|(uid=%uid)(|(mailPrimaryAddress=%uid)(mail=%uid))(|(uid=%uid)))) |
| ldapLoginFilterAttributes     | uid                                                                                               |
| ldapLoginFilterEmail          | 1                                                                                                 |
| ldapLoginFilterMode           | 0                                                                                                 |
| ldapLoginFilterUsername       | 1                                                                                                 |
| ldapMatchingRuleInChainState  | unknown                                                                                           |
| ldapNestedGroups              | 0                                                                                                 |
| ldapOverrideMainServer        |                                                                                                   |
| ldapPagingSize                | 500                                                                                               |
| ldapPort                      | 389                                                                                               |
| ldapQuotaAttribute            |                                                                                                   |
| ldapQuotaDefault              |                                                                                                   |
| ldapTLS                       | 0                                                                                                 |
| ldapUserAvatarRule            | default                                                                                           |
| ldapUserDisplayName           | cn                                                                                                |
| ldapUserDisplayName2          | uid                                                                                               |
| ldapUserFilter                | (|(objectclass=posixAccount))                                                                     |
| ldapUserFilterGroups          |                                                                                                   |
| ldapUserFilterMode            | 0                                                                                                 |
| ldapUserFilterObjectclass     | posixAccount                                                                                      |
| ldapUuidGroupAttribute        | auto                                                                                              |
| ldapUuidUserAttribute         | auto                                                                                              |
| turnOffCertCheck              | 0                                                                                                 |
| turnOnPasswordChange          | 0                                                                                                 |
| useMemberOfToDetectMembership | 1                                                                                                 |
+-------------------------------+---------------------------------------------------------------------------------------------------+

Client configuration

Browser:
Firefox

Operating system:
Windows 10

Logs

Web server error log

Web server error log
Insert your webserver log here

Nextcloud log (data/nextcloud.log)

Nextcloud log
{"reqId":"ah7FqsKlV6eTZeRHzlru","level":4,"time":"2021-09-30T10:07:04+08:00","remoteAddr":"58.182.68.120","user":"***REMOVED SENSITIVE VALUE*** ","app":"webdav","method":"GET","url":"/remote.php/webdav/TFS%20Data/2021%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20Dc","message":"Could not open file","userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 Firefox/92.0","version":"22.1.1.2","exception":{"Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Message":"Could not open file","Code":0,"Trace":[{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php","line":85,"function":"get","class":"OCA\\DAV\\Connector\\Sabre\\File","type":"->","args":[]},{"file":"/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php","line":89,"function":"httpGet","class":"Sabre\\DAV\\CorePlugin","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":472,"function":"emit","class":"Sabre\\DAV\\Server","type":"->","args":["method:GET",[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":253,"function":"invokeMethod","class":"Sabre\\DAV\\Server","type":"->","args":[{"__class__":"Sabre\\HTTP\\Request"},{"__class__":"Sabre\\HTTP\\Response"}]},{"file":"/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php","line":321,"function":"start","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/apps/dav/appinfo/v1/webdav.php","line":83,"function":"exec","class":"Sabre\\DAV\\Server","type":"->","args":[]},{"file":"/var/www/html/remote.php","line":166,"args":["/var/www/html/apps/dav/appinfo/v1/webdav.php"],"function":"require_once"}],"File":"/var/www/html/apps/dav/lib/Connector/Sabre/File.php","Line":438,"CustomMessage":"--"},"id":"61551bd263ecf"}

Browser log

Browser log
Insert your browser log here, this could for example include:
https://***REMOVED SENSITIVE VALUE***/remote.php/webdav/TFS%20Data/2021%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20Dc

HTTP/2 503 Service Unavailable
cache-control: no-store, no-cache, must-revalidate
content-security-policy: default-src 'none';base-uri 'none';manifest-src 'self';script-src 'self';style-src 'self' 'unsafe-inline';img-src 'self' data: blob:;font-src 'self' data:;connect-src 'self';media-src 'self';frame-ancestors 'self';form-action 'self'
content-type: text/html; charset=UTF-8
date: Thu, 30 Sep 2021 02:10:14 GMT
expires: Thu, 19 Nov 1981 08:52:00 GMT
pragma: no-cache
referrer-policy: no-referrer
server: Apache/2.4.48 (Debian)
strict-transport-security: max-age=31536000
x-content-type-options: nosniff
x-download-options: noopen
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-powered-by: PHP/8.0.11
x-robots-tag: none
x-xss-protection: 1; mode=block
X-Firefox-Spdy: h2

@uniuuu uniuuu added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Sep 30, 2021
@uniuuu
Copy link
Author

uniuuu commented Sep 30, 2021

See the screencast attached

external_share_issue.mp4

@uniuuu
Copy link
Author

uniuuu commented Sep 30, 2021

The main point that the path changed.
It has to be https://sample.com/remote.php/webdav/TFS%20Data/2021
But it comes with spaces and Dc at the end.
https://sample.com/remote.php/webdav/TFS%20Data/2021%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20Dc

In mobile application the same impact:
WhatsApp Image 2021-09-29 at 1 56 07 PM

No issue found on source file server.
file-server

@solracsf

This comment has been minimized.

@solracsf solracsf changed the title After upgrade to 22.1.1 the SMB External storage webdav path adds extra srting "Dc" and causes share to be not accessible SMB External storage webdav path adds extra string "Dc" and causes share to be not accessible Oct 3, 2021
@NA-NOOK
Copy link

NA-NOOK commented Oct 5, 2021

I can confirm this bug on NC 20.0.12, the problem is caused by an incorrect interpretation of file attributes, see this thread I started recently. In short: this "extra string" is filename attribute merged with the file name. In this context "Dc" suffix means that directory and compressed attribute is set on smb share.

@ghost

This comment has been minimized.

@ghost ghost added the stale Ticket or PR with no recent activity label Nov 4, 2021
@SteffenDE
Copy link

SteffenDE commented Nov 7, 2021

I can confirm the same issue using 22.2.0 with directories showing as files ending with "Dr":
image

smbclient listing:
tg_image_753850213

@ghost ghost removed the stale Ticket or PR with no recent activity label Nov 7, 2021
@SteffenDE
Copy link

It works when using the native smbclient (https://github.com/eduardok/libsmbclient-php). After purging the external storages from the database, the directories are fine again for me.
This is my new Dockerfile:

FROM nextcloud:22-fpm

RUN apt-get update && apt-get install -y procps libsmbclient-dev && rm -rf /var/lib/apt/lists/*
RUN pecl install smbclient && docker-php-ext-enable smbclient

It can also be fixed by adding lowercase modes to the regex in apps/files_external/3rdparty/icewind/smb/src/Wrapped/Parser.php and adding a strtoupper:

        /**
	 * @param string[] $output
	 * @param string $basePath
	 * @param callable(string):ACL[] $aclCallback
	 * @return FileInfo[]
	 */
	public function parseDir(array $output, string $basePath, callable $aclCallback): array {
		//last line is used space
		array_pop($output);
		$regex = '/^\s*(.*?)\s\s\s\s+(?:([NDHARSndhars]*)\s+)?([0-9]+)\s+(.*)$/';
		//2 spaces, filename, optional type, size, date
		$content = [];
		foreach ($output as $line) {
			if (preg_match($regex, $line, $matches)) {
				list(, $name, $mode, $size, $time) = $matches;
				if ($name !== '.' and $name !== '..') {
					$mode = $this->parseMode(strtoupper($mode));
					$time = strtotime($time . ' ' . $this->timeZone);
					$path = $basePath . '/' . $name;
					$content[] = new FileInfo($path, $name, (int)$size, $time, $mode, function () use ($aclCallback, $path): array {
						return $aclCallback($path);
					});
				}
			}
		}
		return $content;
	}

@ghost

This comment has been minimized.

@ghost ghost added the stale Ticket or PR with no recent activity label Dec 7, 2021
@uniuuu
Copy link
Author

uniuuu commented Dec 20, 2021

Do you confirm the same issue with 22.2.0?

Hi @acsfer

On the Nextcloud Hub II (23.0.0) the issue is still persisting.
image

@ghost ghost removed the stale Ticket or PR with no recent activity label Dec 20, 2021
@solracsf
Copy link
Member

@uniuuu can you try the patch by #29003 (comment)
(modify the file apps/files_external/3rdparty/icewind/smb/src/Wrapped/Parser.php)

@uniuuu
Copy link
Author

uniuuu commented Dec 20, 2021

@uniuuu can you try the patch by #29003 (comment) (modify the file apps/files_external/3rdparty/icewind/smb/src/Wrapped/Parser.php)

Thank you.
Tried change Dockerfile instead and it worked out by removing smbclient deb package and installing as per below.
RUN pecl install smbclient && docker-php-ext-enable smbclient
I confirm the folder's content started to show.

@solracsf
Copy link
Member

Closing and moving issue + possible fix to icewind1991/SMB#110

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap bug needs info
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants