Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Nextcloud Hub 8, copying files to Object Storage configured as Primary Storage isn't reliable #45082

Closed
5 of 8 tasks
sl-lebrit opened this issue Apr 27, 2024 · 4 comments · Fixed by #43794
Closed
5 of 8 tasks

[Bug]: Nextcloud Hub 8, copying files to Object Storage configured as Primary Storage isn't reliable #45082

sl-lebrit opened this issue Apr 27, 2024 · 4 comments · Fixed by #43794
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 29-feedback bug feature: object storage

Comments

@sl-lebrit
Copy link

⚠️ This issue respects the following points: ⚠️

Bug description

When you copy a hierarchy of folders via browser –or even a single folder with nested subfolders inside– all folders/files are successfully copied and usable too but, very often but not always, folder sizes are shown incorrectly (i.e. 0 bytes) moreover the copy is unexpectedly halted too. This makes copying to external storage as primary storage impossible. I didn't test copying to other external storages.
One and only one error is costantly reported:
<Array to string conversion at /var/www/html/lib/private/Files/Cache/Scanner.php#224>

Steps to reproduce

  1. Set an External Storage (Amazon s3), with multipart enabled, as Primary Storage
  2. From your PC/Mac⁽*⁾ copy a folder containing a hierarchy of subfolders and files
  3. This is the result in the 90% of cases:
Screenshot 2024-04-27 at 21 48 21 but, actually those empty folders contain other folders and files: Screenshot 2024-04-27 at 21 48 47 4. and..., often, the copy is stopped

⁽*⁾ a Mac in my case

Expected behavior

I would like the batch of files and folders to be entirely copied and the metadata to be evaluated correctly.

Installation method

Official All-in-One appliance

Nextcloud Server version

29

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

{
    "system": {
        "one-click-instance": true,
        "one-click-instance.user-limit": 100,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "appsallowlist": false,
        "check_data_directory_permissions": false,
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "overwritehost": "*****.*****.com",
        "overwriteprotocol": "https",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
            "localhost",
            "****.****.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "29.0.0.19",
        "overwrite.cli.url": "https:\/\/****.****.com\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "updatedirectory": "\/nc-updater",
        "loglevel": 3,
        "app_install_overwrite": [
            "nextcloud-aio"
        ],
        "log_type": "file",
        "logfile": "\/var\/www\/html\/data\/nextcloud.log",
        "log_rotate_size": "10485760",
        "log.condition": {
            "apps": [
                "admin_audit"
    ]
        },
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "enabledPreviewProviders": {
            "1": "OC\\Preview\\Image",
            "2": "OC\\Preview\\MarkDown",
            "3": "OC\\Preview\\MP3",
            "4": "OC\\Preview\\TXT",
            "5": "OC\\Preview\\OpenDocument",
            "6": "OC\\Preview\\Movie",
            "7": "OC\\Preview\\Krita",
            "0": "OC\\Preview\\Imaginary"
        },
        "enable_previews": true,
        "upgrade.disable-web": true,
        "mail_smtpmode": "smtp",
        "trashbin_retention_obligation": "auto, 30",
        "versions_retention_obligation": "auto, 30",
        "activity_expire_days": "30",
        "simpleSignUpLink.shown": false,
        "share_folder": "\/Shared",
        "one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",
        "upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",
        "maintenance_window_start": 100,
        "allow_local_remote_servers": true,
        "davstorage.request_timeout": 3600,
"htaccess.RewriteBase": "\/",
        "dbpersistent": false,
        "auth.bruteforce.protection.enabled": true,
        "ratelimit.protection.enabled": true,
        "files_external_allow_create_new_local": true,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "preview_imaginary_key": "e98427df8dc02767b14ec376d116129d156a79af2311ba2c",
        "objectstore": {
            "class": "\\OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "***REMOVED SENSITIVE VALUE***",
                "key": "***REMOVED SENSITIVE VALUE***",
                "secret": "***REMOVED SENSITIVE VALUE***",
                "use_ssl": true,
                "hostname": "***REMOVED SENSITIVE VALUE***"
            }
        },
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpsecure": "ssl",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "defaultapp": "dashboard,files,mail,calendar,contacts,spreed",
        "auth.bruteforce.protection.enabled": true,
        "ratelimit.protection.enabled": true,
        "files_external_allow_create_new_local": true,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "preview_imaginary_key": "e98427df8dc02767b14ec376d116129d156a79af2311ba2c",
        "objectstore": {
            "class": "\\OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "***REMOVED SENSITIVE VALUE***",
                "key": "***REMOVED SENSITIVE VALUE***",
                "secret": "***REMOVED SENSITIVE VALUE***",
                "use_ssl": true,
                "hostname": "***REMOVED SENSITIVE VALUE***"
            }
        },
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "mail_smtpsecure": "ssl",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "defaultapp": "dashboard,files,mail,calendar,contacts,spreed"
    }
}

List of activated Apps

Enabled:
  - activity: 2.21.1
  - admin_audit: 1.19.0
  - app_api: 2.5.0
  - approval: 1.2.0
  - calendar: 4.7.1
  - circles: 29.0.0-dev
  - cloud_federation_api: 1.12.0
  - comments: 1.19.0
  - contacts: 6.0.0
  - contactsinteraction: 1.10.0
  - dashboard: 7.9.0
  - dav: 1.30.1
  - deck: 1.13.0
  - federatedfilesharing: 1.19.0
  - federation: 1.19.0
  - files: 2.1.0
  - files_accesscontrol: 1.19.0
  - files_automatedtagging: 1.19.0
  - files_confidential: 2.1.0
  - files_downloadlimit: 2.0.0
  - files_external: 1.21.0
  - files_pdfviewer: 2.10.0
  - files_reminders: 1.2.0
  - files_retention: 1.18.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - firstrunwizard: 2.18.0
  - forms: 4.2.3
  - logreader: 2.14.0
  - lookup_server_connector: 1.17.0
  - mail: 3.6.0
  - nextcloud-aio: 0.5.0
  - nextcloud_announcements: 1.18.0
  - notes: 4.10.0
  - notifications: 2.17.0
  - notify_push: 0.6.10
  - oauth2: 1.17.0
  - password_policy: 1.19.0
  - photos: 2.5.0
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - related_resources: 1.4.0
  - richdocuments: 8.4.0
  - richdocumentscode: 24.4.103
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - spreed: 19.0.0
  - stt_helper: 1.1.1
  - support: 1.12.0
  - survey_client: 1.17.0
  - suspicious_login: 7.0.0
  - systemtags: 1.19.0
  - tables: 0.7.0
  - text: 3.10.0
  - theming: 2.4.0
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - related_resources: 1.4.0
  - richdocuments: 8.4.0
  - richdocumentscode: 24.4.103
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - spreed: 19.0.0
  - stt_helper: 1.1.1
  - support: 1.12.0
  - survey_client: 1.17.0
  - suspicious_login: 7.0.0
  - systemtags: 1.19.0
  - tables: 0.7.0
  - text: 3.10.0
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - user_status: 1.9.0
  - viewer: 2.3.0
  - weather_status: 1.9.0
  - workflowengine: 2.11.0
Disabled:
  - bruteforcesettings: 2.9.0
  - encryption: 2.17.0
  - recommendations: 2.1.0 (installed 2.1.0)
  - tasks: 0.15.0
  - twofactor_totp: 11.0.0-dev (installed 11.0.0-dev)
  - user_ldap: 1.20.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{"reqId":"vfjSUwVApKFg9XMY67Cl","level":3,"time":"2024-04-27T18:32:49+00:00","remoteAddr":"***.***.***.***","user":"Stefano","app":"PHP","method":"PROPFIND","url":"/remote.php/dav/files/Stefano/OLD-Storico/Progetti%20CLOSED/300_LB_myDanishContainers/","message":"Array to string conversion at /var/www/html/lib/private/Files/Cache/Scanner.php#224","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.5112.81 Safari/537.36","version":"29.0.0.19","data":{"app":"PHP"}}
. . .

Additional info

Same behaviours with Safari 17.4 and [unsupported] Brave 1.42.95
@sl-lebrit sl-lebrit added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Apr 27, 2024
@solracsf
Copy link
Member

solracsf commented Apr 29, 2024 •
edited
Loading

Related:
#43642
#43408

@compuguy
Copy link

Pretty sure this would be fixed by #43794. At least in my limited testing using the merge request as a patch: https://patch-diff.githubusercontent.com/raw/nextcloud/server/pull/43794.diff

@DaanSelen
Copy link

I have the same error spam... but without

alias nocc-cldtst01="sudo -u www-data php /var/www/cldtst01.site1.com/html/occ"

@joshtrichards joshtrichards changed the title [Bug]: Nextcloud Hub 8, copying files to an External Storage configured as primary storage isn't reliable [Bug]: Nextcloud Hub 8, copying files to Object Storage configured as Primary Storage isn't reliable May 6, 2024
@joshtrichards
Copy link
Member

joshtrichards commented May 6, 2024 •
edited
Loading

@sl-lebrit Any chance you can test PR #43794 in your environment? It addresses the array to string conversion matter. It may address your overall report as well, but not certain. Some feedback would be helpful.

https://docs.nextcloud.com/server/latest/admin_manual/issues/applying_patch.html

@joshtrichards joshtrichards mentioned this issue May 6, 2024
Merged
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap 29-feedback bug feature: object storage
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(files): handle multidimensional arrays in scanner nextcloud/server
6 participants
@compuguy @joshtrichards @solracsf @szaimen @sl-lebrit @DaanSelen