Files from external storage (Dropbox) never get updated.

[powerpaul17]

Steps to reproduce

1. connect Dropbox as external storage
2. sync the folder with the desktop client
3. files get downloaded once but never updated again, update also does not happen in web interface

Expected behaviour

Files should update.

Actual behaviour

Files are not updated.

Server configuration

Operating system: Linux (Ubuntu 16.10)

Web server: Apache

Database: MySQL (5.7.16)

PHP version: 7.0.13

Nextcloud version: 11.0.1

Updated from an older Nextcloud/ownCloud or fresh install: upgraded from NC 10

Where did you install Nextcloud from: from NC homepage

Signing status:

Signing status

No errors have been found.

Are you using encryption: no

[nickvergessen]

@icewind1991 what was the name of the config for discovering changes?

[philsson]

Interesting. Just noticed the same thing myself. I'll follow this thread

[borekon]

The same happens to me. It seems if run occ files:scan --all it updates, but the process takes like 3 hours...

[heskja]

Also following.

I tried to move some files to a subfolder in Dropbox, then came back to nextcloud to find that nextcloud did not detect the changes. When I tried to move the files from nextcloud, I get an error saying that the files already exist in the directory, but no files are visible.

[heskja]

Doing some further research yielded this Dropbox developer blog post: https://blogs.dropbox.com/developers/2013/11/low-latency-notification-of-dropbox-file-changes/

Seems like long polling with some implementation of the delta API calls would solve this. Not sure where in nextcloud code this should be added though.

[ManOki]

Workaround/Script for shell-users. No warranty, tested on NC12.

#!/bin/bash

PHP="/usr/bin/php"
OCC="/var/www/nextcloud/occ"


# Admin Mountpoints
${PHP} ${OCC} files_external:list | grep "| [0-9]" | while read MOUNTLINE ; do
	TYPE=$(echo "${MOUNTLINE}" | awk -F'[ \t]*[|][ \t]*' '{ print $4 }')
	if [ "$TYPE" == "Dropbox" ] ; then
		MOUNTPOINT=$(echo "${MOUNTLINE}" | awk -F'[ \t]*[|][ \t]*' '{ print $3 }')
		# Applicable Users
		echo "${MOUNTLINE}" | awk -F'[ \t]*[|][ \t]*' '{ print $8 }' | while IFS=', ' read -ra USERNAMES ; do
			for USERNAME in "${USERNAMES[@]}" ; do
				${PHP} ${OCC} files:scan --path="${USERNAME}/files${MOUNTPOINT}" --quiet
			done
		done
		# Applicable Groups
		echo "${MOUNTLINE}" | awk -F'[ \t]*[|][ \t]*' '{ print $9 }'| while IFS=', ' read -ra GROUPNAMES ; do
			for GROUPNAME in "${GROUPNAMES[@]}" ; do
				${PHP} ${OCC} group:list | awk "/${GROUPNAME}:/{flag=1;next}/:/{flag=0}flag" | cut -c 7- | while read USERNAME ; do
					${PHP} ${OCC} files:scan --path="${USERNAME}/files${MOUNTPOINT}" --quiet
				done
			done
		done
	fi
done

# User Mointpoints
${PHP} ${OCC} user:list | awk -F'[ \t]*[-:][ \t]*' '{ print $2 }' | while read USERNAME ; do
	${PHP} ${OCC} files_external:list $USERNAME | grep "| [0-9]" | while read MOUNTLINE ; do
		TYPE=$(echo "${MOUNTLINE}" | awk -F'[ \t]*[|][ \t]*' '{ print $4 }')
		if [ "$TYPE" == "Dropbox" ] ; then
			MOUNTPOINT=$(echo "${MOUNTLINE}" | awk -F'[ \t]*[|][ \t]*' '{ print $3 }')
			${PHP} ${OCC} files:scan --path="${USERNAME}/files${MOUNTPOINT}" --quiet
		fi
	done
done

Currently user:list and group:list only lists 500 (default), change that to your needs.

[towerlexa]

Hi, i would like to ask, if this script is working to refresh the content of the integrated dropbox account.
In my mind, the connector don't refresh the file listing from the dropbox data.
This is also a linked problem: https://help.nextcloud.com/t/desktop-client-delta-updates-to-files/7790/6
Does your script activate the dropbox connector itself, of does is only do a local filescan? (In my mind you do a local filesync for nextcloud, but not reading nor activating the dropbox connector, right?

Thank you very much.

[paour]

I have this problem with an admin-added external folder. I've tried switching to a user-added external storage, hoping that until this is resolved, at least @ManOki's script will help (it doesn't appear to work for admin-added folders).

[towerlexa]

Hi Paour,
thanks for your answer. I'll try it again as a user added external storage!
Cool!
regards, Axel

[ConorIA]

My solution, inspired by @ManOki is as follows, not that I'm on Manjaro and Nextcloud is running on Docker.

~ >>> cat /etc/systemd/system/dropbox.service

[Unit]
Description=Scan for new files on Dropbox

[Service]
Type=simple
ExecStart=/usr/bin/docker exec -i nextcloud_server_1 occ files:scan --path="/conor/files/Dropbox"

~ >>> cat /etc/systemd/system/dropbox.timer

[Unit]
Description=Scrape Dropbox Hourly

[Timer]
OnCalendar=hourly
Persistent=true

[Install]
WantedBy=timers.target

[ManOki]

@paour what are admin-added external folders? are those directly shared to all users? are they added through a plugin another app?

maybe just a hint, how this script works:
first line, all usernames are queried from nextcloud by occ user:list
second line, all mountpoint for that user are queried from nextcloud by occ files_external:list $USERNAME
for each of those user mountpoints, an refresh is triggered via occ files:scan --path="${USERNAME}/files${MOUNTPOINT}"

so if the admin-added mountpoints are not listed by the nextcloud commands mentioned above, the script will fail. but maybe we can add some lines to complete it.

EDIT: @ConorIA you hardcoded your path --path="/conor/files/Dropbox", but this is not generic for all and only supports single user

[paour]

@ManOki yes, the admin can allow users to add their own mappings, or can set mappings for one or several users from the admin console. These do not appear in occ files_external:list $USERNAME.

[ManOki]

@paour can you show me such a command line for the admin console to set a mapping, please.

[paour]

I'm not aware of a command-line to do this, but you can do it in the web admin console:

[ManOki]

Thanks for the hint, i wasn't aware of that. Admin mountpoints are listed without any user_id, just occ files_external:list.
I will update my comment with the script...just checking, how to update the path without a valid username

[paour]

Aha, then my issue may have been that NC forgot my mountpoint: it wasn't in the list when I ran that command. It still appeared in the web UI though.

[ManOki]

Ok, i updated the script to support also admin mountpoints. I didn't test, if it is valid to just trigger a single update on any user. current state is, that all applicable users and groups with their users get updated, which might cause a lot of traffic (to dropbox).

@paour Maybe your mountpoint was defined a long time ago with an old owncloud/nextcloud instance, so those information are not stored properly. Now the script should work, if occ files_external:list list your mountpoint.

[MorrisJobke]

Dropbox support was moved to a separate app, please report issues with this over there: https://github.com/icewind1991/files_external_dropbox

Please try this new app, because it uses updated libraries that support newer Dropbox APIs.

The app works with Nextcloud 12+.