Listen to scanned files and to generate previews for externally added files (if possible)

[DriftingShadows]

I have a snap install of Nextcloud on an aws instance and running into issues with generating previews for a smb share. The external drive is mounted and its added to nextcloud's UI via External Storage -> Local. The problem is when new files are being added to the external storage, pre-generate does not do anything (it is on a hourly cron job). Running it manually with the -vvv flag finishes in a second without any additional output. Meanwhile running '''generate-all -vvv''' slowly goes through all the new files as per usual. How would i go about addressing this issue?

[st3iny]

The problem is when new files are being added to the external storage

Are the files added to the external storage directly or via Nextcloud?

[DriftingShadows]

The problem is when new files are being added to the external storage

Are the files added to the external storage directly or via Nextcloud?

apologies for delayed response. it is added directly via rsync, not via nextcloud. if doing it this way is a problem, is there any other way to get preview generation working?

[DriftingShadows]

@st3iny sorry to bump but would you have any suggestions for my situation?

[nikolas-digitalBabylon]

It seems that when files are added externally (e.g. rsync) changes are not caught. Same situation here, running on docker and using locally mounted external storage.

[DriftingShadows]

It seems that when files are added externally (e.g. rsync) changes are not caught. Same situation here, running on docker and using locally mounted external storage.

i suspected as much. Is there another way to get these files to be picked up by preview generator?

[nikolas-digitalBabylon]

It seems that when files are added externally (e.g. rsync) changes are not caught. Same situation here, running on docker and using locally mounted external storage.

i suspected as much. Is there another way to get these files to be picked up by preview generator?

Add external files via the webdav interface provided by nextcloud. Should work then, let us know if it does (for reference).

[st3iny]

Yeah, that is correct. Only files that are added or modified using either the desktop client, webdav or the web interface are registered.

Adding the files externally and running a scan will not pick up changes unfortunately.

For now there is no workaround. The proper fix would be to implement a new listener to listen for scanned files (if possible).

[agoeckner]

This is definitely a major issue for those of us manipulating files externally.

It seems that this app should listen for OCP\Files\Events\FileScannedEvent, from the limited documentation here: https://docs.nextcloud.com/server/latest/developer_manual/basics/events.html

[agoeckner]

I started working on this but then realized that we don't have node ID information in the FileScannedEvent. See the "to-do" below:

namespace OCA\PreviewGenerator\Listeners;

use OCP\EventDispatcher\Event;
use OCP\EventDispatcher\IEventListener;
use OCP\Files\Events\FileScannedEvent;
use OCP\Files\Folder;
use OCP\IDBConnection;
use OCP\IUserManager;

class PostWriteListener implements IEventListener {
	private IDBConnection $connection;
	private IUserManager $userManager;

	public function __construct(IDBConnection $connection,
		IUserManager $userManager) {
		$this->connection = $connection;
		$this->userManager = $userManager;
	}

	public function handle(Event $event): void {
		if (!($event instanceof FileScannedEvent)) {
			return;
		}

		$absPath = $event->getAbsolutePath();
		$id = #TODO --- FIGURE OUT HOW TO GET THE ID OF THE FILE;
		$owner = explode('/', $absPath)[0];

		if (!$this->userManager->userExists($owner)) {
			return;
		}

		$qb = $this->connection->getQueryBuilder();
		$qb->select('id')
			->from('preview_generation')
			->where(
				$qb->expr()->andX(
					$qb->expr()->eq('uid', $qb->createNamedParameter($owner)),
					$qb->expr()->eq('file_id', $qb->createNamedParameter($id))
				)
			)->setMaxResults(1);
		$cursor = $qb->execute();
		$inTable = $cursor->fetch() !== false;
		$cursor->closeCursor();

		// Don't insert if there is already such an entry
		if ($inTable) {
			return;
		}

		$qb = $this->connection->getQueryBuilder();
		$qb->insert('preview_generation')
			->setValue('uid', $qb->createNamedParameter($owner))
			->setValue('file_id', $qb->createNamedParameter($id));
		$qb->execute();
	}
}

========

One option might be to copy what the Recognize app does: https://github.com/nextcloud/recognize/blob/main/lib/Hooks/FileListener.php