Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[regression] symlinked apps and CSS - ResourceLocator can not find a web root #13556

Closed
Daryes opened this issue Jan 13, 2019 · 43 comments · Fixed by #22550
Closed

[regression] symlinked apps and CSS - ResourceLocator can not find a web root #13556

Daryes opened this issue Jan 13, 2019 · 43 comments · Fixed by #22550
Labels
1. to develop Accepted and waiting to be taken care of bug

Comments

@Daryes
Copy link

Daryes commented Jan 13, 2019

Seems to be the same problem as #5057 and others simlar.
Given the logs, it seems this was fixed in 12.x, but a change somewhere later removed the fix partially. I can go back to at least v13.0.1.1, up to 15.0.2.0

Currently, the apps work fine, while stored in a different dir symlinked back to the nextcloud main directory. But each time a user access them in the UI, an entry is generated in the log with an access error to a css file.

Steps to reproduce

  1. store the non built-in application in another directory, which is a symlink
  • nexcloud is under /server/websites/nextcloud/nextcloud-<version>/
  • external app are under /server/websites/nextcloud/conf/apps2/
  • create a symlink for nextcloud current version :
    ln -s /server/websites/nextcloud/nextcloud-<version> /server/websites/nextcloud/nextcloud
  • create the apps symlink :
    ln -s /server/websites/nextcloud/conf/apps2 /server/websites/nextcloud/nextcloud/apps2
  1. change the config.php with the app2 path information :
  'apps_paths' =>
  array (
    0 =>
    array (
      'path' => '/server/websites/nextcloud/nextcloud/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 =>
    array (
      'path' => '/server/websites/nextcloud/nextcloud/apps2',
      'url' => '/apps2',
      'writable' => true,
    ),
  ),
  1. use nextcloud, add some apps (like calendar, contacts, ...)
  2. click on the app bar at the top to show each of them, no need to do more. You can also see they appear correctly, and in fact they work correctly.
  3. check the log. Multiply the number of line by the number of external apps, and the number of users.

Expected behaviour

no warning in the logs

Actual behaviour

these entries in the log, one for each application under a symlink in the path, each time a user selects the application panel in his browser. Given the filename, I suppose this is automatically generated, as nothing like this exists physically.

ResourceLocator can not find a web root (root: /server/websites/nextcloud/conf/apps2/contacts, file: index.php/css/contacts/3d4b-bdca-contacts.css?v=96f9b18ad190af3746726337d98b2100, webRoot: , throw: true)

ResourceLocator can not find a web root (root: /server/websites/nextcloud/conf/apps2/calendar, file: index.php/css/calendar/029b-bdca-app.min.css?v=96f9b18ad190af3746726337d98b2100, webRoot: , throw: true)

ResourceLocator can not find a web root (root: /server/websites/nextcloud/conf/apps2/notes, file: index.php/css/notes/21f4-bdca-notes.css?v=96f9b18ad190af3746726337d98b2100, webRoot: , throw: true)

(... continue for any application under the apps2 directory)

Server configuration

Operating system: Ubuntu 18.04
Web server: Apache 2.4
Database: Mariadb 10.3
PHP version: PHP 7.2.14

Nextcloud version: 15.0.2.0

Updated from an older Nextcloud/ownCloud or fresh install: upgraded from 14.0.4
Where did you install Nextcloud from: : tar.gz from nextcloud.com

@Daryes Daryes added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Jan 13, 2019
@kesselb
Copy link
Contributor

kesselb commented Jan 15, 2019

Ref #10437

@nebulade
Copy link

nebulade commented May 2, 2019

This also happens with the Cloudron app package with Nextcloud version 16.0.0
On Cloudron the apps folder is also symlinked. The files itself are publicly reachable just fine, still the ResourceLocator shows them missing in the error log.

@the-sane
Copy link

Confirming that this remains an issue in the v16 release series. It affects the new text app (nextcloud/text#157)

I believe this is the same issue that's being discussed over on the snap here: nextcloud-snap/nextcloud-snap#902. @juliushaertl stated in nextcloud-snap/nextcloud-snap#902 (comment):

This is most likely an issue of

protected function findWebRoot($root) {
that doesn't detect the webroot properly if the apps directory is in a different place.

@szaimen
Copy link
Contributor

szaimen commented Jul 24, 2019

Please fix this issue finally! That issue is very annoying and now also affects the usability (especially when enabling the dark mode through accessibility settings. If doing so there are a lot of icons missing in many apps (also in nextcloud_files).

@szaimen
Copy link
Contributor

szaimen commented Jul 24, 2019

I think these are also the same error:
nextcloud-snap/nextcloud-snap#402
nextcloud-snap/nextcloud-snap#625

@kesselb
Copy link
Contributor

kesselb commented Jul 24, 2019

$app_path = realpath($app_path);

Symlink is resolved to early. findWebRoot method tries to determinate the webroot from the path. Nextcloud is installed to /home/nextcloud and the apps are in /home/nextcloud-apps. /home/nextcloud/apps2 is a symlink to /home/nextcloud/apps2. Its possible to detect the webroot for /home/nextcloud/apps2 but not /home/nextcloud-apps.

This one is the faulty commit b0d2966

cc @kyrofa @juliushaertl

@kyrofa
Copy link
Member

kyrofa commented Jul 24, 2019

This one is the faulty commit b0d2966

I'm not sure that's accurate given that the commit you reference is very old (it's what fixed things back in v12). Behavior around it might have changed in order to cause this regression, though.

@kesselb
Copy link
Contributor

kesselb commented Jul 24, 2019

I'm not sure that's accurate given

That's right! I'm sorry my statement was misleading.

Behavior around it might have changed in order to cause this regression

I think so. Could you check if it works for snap without realpath?

@georgehrke
Copy link
Member

Summoning our experts here: @skjnldsv @juliushaertl :)

@skjnldsv suspects it's just a debug log accidentally logged as error.

@szaimen
Copy link
Contributor

szaimen commented Aug 5, 2019

@skjnldsv suspects it's just a debug log accidentally logged as error.

I don't think so as i mentioned in nextcloud/text#187

@georgehrke
Copy link
Member

@skjnldsv Could you specify what kind if info is missing here? :)

@the-sane
Copy link

the-sane commented Aug 5, 2019

@skjnldsv I can also confirm the missing toolbar buttons as a result of this problem, and I also have Nextcloud installed via snap. For me, it's a problem with both the light and dark themes though. It is not just a logging mistake, and @juliushaertl has confirmed that (see below, or above).

@georgehrke, thank you for pointing the experts here!


As for the info requested, okay, I think this is everything we know about this issue so far, gathered from this and related issues:

@szaimen posted a screenshot of the missing buttons in the new text app here:
nextcloud/text#187 (comment)

@juliushaertl has said this server issue is the cause of the problem, also explaining why it's not just a bug in the logger:
nextcloud/text#187 (comment)
nextcloud/text#187 (comment)
nextcloud/text#187 (comment)

This is most likely an issue of the server in combination with the snap: nextcloud/server#13556

We cannot workaround that because the apps directory depends on the server configuration. You might have apps-extra where the text app is located but others might have it in the regular apps or an apps2 folder. The issue here is that the server cannot find the folder, since it is not inside of the nextcloud folder when using the snap.

https://domain/index.php/svg/files/public is a dynamic endpoint (which is handled by the Nextcloud server code) that looks for the apps folder of the files app and returns the image. (this is what fails on your setup) https://domain/apps/files/img/recent.svg will be served by your webserver so Nextcloud is not involved there.

It is an issue with many apps for anyone using Nextcloud via snap, as is being discussed here:
nextcloud-snap/nextcloud-snap#902

@kyrofa, the snap maintainer, has said it's not a problem that the snap can fix:
nextcloud-snap/nextcloud-snap#902 (comment)

Nothing has changed in the snap to cause this (the config has always been the same), which leads me to suspect the apps themselves, unless something broke in Nextcloud itself again.

I've tried to give guidance on how to fix it 🤷‍♂️ .

@Daryes ,the OP, has stated this was fixed in a previous version of Nextcloud but has re-presented itself.
@kyrofa has given some examples:
nextcloud-snap/nextcloud-snap#902 (comment)

we've dealt with this issue in the past in upstream Nextcloud (and upstream apps), for example:

nextcloud/server#7257
nextcloud/server#7061
nextcloud/server#8463 (theming app)

Here are some examples of the apps that have problems:
nextcloud/calendar#1348
nextcloud/spreed#1615
nextcloud/text#157
rullzer/dropit#22
nextcloud/mail#1066

@juliushaertl has given some guidance on where the problem may lie:
nextcloud-snap/nextcloud-snap#902 (comment)

This is most likely an issue of

protected function findWebRoot($root) {
that doesn't detect the webroot properly if the apps directory is in a different place.

As for how many people are affected by this, @jospoortvliet noted the last time something similar was fixed in 2017 that many thousands use Nextcloud via snap:
#5057 (comment)

It might be a small patch but we have over 1500 people with a Nextcloud Box at home who can now get 12.0.1 and that's ignoring the many thousands who use the Snap on a normal Ubuntu or other Linux system!

@kyrofa
Copy link
Member

kyrofa commented Aug 5, 2019

As for how many people are affected by this [...]

It's around 30k today, ignoring folks using similar configurations outside the snap (like the OP).

@skjnldsv
Copy link
Member

skjnldsv commented Aug 5, 2019

@skjnldsv Could you specify what kind if info is missing here? :)

@georgehrke Julius's input :)

@juliusknorr
Copy link
Member

Well currently the insight given in the text ticket which @the-sane has quoted is all I have. I didn't have time to look further into it or setup a test system with this special setup.

@the-sane
Copy link

@juliushaertl I know you're all focused on the v17 beta right now, so please let me know if there's any information that I can provide from my own install that would help here.

@pricard32
Copy link

Any news? I'm still having this issue.

@CorinthCom
Copy link

CorinthCom commented Nov 13, 2019

Using Snap on Ubuntu 18.04
Started getting the same issue in the log;

ResourceLocator can not find a web root (root: /var/snap/nextcloud/16739/nextcloud/extra-apps/spreed, file: index.php/css/spreed/41bb-1b6e-merged-files.css?v=xxxxxxxxxxx, webRoot: , throw: true)

errors in log for Snap v15.0.11 (15910) also in (15565)
errors after a Snap upgrade to v16.0.5 (16739)
edit:after having a proper search through the log it also happened in Snap 12.0.5.3 (5735) back in 2018-04-01 when we installed the Talk App.

@kyrofa
Copy link
Member

kyrofa commented Nov 20, 2019

@juliushaertl could really use some help here, it's simply been a log annoyance for several major releases but it negatively impacts the text app (as mentioned by @the-sane here) which is default in v17, thus blocking the snap's update to v17.

@kesselb

This comment has been minimized.

@kesselb
Copy link
Contributor

kesselb commented Nov 23, 2019

I wrote a patch to fix the ResourceLocator can not find a web root message by adding the appsroots to mappings map but text app's icons still missing 😞 😠

@kyrofa
Copy link
Member

kyrofa commented Nov 24, 2019

If text is shipped with Nextcloud 17 it should be installed to '/snap/'.$snap_name.'/current/htdocs/apps', right?

@kesselb you're exactly correct.

Is there a Nextcloud 17 snap somewhere to test?

Sure thing, sudo snap install nextcloud --channel=17/edge will get you a snap built daily off of the latest-stable17 tarball. However, snaps are read-only, so that's not particularly nice for poking around. if you want to do that, you can download the snap without installing like so:

$ snap download nextcloud --channel=17/edge

Unpack it:

$ unsquashfs -d unpacked nextcloud_<revision>.snap

And then you can "try" it instead of installing it, which is a developer tool for essentially installing it read/write from the "unpacked" directory you just created:

$ sudo snap try unpacked/

Now the snap is installed and listening on localhost:80, but you can modify unpacked/htdocs to your hearts content.

Note that I don't develop against Nextcloud that way, though, I typically just setup a similar config.

@kesselb
Copy link
Contributor

kesselb commented Nov 24, 2019

$appRootPath = $this->appManager->getAppPath($app);
$appPath = substr($appRootPath, strlen($this->serverRoot));
if (!$appPath) {
return new NotFoundResponse();
}
$path = $this->serverRoot . $appPath ."/img/$fileName.svg";

Fails.

appRootPath: /snap/nextcloud/current/htdocs/apps/text
appPath: tdocs/apps/text
path: /snap/nextcloud/x1/htdocstdocs/apps/text/img/h1.svg

Hmm. I don't see the reason for substr($appRootPath, strlen($this->serverRoot)). Probably some security check like "don't read svg files outside the webroot". If $appPath should be the app path relative to https://nextcloud.com/ (e.g. /extra-apps/text) \OC_App::getAppWebPath($appId); is the way to go. People will still see the ResourceLocator warning.

Index: core/Controller/SvgController.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- core/Controller/SvgController.php	(revision 4f2837017c95f063d99a05b6cd6a5e73ac388d75)
+++ core/Controller/SvgController.php	(date 1574613795696)
@@ -24,6 +24,7 @@
 
 namespace OC\Core\Controller;
 
+use OCP\App\AppPathNotFoundException;
 use OCP\AppFramework\Controller;
 use OCP\AppFramework\Http;
 use OCP\AppFramework\Http\DataDisplayResponse;
@@ -96,13 +97,13 @@
 			return $this->getSvg($path, $color, $fileName);
 		}
 
-		$appRootPath = $this->appManager->getAppPath($app);
-		$appPath = substr($appRootPath, strlen($this->serverRoot));
-
-		if (!$appPath) {
+		try {
+			$appPath = $this->appManager->getAppPath($app);
+		} catch (AppPathNotFoundException $e) {
 			return new NotFoundResponse();
 		}
-		$path = $this->serverRoot . $appPath ."/img/$fileName.svg";
+
+		$path = $appPath . "/img/$fileName.svg";
 		return $this->getSvg($path, $color, $fileName);
 	}

cc @juliushaertl @skjnldsv @rullzer does that make sense? 🤔 Smoke test looked good to me. I don't have time next week to create a pull request and look into the tests 🙈

@the-sane
Copy link

A note as a user: While fixing the actual problem is obviously more important, those ResourceLocator errors are extremely annoying and have made our logs entirely unreadable. Nearly every app installed produces at least one of those errors every time a page is accessed.

@kyrofa
Copy link
Member

kyrofa commented Nov 24, 2019

appRootPath: /snap/nextcloud/current/htdocs/apps/text
appPath: tdocs/apps/text
path: /snap/nextcloud/x1/htdocstdocs/apps/text/img/h1.svg

That sounds very familiar to the issue in the JS and CSS resource locator back in the day (fix here). Your patch also looks similar to the theming fix.

@kesselb
Copy link
Contributor

kesselb commented Nov 24, 2019

A note as a user: While fixing the actual problem is obviously more important, those ResourceLocator errors are extremely annoying and have made our logs entirely unreadable.

Made the web root detection work with https://gist.github.com/kesselb/d035c4a7334b155143686466370bc1eb but the generated path for the resources is wrong. I assume that for scss files the webRoot is not relevant. SCSS file is compiled and served by core.Css.getCss. Could not see anything broken with the patch below:

Index: lib/private/Template/CSSResourceLocator.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- lib/private/Template/CSSResourceLocator.php	(revision 4f2837017c95f063d99a05b6cd6a5e73ac388d75)
+++ lib/private/Template/CSSResourceLocator.php	(date 1574619791911)
@@ -125,26 +125,7 @@
 		if (!$scss) {
 			parent::append($root, $file, $webRoot, $throw);
 		} else {
-			if (!$webRoot) {
-				$webRoot = $this->findWebRoot($root);
-
-				if ($webRoot === null) {
-					$webRoot = '';
-					$this->logger->error('ResourceLocator can not find a web root (root: {root}, file: {file}, webRoot: {webRoot}, throw: {throw})', [
-						'app' => 'lib',
-						'root' => $root,
-						'file' => $file,
-						'webRoot' => $webRoot,
-						'throw' => $throw ? 'true' : 'false'
-					]);
-
-					if ($throw && $root === '/') {
-						throw new ResourceNotFoundException($file, $webRoot);
-					}
-				}
-			}
-
-			$this->resources[] = array($webRoot? : \OC::$WEBROOT, $webRoot, $file);
+			$this->resources[] = array(\OC::$WEBROOT, \OC::$WEBROOT, $file);
 		}
 	}
 }

@skjnldsv
Copy link
Member

@kyrofa can you confirm @kesselb's patch fixes your issues?

@dtzWill
Copy link

dtzWill commented Dec 3, 2019

Thanks for working on this, all, been following this and related issues for some time.

Anyway, sorry for the dumb question but:

@kyrofa can you confirm @kesselb's patch fixes your issues?

I was interested in trying this as well, but... just wanted to clarify: is applying the inlined patch by itself intended to be what is tested, or should the gist patch be used as well?
(unsure if the latest inline patch is meant to be used in addition to the latest gist, or instead of it...?)

Thanks again and I'll of course report if I find anything useful but so far I've been quiet as I've only found ways to not solve my variant of the issue 😸 .

@kyrofa
Copy link
Member

kyrofa commented Dec 3, 2019

Good question, I took a crack at applying the changes here (just the ones given here, not the gist), but at least at first blush it doesn't appear to work. I need to test some more.

@kesselb
Copy link
Contributor

kesselb commented Dec 4, 2019

or should the gist patch be used as well?

No. Only the patches in this thread: #13556 (comment) & #13556 (comment)

least at first blush it doesn't appear to work.

More digging ;)

solve my variant of the issue

I checked the setup from post 1 #13556 (comment) and the snap package #13556 (comment)

Feel free to use the original code and something like below. Change the loglevel in config.php to 0 and it will write the paths to the log.

Index: core/Controller/SvgController.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- core/Controller/SvgController.php	(revision be54491ae15ce76c56549fc07d5cd810cdc49d62)
+++ core/Controller/SvgController.php	(date 1575493608385)
@@ -92,11 +92,15 @@
 	public function getSvgFromApp(string $app, string $fileName, string $color = 'ffffff') {
 		$appRootPath = $this->appManager->getAppPath($app);
 		$appPath = substr($appRootPath, strlen($this->serverRoot));
+		\OC::$server->getLogger()->debug('$appRootPath: "' . $appRootPath . '"');
+		\OC::$server->getLogger()->debug('$serverRoot: "' . $this->serverRoot . '"');
+		\OC::$server->getLogger()->debug('$appPath: "' . $appPath . '"');
 
 		if (!$appPath) {
 			return new NotFoundResponse();
 		}
 		$path = $this->serverRoot . $appPath ."/img/$fileName.svg";
+		\OC::$server->getLogger()->debug('$path: "' . $path . '"');
 		return $this->getSvg($path, $color, $fileName);
 	}
 

@janix82
Copy link

janix82 commented Dec 30, 2019

A note as a user: While fixing the actual problem is obviously more important, those ResourceLocator errors are extremely annoying and have made our logs entirely unreadable. Nearly every app installed produces at least one of those errors every time a page is accessed.

It was my soluition to get rid of this logs, after this my log is readable. You can make a cronjob to clear the nextcloud.log file.

Example:
#Nextcloud log cleaner (in every hour)
30 * * * * grep -v "webRoot: " /var/snap/nextcloud/common/nextcloud/data/nextcloud.log > /var/snap/nextcloud/common/nextcloud/data/nextcloud2.log; mv /var/snap/nextcloud/common/nextcloud/data/nextcloud2.log /var/snap/nextcloud/common/nextcloud/data/nextcloud.log

@kyrofa
Copy link
Member

kyrofa commented Jan 14, 2020

I took a crack at applying the changes here (just the ones given here, not the gist), but at least at first blush it doesn't appear to work. I need to test some more.

I'm loathe to admit that I apparently didn't fully understand this issue and how it impacted the text app. I missed two key points:

  1. The toolbar only shows up for .md files. For .txt files it doesn't, by design
  2. The toolbar only fails to show up when editing .md files in dark mode

@the-sane finally hammered those two points home, and I realize now that these patches DO work @kesselb. Ship them!

@wiswedel

This comment has been minimized.

@solracsf

This comment has been minimized.

@MorrisJobke
Copy link
Member

#19084 is merged

@ddoherty03
Copy link

@kyrofa I have noticed that syncing my calendar with Emacs org-mode has stopped working. I suspect this issue. I'm using org-caldav, which has worked nicely for a long time, but lately am getting this:

<?xml version="1.0" encoding="utf-8"?>
<d:error xmlns:d="DAV:" xmlns:s="http://sabredav.org/ns">
  <s:exception>Sabre\DAV\Exception\NotFound</s:exception>
  <s:message>File with name /calendars could not be located</s:message>
</d:error>

Using nextcloud snap on ubuntu 18.04:

# snap list
Name       Version      Rev    Tracking       Publisher   Notes
core       16-2.44.3    9066   latest/stable  canonical✓  core
nextcloud  18.0.4snap1  20498  latest/stable  nextcloud✓  -

Will this fix be coming soon to the snap? Or is it already in, and I have a different problem?

Thanks for your work on this, BTW. The snap is a hugely convenient way to get a nextcloud server up and running.

@kyrofa
Copy link
Member

kyrofa commented May 6, 2020

@ddoherty03 looks like #19084 won't be backported to v18, so it's not in the snap you have installed. However, I'd be a bit surprised if that was your issue. This has been a problem for a long time, so if something suddenly stopped working I suspect you have unrelated issues.

@ddoherty03
Copy link

Ok. Thanks, @kyrofa .

@nickvergessen
Copy link
Member

can people please try if https://github.com/nextcloud/server/pull/22550/files solves the issue for them?

@Daryes
Copy link
Author

Daryes commented Sep 2, 2020

Just applied, seems good at first glance, no more log warnings, and no problem in the UI,
I'll check more later.

BTW, as I'm not using snap for Nexcloud, I can't speak for it.

@Magicrafter13
Copy link

Magicrafter13 commented May 25, 2021

This appears to still be an issue, or at least something related?
Taken from my nextcloud.log:

"Message": "`../../../core/css/whatsnew.scss` file not found for @import: /mnt/nc-data/apps/files/css/merged.scss on line 5, at column 1\nCall Stack:\n#0 import /mnt/nc-data/apps/files/css/merged.scss (unknown file) on line 1",

As you can see, I have my apps directory stored in /mnt/nc-data which is a separate partition from /. The web files however, are located in /var/www/nextcloud.
My "solution" to this problem was to symlink core on the data partition: ln -s /var/www/nextcloud/core/ /mnt/nc-data/, which seems to work fine, but is only a workaround.

Had to do this in order to use the Files app (everything else seemed to work fine though, including Photos...), otherwise I got an Internal Server Error (what led me to find the log entry).

Nextcloud 21.0.2 on Fedora 34 if it matters.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1. to develop Accepted and waiting to be taken care of bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.