file_server browse does not work

[linuxgemini]

Some parts of the issue report has been altered for privacy reasons.

1. Caddy version (caddy version):

v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=

2. How I run Caddy:

a. System environment:

root@ams1:~# uname -a
Linux ams1.innerlink.local 5.10.0-0.bpo.9-amd64 #1 SMP Debian 5.10.70-1~bpo10+1 (2021-10-10) x86_64 GNU/Linux

root@ams1:~# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 10 (buster)
Release:        10
Codename:       buster

root@ams1:~#

b. Command:

root@ams1:~# systemctl status caddy
● caddy.service - Caddy
   Loaded: loaded (/lib/systemd/system/caddy.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2021-12-05 11:35:33 UTC; 5min ago
     Docs: https://caddyserver.com/docs/
 Main PID: 7271 (caddy)
    Tasks: 7 (limit: 2343)
   Memory: 11.8M
   CGroup: /system.slice/caddy.service
           └─7271 /usr/bin/caddy run --environ --config /etc/caddy/Caddyfile

c. Service/unit/compose file:

root@ams1:~# cat /lib/systemd/system/caddy.service
# caddy.service
#
# For using Caddy with a config file.
#
# Make sure the ExecStart and ExecReload commands are correct
# for your installation.
#
# See https://caddyserver.com/docs/install for instructions.
#
# WARNING: This service does not use the --resume flag, so if you
# use the API to make changes, they will be overwritten by the
# Caddyfile next time the service is restarted. If you intend to
# use Caddy's API to configure it, add the --resume flag to the
# `caddy run` command or use the caddy-api.service file instead.

[Unit]
Description=Caddy
Documentation=https://caddyserver.com/docs/
After=network.target network-online.target
Requires=network-online.target

[Service]
Type=notify
User=caddy
Group=caddy
ExecStart=/usr/bin/caddy run --environ --config /etc/caddy/Caddyfile
ExecReload=/usr/bin/caddy reload --config /etc/caddy/Caddyfile
TimeoutStopSec=5s
LimitNOFILE=1048576
LimitNPROC=512
PrivateTmp=true
ProtectSystem=full
AmbientCapabilities=CAP_NET_BIND_SERVICE

[Install]
WantedBy=multi-user.target

root@ams1:~#

d. My complete Caddyfile or JSON config:

root@ams1:~# cat /etc/caddy/Caddyfile
{
        admin off
}

http://ams1.innerlink.local {
        header -Server

        handle /arandompath/* {
                uri strip_prefix /arandompath

                file_server {
                        root /var/www
                        index none.index.index.none.txt
                        browse
                }
        }

        handle /* {
                respond "what." 404
        }
}

root@ams1:~#

3. The problem I'm having:

Caddy tries to load the given (non-existant) index file none.index.index.none.txt and when it can't find the file; instead of rendering the file browser, it throws a 404.

4. Error messages and/or full log output:

root@ams1:~# curl -sv --resolve ams1.innerlink.local:443:127.0.0.1 https://ams1.innerlink.local/arandompath/ 2>&1 >/dev/null | grep "< HTTP/"
< HTTP/2 404
root@ams1:~#

root@ams1:~# systemctl restart caddy && journalctl -fu caddy
-- Logs begin at Sun 2021-12-05 03:34:33 UTC. --
Dec 05 11:49:49 ams1.innerlink.local caddy[20654]: {"level":"debug","ts":1638704989.4130344,"logger":"http","msg":"starting server loop","address":"[::]:8443","http3":false,"tls":true}
Dec 05 11:49:49 ams1.innerlink.local caddy[20654]: {"level":"debug","ts":1638704989.4136665,"logger":"http","msg":"starting server loop","address":"[::]:443","http3":false,"tls":true}
Dec 05 11:49:49 ams1.innerlink.local caddy[20654]: {"level":"info","ts":1638704989.4171598,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/.config/caddy/autosave.json"}
Dec 05 11:49:49 ams1.innerlink.local systemd[1]: Started Caddy.
Dec 05 11:49:49 ams1.innerlink.local caddy[20654]: {"level":"info","ts":1638704989.4223723,"msg":"serving initial configuration"}
Dec 05 11:49:50 ams1.innerlink.local caddy[20654]: {"level":"debug","ts":1638704990.6840165,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_addr":"127.0.0.1:58676","proto":"HTTP/2.0","method":"GET","host":"ams1.innerlink.local","uri":"/arandompath/","headers":{"User-Agent":["curl/7.64.0"],"Accept":["*/*"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"ams1.innerlink.local"}},"method":"GET","uri":"/"}
Dec 05 11:49:50 ams1.innerlink.local caddy[20654]: {"level":"debug","ts":1638704990.684057,"logger":"http.handlers.file_server","msg":"sanitized path join","site_root":"/var/www","request_path":"/","result":"/var/www"}
Dec 05 11:49:50 ams1.innerlink.local caddy[20654]: {"level":"debug","ts":1638704990.6840937,"logger":"http.handlers.file_server","msg":"no index file in directory","path":"/var/www","index_filenames":["none.index.index.none.txt"]}
Dec 05 11:49:50 ams1.innerlink.local caddy[20654]: {"level":"debug","ts":1638704990.6841054,"logger":"http.handlers.file_server","msg":"browse enabled; listing directory contents","path":"/var/www","root":"/var/www"}
^C

root@ams1:~#

5. What I already tried:

N/A

6. Links to relevant resources:

N/A

[francislavoie]

Hmm. Either I don't understand what you're reporting, or I can't replicate the issue.

I have a directory structure like this:

│   Caddyfile
└───foo
    └───foo.txt

caddy run from that directory, with a Caddyfile like this:

{
	debug
}

:8880 {
	root .
	file_server {
		index bar.txt
		browse
	}
}

• If I load http://localhost:8880/ in my browser, I get the file browser.
• If I load http://localhost:8880/foo, I get redirected to http://localhost:8880/foo/ which also gives me the file browser.
• If I change the index to foo.txt then loading http://localhost:8880/foo/ shows me the contents of /foo/foo.txt.

So, I don't understand what the problem is.

[linuxgemini]

Folder /var/www:

root@ams1:/opt/caddytest# ls -lah /var/www
total 24K
drwxrwxr-x  2 caddy www-data 4.0K Feb 11  2021 .
drwxr-xr-x 12 root  root     4.0K Jan 10  2021 ..
lrwxrwxrwx  1 caddy www-data   22 Jan 10  2021 nevergonna -> /home/give/you/up
-rw-rw-r--  1 caddy www-data  223 Jan 10  2021 index.css
-rw-rw-r--  1 caddy www-data 1.1K Jan 10  2021 index.html
-rw-rw-r--  1 caddy www-data  594 Jan 10  2021 index.md
lrwxrwxrwx  1 caddy www-data   11 Jan 10  2021 never -> /opt/gonna/let/you/down
-rw-rw-r--  1 caddy www-data   26 Jan 13  2021 robots.txt
root@ams1:/opt/caddytest#

Caddyfile:

root@ams1:/opt/caddytest# cat Caddyfile
{
	admin off
	debug
	auto_https off
}

http://172.16.1.1:64223 {
	handle /m9ZVVwChbVC76X/* {
		uri strip_prefix /m9ZVVwChbVC76X
		file_server {
			index bar.txt
			root /var/www
			browse
		}
	}

	handle /* {
		respond "pong"
	}
}
root@ams1:/opt/caddytest#

Curl output:

root@ams1:~# curl -sv 172.16.1.1:64223/m9ZVVwChbVC76X/
* Expire in 0 ms for 6 (transfer 0x55e58f1f7fb0)
*   Trying 172.16.1.1...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55e58f1f7fb0)
* Connected to 172.16.1.1 (172.16.1.1) port 64223 (#0)
> GET /m9ZVVwChbVC76X/ HTTP/1.1
> Host: 172.16.1.1:64223
> User-Agent: curl/7.64.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: Caddy
< Date: Tue, 07 Dec 2021 07:53:30 GMT
< Content-Length: 0
<
* Connection #0 to host 172.16.1.1 left intact
root@ams1:~#

Debug output with caddy run --environ --config ./Caddyfile:

caddy.HomeDir=/root
caddy.AppDataDir=/root/.local/share/caddy
caddy.AppConfigDir=/root/.config/caddy
caddy.ConfigAutosavePath=/root/.config/caddy/autosave.json
caddy.Version=v2.4.6 h1:HGkGICFGvyrodcqOOclHKfvJC0qTU7vny/7FhYp9hNw=
runtime.GOOS=linux
runtime.GOARCH=amd64
runtime.Compiler=gc
runtime.NumCPU=1
runtime.GOMAXPROCS=1
runtime.Version=go1.17.2
os.Getwd=/opt/caddytest

SHELL=/bin/bash
PWD=/opt/caddytest
LOGNAME=root
XDG_SESSION_TYPE=tty
HOME=/root
LANG=en_US.UTF-8
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
LC_TERMINAL=iTerm2
SSH_CONNECTION=SCRUBBED
XDG_SESSION_CLASS=user
TERM=xterm-256color
USER=root
LC_TERMINAL_VERSION=3.4.14
SHLVL=1
XDG_SESSION_ID=231
LC_CTYPE=UTF-8
XDG_RUNTIME_DIR=/run/user/0
SSH_CLIENT=SCRUBBED
LC_ALL=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
MAIL=/var/mail/root
SSH_TTY=/dev/pts/0
_=/usr/bin/caddy
OLDPWD=/root
2021/12/07 07:53:28.562	INFO	using provided configuration	{"config_file": "./Caddyfile", "config_adapter": ""}
2021/12/07 07:53:28.567	WARN	admin	admin endpoint disabled
2021/12/07 07:53:28.569	DEBUG	http	starting server loop	{"address": "[::]:64223", "http3": false, "tls": false}
2021/12/07 07:53:28.571	INFO	autosaved config (load with --resume flag)	{"file": "/root/.config/caddy/autosave.json"}
2021/12/07 07:53:28.571	INFO	serving initial configuration
2021/12/07 07:53:28.569	INFO	tls	cleaning storage unit	{"description": "FileStorage:/root/.local/share/caddy"}
2021/12/07 07:53:28.573	INFO	tls	finished cleaning storage units
2021/12/07 07:53:28.569	INFO	tls.cache.maintenance	started background certificate maintenance	{"cache": "0xc00053d7a0"}
2021/12/07 07:53:30.485	DEBUG	http.handlers.rewrite	rewrote request	{"request": {"remote_addr": "172.16.1.1:39738", "proto": "HTTP/1.1", "method": "GET", "host": "172.16.1.1:64223", "uri": "/m9ZVVwChbVC76X/", "headers": {"User-Agent": ["curl/7.64.0"], "Accept": ["*/*"]}}, "method": "GET", "uri": "/"}
2021/12/07 07:53:30.485	DEBUG	http.handlers.file_server	sanitized path join	{"site_root": "/var/www/", "request_path": "/", "result": "/var/www"}
2021/12/07 07:53:30.485	DEBUG	http.handlers.file_server	no index file in directory	{"path": "/var/www", "index_filenames": ["bar.txt"]}
2021/12/07 07:53:30.485	DEBUG	http.handlers.file_server	browse enabled; listing directory contents	{"path": "/var/www", "root": "/var/www/"}
^C2021/12/07 07:53:33.711	INFO	shutting down	{"signal": "SIGINT"}
2021/12/07 07:53:33.712	WARN	exiting; byeee!! 👋	{"signal": "SIGINT"}
2021/12/07 07:53:33.713	INFO	tls.cache.maintenance	stopped background certificate maintenance	{"cache": "0xc00053d7a0"}
2021/12/07 07:53:33.713	INFO	shutdown complete	{"signal": "SIGINT", "exit_code": 0}

root@ams1:/opt/caddytest# groups caddy
caddy : caddy www-data
root@ams1:/opt/caddytest#

[linuxgemini]

I have also temporarily disabled the systemd protection measures:

root@ams1:/opt/caddytest# cat /etc/systemd/system/caddy.service.d/override.conf
[Service]
PrivateTmp=
ProtectSystem=
root@ams1:/opt/caddytest#

[linuxgemini]

What is interesting is that it does load files under /var/www its just not able to serve the file browser.

[linuxgemini]

strace output:

[pid 25745] openat(AT_FDCWD, "/var/www", O_RDONLY|O_CLOEXEC) = 8
[pid 25745] epoll_ctl(4, EPOLL_CTL_ADD, 8, {EPOLLIN|EPOLLOUT|EPOLLRDHUP|EPOLLET, {u32=2006040648, u64=140224098320456}}) = -1 EPERM (Operation not permitted)
[pid 25745] getdents64(8, /* 8 entries */, 8192) = 240
[pid 25745] newfstatat(AT_FDCWD, "/var/www/nevergonna", {st_mode=S_IFLNK|0777, st_size=22, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 25745] newfstatat(AT_FDCWD, "/var/www/never", {st_mode=S_IFLNK|0777, st_size=11, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 25745] newfstatat(AT_FDCWD, "/var/www/index.md", {st_mode=S_IFREG|0664, st_size=594, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 25745] newfstatat(AT_FDCWD, "/var/www/index.css", {st_mode=S_IFREG|0664, st_size=223, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 25745] newfstatat(AT_FDCWD, "/var/www/index.html", {st_mode=S_IFREG|0664, st_size=1098, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 25745] newfstatat(AT_FDCWD, "/var/www/robots.txt", {st_mode=S_IFREG|0664, st_size=26, ...}, AT_SYMLINK_NOFOLLOW) = 0
[pid 25745] getdents64(8, /* 0 entries */, 8192) = 0
[pid 25745] newfstatat(AT_FDCWD, ".", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
[pid 25745] newfstatat(AT_FDCWD, "/opt/caddytest", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 0) = 0
[pid 25745] newfstatat(AT_FDCWD, "/var/www/nevergonna", {st_mode=S_IFDIR|0777, st_size=4096, ...}, 0) = 0
[pid 25745] newfstatat(AT_FDCWD, "nevergonna", 0xc000653968, 0) = -1 ENOENT (No such file or directory)
[pid 25745] close(8)                    = 0
[pid 25745] write(7, "HTTP/1.1 404 Not Found\r\nServer: "..., 97) = 97
[pid 25745] read(7, 0xc00067e000, 4096) = -1 EAGAIN (Resource temporarily unavailable)
[pid 25745] epoll_pwait(4, [], 128, 0, NULL, 268843022) = 0

[linuxgemini]

Ah my issue looks like a dupe of #4411 and fix is slated for 2.5.0, sorry for wasting your time!

You can close this issue after verifying that it indeed is a dupe.

[linuxgemini]

Closing, v2.4.7-0.20211206064840-dce81e85d5da h1:aFopTehukizVr7V/pgda0oQmAe3yaHydR2BAWPZQwHU= works.

[francislavoie]

Ah good, thanks for noticing that. I did test on a build from master and not from 2.4.6, so that was my mistake. But I guess I also didn't use a symlink when testing it so I probably would have never realized that was the cause.

Thanks for digging deeper to confirm! Appreciated.