request.url and request.scheme should obey force_https_urls config setting

[simonw]

I'm trying to get the https://www.niche-museums.com/browse/feed.atom feed to validate and I git this from https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom

This feed is valid, but interoperability with the widest range of feed readers could be improved by implementing the following recommendations.

line 6, column 73: Self reference doesn't match document location [help]

<link href="http://www.niche-museums.com/browse/feed.atom" rel="self"/>

I tried to fix this using force_https_urls (commit) but it didn't work - because that setting isn't respected by the Request class:

datasette/datasette/utils/asgi.py

Lines 15 to 32 in 40885ef

	class Request:
	def __init__(self, scope, receive):
	self.scope = scope
	self.receive = receive
	@property
	def method(self):
	return self.scope["method"]
	@property
	def url(self):
	return urlunparse(
	(self.scheme, self.host, self.path, None, self.query_string, None)
	)
	@property
	def scheme(self):
	return self.scope.get("scheme") or "http"

[simonw]

I think the right way to fix this is to modify the scope["scheme"] ASGI key before the Request object is constructed - otherwise that Request class will have to gain knowledge of Datasette's configuration mechanism.

[simonw]

I'm inclined to do this at the earliest possible moment. I think that's probably in the DatasetteRouter class, which should see every scope first and be able to apply that setting to it. It already has some special case logic to deal with the base_url setting here:

datasette/datasette/app.py

Lines 779 to 789 in 40885ef

	class DatasetteRouter(AsgiRouter):
	def __init__(self, datasette, routes):
	self.ds = datasette
	super().__init__(routes)
	async def route_path(self, scope, receive, send, path):
	# Strip off base_url if present before routing
	base_url = self.ds.config("base_url")
	if base_url != "/" and path.startswith(base_url):
	path = "/" + path[len(base_url) :]
	return await super().route_path(scope, receive, send, path)

[simonw]

https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fwww.niche-museums.com%2Fbrowse%2Ffeed.atom validates now!