Implement redirects from old % encoding to new dash encoding

[simonw]

One big advantage to this scheme is that redirecting old links to %2F pages (e.g. https://fivethirtyeight.datasettes.com/fivethirtyeight/twitter-ratio%2Fsenators) is easy - if you see a % in the raw_path, redirect to that page with the % replaced by -.

Originally posted by @simonw in #1439 (comment)

[simonw]

This is a bit tricky.

I tried this, sending a redirect only if a 404 happens:

diff --git a/datasette/app.py b/datasette/app.py
index 8c5480c..420664c 100644
--- a/datasette/app.py
+++ b/datasette/app.py
@@ -1211,6 +1211,10 @@ class DatasetteRouter:
         return await self.handle_404(request, send)
 
     async def handle_404(self, request, send, exception=None):
+        # If path contains % encoding, redirect to dash encoding
+        if '%' in request.scope["path"]:
+            await asgi_send_redirect(send, request.scope["path"].replace("%", "-"))
+            return
         # If URL has a trailing slash, redirect to URL without it
         path = request.scope.get(
             "raw_path", request.scope["path"].encode("utf8")

But this URL didn't work:

• http://127.0.0.1:8001/fivethirtyeight/twitter-ratio%2Fsenators

I was expecting that to redirect to this page:

• http://127.0.0.1:8001/fivethirtyeight/twitter-2Dratio-2Fsenators

But instead it took me to another 404:

• http://127.0.0.1:8001/fivethirtyeight/twitter-ratio%2Fsenators

This is because that URL contains both a %-escaped / AND a plain - - which was not escaped in the old system but is escaped in the new system.

[simonw]

This doesn't seem to work.

https://latest.datasette.io/fixtures/table%2Fwith%2Fslashes.csv should be redirecting now that this is deployed - which it is, because https://latest.datasette.io/-/versions shows 644d25d - but I'm not getting that redirect.

[simonw]

Same problem here: https://fivethirtyeight.datasettes.com/fivethirtyeight/ahca-2Dpolls%2Fahca_polls should redirect but doesn't.

[simonw]

The problem seems to be that http://127.0.0.1:8002/fixtures/table%2Fwith%2Fslashes.csv doesn't result in a 404 at all. If it did, it would be redirected.

[simonw]

Here's the problem:

datasette/datasette/utils/__init__.py

Lines 1173 to 1175 in 020effe

	def dash_decode(s: str) -> str:
	"Decodes a dash-encoded string, so ``-2Ffoo-2Fbar`` -> ``/foo/bar``"
	return urllib.parse.unquote(s.replace("-", "%"))

Which is called here:

datasette/datasette/views/base.py

Lines 469 to 473 in 1baa030

	table, _ext_format = await resolve_table_and_format(
	table_and_format=dash_decode(args["table_and_format"]),
	table_exists=async_table_exists,
	allowed_formats=self.ds.renderers.keys(),
	)

So table%2Fwith%2Fslashes ends up decoded as if it was using dash encoding.