Plugin hook: filters_from_request

[simonw]

I meant to add this as part of the facets plugin mechanism but didn't quite get to it. Original idea was to allow plugins to register extra filters, as seen in datasette/filters.py:

datasette/datasette/filters.py

Lines 83 to 98 in 2600858

	class Filters:
	_filters = (
	[
	# key, display, sql_template, human_template, format=, numeric=, no_argument=
	TemplatedFilter(
	"exact",
	"=",
	'"{c}" = :{p}',
	lambda c, v: "{c} = {v}" if v.isdigit() else '{c} = "{v}"',
	),
	TemplatedFilter(
	"not",
	"!=",
	'"{c}" != :{p}',
	lambda c, v: "{c} != {v}" if v.isdigit() else '{c} != "{v}"',
	),

[simonw]

This expands on the refactoring work from 6da567d

[simonw]

It would be nice if this plugin was passed the current database/table so it can decide to enable new filters only for specific tables. This will require a bit of refactoring because the filters list is static at the moment - it would instead have to be returned by a function that runs when the table view is rendered.

[simonw]

I wonder if this is the right hook?

The more likely case is that we need a hook that registers a new type of lookup entirely - ?_spatial_within={geojson} for example. These aren't necessarily attached to a specific column and may have entirely custom syntax - but they still need to be able to influence the query (like the _where= clause does) and the human-readable description of the page.

Is there a strong case for supporting both custom filter plugins AND custom table where plugins, or could those where plugins cover both bases?

[simonw]

I'm having trouble coming up with interesting column-based filters which don't make sense to ship as default behaviour.

[simonw]

Assuming I do go ahead with this plugin hook, the existing InFilter makes for a nice simple example that illustrates the two key methods: .where_clause() and .human_clause():

class InFilter(Filter):
    key = "in"
    display = "in"

    def split_value(self, value):
        if value.startswith("["):
            return json.loads(value)
        else:
            return [v.strip() for v in value.split(",")]

    def where_clause(self, table, column, value, param_counter):
        values = self.split_value(value)
        params = [":p{}".format(param_counter + i) for i in range(len(values))]
        sql = "{} in ({})".format(escape_sqlite(column), ", ".join(params))
        return sql, values

    def human_clause(self, column, value):
        return "{} in {}".format(column, json.dumps(self.split_value(value)))

[simonw]

Here's an older version of what that custom table filtering plugin might look like: 5116c4e

from datasette.utils import TableFilter

@hookimpl
def table_filter():
    async def inner(view, name, table, request):
        extra_human_descriptions = []
        where_clauses = []
        params = {}
        # ... build those things here
        return TableFilter(
            human_description_extras=extra_human_descriptions,
            where_clauses=where_clauses,
            params=params,
        )
    return inner

I built this for the https://github.com/simonw/russian-ira-facebook-ads-datasette project.

It's pretty neat. Maybe I should go with that?

[simonw]

I'm leaning towards supporting both hooks.

[simonw]

I revisited this idea in #1518 and came up with a slightly different name and design for the hook:

@hookspec
def filters_from_request(request, database, table, datasette):
    """
    Return FilterArguments(
        where_clauses=[str, str, str],
        params={},
        human_descriptions=[str, str, str],
        extra_context={}
    ) based on the request"""

[simonw]

This filter design can only influence the where component of the SQL clause - it's not able to modify the SELECT columns or adjust the ORDER BY or OFFSET LIMIT parts. I think that's OK.

[simonw]

Documentation for that hook in the PR branch: https://github.com/simonw/datasette/blob/54e9b3972f277431a001e685f78e5dd6403a6d8d/docs/plugin_hooks.rst#filters_from_requestrequest-database-table-datasette

[simonw]

I could use this hook to add table filtering on a map to the existing datasette-leaflet-freedraw plugin.

[simonw]

The one slightly weird thing about this hook is how it adds extra_context without an obvious way for plugins to add extra HTML to the templates based on that context.

Maybe I need the proposed mechanism from

• Ability for plugins to collaborate when adding extra HTML to blocks in default templates #1191

Which has an in-progress PR:

• WIP: Plugin includes #1204

[simonw]

I'm happy with how the prototype that used this plugin in datasette-leaflet-freedraw turned out: https://github.com/simonw/datasette-leaflet-freedraw/blob/e8a16a0fe90656b8d655c02881d23a2b9833281d/datasette_leaflet_freedraw/__init__.py