API explorer tool

[simonw]

The API will be much easier to develop if there's a page that helps you execute JSON POSTs against it.

[simonw]

Host it at /-/api

It's an input box with a path in and a textarea you can put JSON in, plus a submit button to post the request.

It lists the API endpoints you can use - click on a link to populate the form field plus a example.

[simonw]

This is even more important now I have pushed:

• API for bulk inserting records into a table #1866

[simonw]

That initial prototype looks like this:

It currently shows the returned JSON from the API in an alert(). Next I should make that part of the page instead.

[simonw]

Initial prototype API explorer is now live at https://latest-1-0-dev.datasette.io/-/api

[simonw]

Realized the API explorer doesn't need the API key piece at all - it can work with standard cookie-based auth.

This also reflects how most plugins are likely to use this API, where they'll be adding JavaScript that uses fetch() to call the write API directly.

[simonw]

I do need to skip CSRF for these API calls. I'm going to start out by doing that using the skip_csrf() hook to skip CSRF checks on anything with a content-type: application/json request header.

@hookimpl
def skip_csrf(scope):
    if scope["type"] == "http":
        headers = scope.get("headers")
        if dict(headers).get(b'content-type') == b'application/json':
            return True

[simonw]

It's weird that the API explorer only lets you explore POST APIs. It should probably also let you explore GET APIs, or be renamed.

[simonw]

I want JSON syntax highlighting.

https://github.com/luyilin/json-format-highlight is an MIT licensed tiny highlighter that looks decent for this.

https://unpkg.com/[email protected]/dist/json-format-highlight.js

[simonw]

Demo of the latest API explorer:

[simonw]

This is pretty useful now. Two features I still want to add:

• The ability to link to the API explorer such that the form is pre-filled with material from the URL. Need to guard against clickjacking first though, so no-one can link to it in an invisible iframe and trick the user into hitting POST.
• Some kind of list of endpoints so people can click links to start using the API explorer. A list of every table the user can write to with each of their /db/table/-/insert endpoints for example.

[simonw]

Here's a slightly wild idea: what if there was a button on /-/api that you could click to turn on "API explorer mode" for the rest of the Datasette interface - which sets a cookie, and that cookie means you then see "API explorer" links in all sorts of other relevant places in the Datasette UI (maybe tucked away in cog menus).

Only reason I don't want to show these to everyone is that I don't think this is a very user-friendly feature: if you don't know what an API is I don't want to expose you to it unnecessarily.

[simonw]

The API Explorer should definitely link to the /-/create-token page for users who have permission though.

And it should probably go in the Datasette application menu?

[simonw]

I'm going to add a link to the Datasette API docs for the current running version of Datasette, e.g. to https://docs.datasette.io/en/0.63/json_api.html

[simonw]

I can close this issue once I fix it so it no longer hard-codes a potentially invalid example endpoint:

datasette/datasette/templates/api_explorer.html

Lines 24 to 26 in bcc781f

	<label for="path">API path:</label>
	<input type="text" id="path" name="path" value="/data/docs.json" style="width: 60%">
	<input type="submit" value="GET">

datasette/datasette/templates/api_explorer.html

Lines 34 to 35 in bcc781f

	<label for="path">API path:</label>
	<input type="text" id="path" name="path" value="/data/docs/-/insert" style="width: 60%">

[simonw]

I'll probably enhance it a bit more though, I want to provide a UI that lists all the tables you can explore and lets you click to pre-fill the forms with them.

Though at that point what should I do about the other endpoints? Probably list those too. Gets a bit complex, especially with the row-level update and delete endpoints.

[davidbgk]

Realized the API explorer doesn't need the API key piece at all - it can work with standard cookie-based auth.

This also reflects how most plugins are likely to use this API, where they'll be adding JavaScript that uses fetch() to call the write API directly.

I agree (that's what I did with the previous insert plugin), maybe a complete example using fetch() in the documentation would be valuable as a “Getting started with the API” or similar?

[simonw]

The current API explorer uses details/summary elements for the GET and POST dialogs.

I only want one of these to be open at a time, to reflect that you can make either a GET or a POST.

I just noticed that clicking anywhere else on the page closes both elements, which isn't what I want to happen. Turns out that's because of this code I added as part of Datasette's menu implementation!

datasette/datasette/templates/_close_open_menus.html

Lines 2 to 15 in 9f54f00

	document.body.addEventListener('click', (ev) => {
	/* Close any open details elements that this click is outside of */
	var target = ev.target;
	var detailsClickedWithin = null;
	while (target && target.tagName != 'DETAILS') {
	target = target.parentNode;
	}
	if (target && target.tagName == 'DETAILS') {
	detailsClickedWithin = target;
	}
	Array.from(document.getElementsByTagName('details')).filter(
	(details) => details.open && details != detailsClickedWithin
	).forEach(details => details.open = false);
	});

[simonw]

I'm going to add a special no-auto-close class to these and teach that code not to close them.

[simonw]

Actually no, I'm going to add a class of details-menu to the other details elements that SHOULD be closed. That way custom templates using <details> won't close in a surprising way.

[simonw]

I'm going to need extra code to toggle POST closed when GET opens and vice-versa.

[simonw]

Two things left before I close this issue:

• I want to preserve the state of the forms in the URL - probably after a #
• Instead of hard-coding the current examples, I want to provide a list of links which populate the forms

[simonw]

For the example links - I'm going to have these at the bottom of the page so you don't have to scroll past them.

Ideally these would take the user's permissions into account. This could make the page expensive to load, but I'm going to risk it for the moment.

Something like this then:

• data
  • /data/-/create - create table
  • /data/table1/-/insert - insert into table1
  • /data/table1/-/drop - drop table1

I won't bother with per-row demo links (for update and delete) because there could be thousands of them for each table.

[simonw]

In playing with the API explorer just now I realized it's way too easy to accidentally drop a table using it.

[simonw]

Demo: https://latest-1-0-dev.datasette.io/-/api#path=%2Ffixtures%2Ffacetable%2F-%2Fdrop&json=%7B%22confirm%22%3A+true%7D&method=POST