Feedback on the first released version

[simonw]

You can download it from here: https://github.com/simonw/datasette-app/releases/tag/0.1.0

Grab the Datasette.app.zip file, unzip it and drag Datasette.app to your /Applications folder to install it.

This is a very early release - the main thing I want to confirm is that people can install it, run it and get a working copy of Datasette running in a window on their computer.

Here's the list of features so far:

• Install this macOS package to get a Datasette application which you can launch from an icon
• Includes full copy of Python which stays separate from any other Python versions you may have installed
• Installs the latest Datasette release the first time it runs
• The application can open existing SQLite database files or read CSV files into an in-memory database
• It can also create a new, empty SQLite database file and create tables in that database by importing CSV data
• By default the server only accepts connections from your computer, but you can use "File -> Access Control -> Anyone on my networks" to make it visible to other computers on your network (or devices on your Tailscale network).
• Datasette plugins can be installed using the "Install Plugin" menu item

Please let me know if it works for you, and let me know what you think of the direction of the app so far.

[wragge]

This is cool! I've successfully installed it. I tried to open a CSV file and received the following error.

However, I just clicked OK and the CSV file was opened as expected.

[simonw]

I've seen that error intermittently too! Haven't tracked down good steps to reproduce yet though.

Could you share the first header row of your file? I'm suspicious it might have to do with strange characters in column names.

[Mjboothaus]

I had this too for this file (once uncompressed) - https://files.consumerfinance.gov/ccdb/complaints.csv.zip

p.s. See this cool US "map" viz - https://www.consumerfinance.gov/data-research/consumer-complaints/search/?dataNormalization=None&dateRange=3y&date_received_max=2021-10-10&date_received_min=2018-10-10&searchField=all&tab=Map

Can we build this as a Datasette-plugin @simonw - even though it is very US-centric -- I would luv to generalise to AUS, UK, Europe and to the "ends of the earth" :)

[Mjboothaus]

Date received,Product,Sub-product,Issue,Sub-issue,Consumer complaint narrative,Company public response,Company,State,ZIP code,Tags,Consumer consent provided?,Submitted via,Date sent to company,Company response to consumer,Timely response?,Consumer disputed?,Complaint ID
2019-06-13,"Credit reporting, credit repair services, or other personal consumer reports",Credit reporting,Incorrect information on your report,Information belongs to someone else,,,CAPITAL ONE FINANCIAL CORPORATION,PA,186XX,,Consent not provided,Web,2019-06-13,Closed with explanation,Yes,N/A,3274605
2019-11-01,Vehicle loan or lease,Loan,Struggling to pay your loan,Denied request to lower payments,"I contacted Ally on Friday XX/XX/XXXX after falling behind on payments due to being out of work for a short period of time due to an illness. I chated with a representative after logging into my account regarding my opitions to ensure I protect my credit and bring my account current.

She advised me that before an extenstion could be done, I had to make a payment in the amount of {$270.00}. I reviewed my finances, as I am playing catch up on all my bills and made this payment on Monday XX/XX/XXXX. This rep advised me, once this payment posts to my account to contact Ally back for an extention or to have a payment deffered to the end of my loan.

With this in mind, I contacted Ally again today and chatted with XXXX. I explained all of the above and the information I was provided when I chatted with the rep last week. She asked several questions and advised me that a one or two month  extension/deffered payment could be done however partial payment is needed! WHAT? She advised me {$230.00} or there abouts would be due within 10 days from me accepting the agreement and then the remaining bal of {$150.00} or there abouts would be due in XX/XX/XXXX. In XX/XX/XXXX, my payments of {$380.00} per month would resume.

If this was the case, I SHOULD HAVE JUST BEEN OFFERED THIS WHEN I JUST MADE MY PAYMENT so that I could catch up on my bills.

[Mjboothaus]

This is the output of head complaints.csv

[psychemedia]

Very nice:-)

mac 10.15.7, no problems:

• opening a legacy sqlite db;
• creating a new db;
• importing CSV into db.

Issues:

• if I install a plugin, my loaded database seems to disappear and I need to reopen it?

Some niggles:

• opening a CSV file into db from the db home page then opens a new window that previews the opened CSV file:
  
  It might be neater to just add the entry to the db home page and make it glow for a few seconds to show it's new?
• trying to open multiple CSV files at once by select multiple files from file chooser: only one of the files (the first?) is opened;
• plugins: it would be handy to be able to select a plugin to install from a list. Gephi desktop used to have a two panel view to select plugins: available and uninstalled plugins in the left hand panel, installed plugins in the right. IIRC, each list had a check box by each item: check items in left and click Install to install, check items in right and click Uninstall to uninstall.
• plugins: how would I install my own plugin, eg from a repo, or in development from a file on my local machine?
• plugins: this is probably a "for the future": VS Code marketplace lets you view extensions on website then open them to install them in VS Code on desktop; Github website repos let you "Open in Github Desktop"; so maybe a "install in Datasette desktop" option on plugins website?
• plugin installed confirmation dialog - maybe name which plugin was installed?
  
• file menu: open database - open recent list might be useful?
• plugins: no option to uninstall plugin? (This raised a related issue for me that is more in scope of datasette-cluster-map. If I have that plugin installed, how can I plot markers explicitly (not clustered?) other than by going in to a high zoom level on the map?
• plugins: no way to edit the configuration of a particular plugin?

[simonw]

This is fantastic feedback, thank you!

[simonw]

I'm starting to file tickets:

• Ability to uninstall a plugin #72 "ability to uninstall a plugin"
• Be smarter about remembering which files you were working on after an app restart #54 "Integrate with recent files menu" already exists
• "Plugin installed" message box should say which plugin was installed #73 "Plugin installed" message box should say which plugin was installed

[simonw]

• Documentation: How to develop your own plugins #75

[jamesking]

Oops. Sorry to be that person, but I have managed to break it.

I installed the datasett-jq plugin (or attempted to - I am not sure if the install worked) I ran into the same issue as @psychemedia:

if I install a plugin, my loaded database seems to disappear and I need to reopen it?

I then quit and restarted the app and am now faced with a blue spinner pictured above indefinitely.

[simonw]

I tried a fresh install of Datasette Desktop (with the most recent changes) and installed datasette-jq on my own machine and it worked. My best guess is that's because I have clang and various build tools installed, which it managed to use to compile the dependencies.

That's not much good for other people though!

May need to lobby the pyjq maintainer to publish pre-compiled wheels for macOS - or even publish them myself under a different name on PyPI.

[psychemedia]

Or at a push maintain your own wheelhouse and add a path to also look to that to install packages?

[simonw]

That's a great idea. Would be really useful to figure out how to use my Apple Developer signing keys for Python wheels.

[simonw]

Opened a research issue here:

• Possible to create wheels for things like pyjq and sign them for distribution? #96

[jamesking]

Thanks for looking into this!

[llimllib]

I tested on an m1 mac, and everything seemed to work fine. I saw two opportunities for improvement:

• Dragging a csv onto datasette ought to open up the csv. The app does show the "plus" cursor indicating that it's a drop target, but nothing happened when I dropped the CSV.
• The app doesn't register itself as a csv handler, or something like that, because Mac doesn't show it as one of the options for opening a CSV when you right click -> open with -> other, if you have recommended applications selected (which is the default):
  

[simonw]

Good news: I almost have opening CSV/SQLite in the application working! #40

[llimllib]

off topic but: your notes to yourself are inspiring, keep up the good work

[llimllib]

I went to plugins -> install plugin -> datasette-jq, hit "ok", and got returned to the home page. Tried clicking on the "plugins" link, and the datasette window went plain white. Seems to be mostly unresponsive, though plugins -> install plugin still pops a window that works. ⌘-R (or clicking "refresh") does not change anything I can tell.

I closed and re-opened the app, and looks like I'm stuck with an infinite loading spinner.

I opened console.app, started up datasette, and here is the log transcript. It definitely looks like some security errors on lines like:

default	22:37:06.746592-0400	kernel	Sandbox: 2601 duplicate reports for Datasette Helper deny(1) sysctl-read sysctl.proc_translated
error	22:37:08.553422-0400	kernel	Sandbox: Datasette Helper(2492) deny(1) sysctl-read sysctl.proc_translated

[jamesking]

@llimllib This might be because of an open issue with datasette-jq rather than a problem with the Desktop app. I had the same experience when trying to install the datasette-jq plugin. simonw/datasette-jq#2. See this thread above

[jamesking]

Really like this project @simonw!

First impressions of using Datasette Desktop:

I had my eye on Datasette for a while, but the Desktop app finally lowered the friction enough for my to try it out. I then also got the cli/web version installed and working too.

I have been using it to explore some product analytics data my company has been collecting. Datasette was immediately useful in exploring this data and making sense of it.

I'm now planning to set up some kind of CI process (Github actions?) where we pull the latest analytics data each day and republish a hosted Datasette instance to a subdomain of our site behind a login (perhaps using datasette-auth-existing-cookies).

Small issues/feedback:

• Like @psychemedia I was also confused when the app opened another window after I imported a CSV. Feels like it would be better to keep the original window open and allow me to click on the newly created table to view it.
• The datasette-vega plugin is really useful, but I found myself wanting to make the visualisations larger.
  • I was initially confused whether the visualisation was representing all the rows in the current scope of my table, or just the ones that were visible on the current page.
• Datasette forgets which database I was viewing when I last opened it. A "Recent databases" list in the File menu would be great.
• Plugins are installed globally, but I would probably want a different set of plugins installed for each project. Perhaps this setup could be saved as metadata somehow?
• Being a desktop app has its benefits, but one of the things I really like about the web-version is that everything has a URL, so if I want to bookmark a particular view, including a visualisation, I can do so. Perhaps a way of bookmarking or saving the current view?

[simonw]

Bookmarking is definitely something I want, issue here:

• Some kind of bookmarking/history mechanism #85

Recent files is now out in release 0.2.0!

That's really useful feedback on datasette-vega - that's one of the oldest plugins now and could definitely do with an overhaul. Clarifying what is being rendering (and providing an option to render all rows) would be a great start!

[jamesking]

Great! Thanks for the update!

[Mjboothaus]

Maybe under Navigate menu the ability "arrange" windows nicely - e.g. functionality equivalent (or similar) to https://rectangleapp.com for Datasette window management.