CI: update python versions

[wisp3rwind]

Given that Python 3.11 is released, we've not been testing the most recent versions anymore. Thus, (try to) use semver x-ranges in order to avoid needing manual updates of this in the future.

Don't merge yet, I'm not sure that the range specification for -dev versions is actually valid as I used it here.
Unfortunately, semver intentionally doesn't give a way to specify 3.x-dev to always get the latest development version. Thus, the 3.12 specification will still need to be updated manually.

[wisp3rwind]

Turns out that our 3.11 CI has been failing on Windows already, but nobody noticed because continue-on-error job steps show up as successful...

[sampsyo]

Aha; that's cool! I didn't realize 3.x was an option; this makes lots of sense to me.

It's nice but also worrisome that this turned up so many perplexing-looking filesystem-related errors on Windows… it looks like something having to do with shutting down importer threads properly? That will certainly be fun to debug. Maybe we should suppress this failure (as it was being unintentionally suppressed before) in order to merge this PR and work on that separately?

[wisp3rwind]

It's nice but also worrisome that this turned up so many perplexing-looking filesystem-related errors on Windows… it looks like something having to do with shutting down importer threads properly? That will certainly be fun to debug. Maybe we should suppress this failure (as it was being unintentionally suppressed before) in order to merge this PR and work on that separately?

I'll make an attempt to fix it quickly here; if that doesn't work this sounds like a good strategy!

The failures happen mostly in test shutdown (where the database file seems to unexpectedly remain open), but there's a different failure in the replaygain tests (due to an unexpectly missing threads config value), as well as a failure in PlaylistTestRelativeToLib.test_name_query_with_absolute_paths_in_playlist which is (superficially at least) different.
So, at least part of the issue is related to multi-threaded execution.

One thing that I noticed is that we never explicitly close sqlite3 connections (But leave that up to the GC I presume?). Doing so seems to fix the dbcore migration tests. Unfortunately, this is much harder to fix in general: I attempted to close the connections in dbcore.db.Database._close, but this errors out saying that sqlite3 objects can only be used from the threads that created them. Which would mean we need to ensure that any thread that shuts down closes its connection. I expect that to be quite tricky.

Maybe, someone (...) should sift through the sqlite3 source code changes between 3.10 and 3.11 and see whether any detailed explanation and rational can be found?

[wisp3rwind]

We're not the first to discover this: python/cpython#97641

[wisp3rwind]

From the discussion over there, explicitly closing sqlite3 connections seems to be the right fix; and indeed all the tests are green now. I guess the other test failures were a result of incomplete test cleanup due the exception in test teardown.

[sampsyo]

Wow; this looks like a tricky issue, according to python/cpython#97641 (comment). Nice work tracking this down. It's a relief that it suffices to close everything in our existing self._connections dict!

[wisp3rwind]

Wow; this looks like a tricky issue, according to python/cpython#97641 (comment). Nice work tracking this down.

Well, luckily, I didn't need to track down the Cpython side of this. I could have saved quite some time by immediately searching for this problem in their issue tracker 😆

It's a relief that it suffices to close everything in our existing self._connections dict!

Luckily, the check_same_thread flag exists, so that doing this is in fact viable. Initially, I didn't know about it; and at that point things would indeed have become very tricky