Support libSQL / Turso drivers

[glommer]

libsql is an open contribution fork of SQLite, that includes a server-mode to make it accessible over HTTP. This allows it to be used from serverless environments that don't have a filesystem.

Turso is a commercial offering of that.

Because SQLite is usually not accessible over the network, supporting libSQL/Turso requires drivers changes. However, the query language is still the same.

I am the founder of Turso, and we can definitely help make this happen.

The current driver for Rust is here, and we are in the process of releasing new drivers that implement the rusqlite API. If that makes things easier, waiting until those drivers are released is a great option.

[dezren39]

+1 to supporting libsql in server mode over http. i'd also like to +1 being able to use sqlx when connecting to a file using libsql directly-- like how sqlite is classically used, except with the libsql binary, as that is sometimes useful.

[aashish2057]

+1 I also would love to see this happen been wanting to try libsql and sqlx support is what is holding me back

[abonander]

We don't use the rusqlite API, as explained here: #2656 (comment)

If libSQL implements the SQLite C API then it should theoretically be a drop-in replacement, although we currently set the bundled feature of libsqlite3-sys for the best UX. It'd be really nice if it had the option to link a pre-built SQLite or fall back to the bundled variant. If that sounds like something you guys want to tackle, I think that'd be a great improvement to the current situation and would allow the use of libSQL with the existing driver code.

Alternatively, a bespoke driver built on libsql-client-rs would be better as an async-native solution.

[glommer]

We have a drop-in replacement API, but it doesn't support any of the networking bits that are the ultimate reason behind people wanting this. The better path would be based on libsql-client-rs. Adding @LucioFranco to this thread so he can be notified about everything here

[LucioFranco]

Hi @abonander I came to the same conclusion, I think we want to implement https://github.com/tursodatabase/libsql/tree/main/libsql which should work. Are the current core traits "stable" enough to invest the time to implement our own version or are there plans to make changes that might affect that?

[abonander]

The biggest pending refactor is changing all the trait methods that currently return BoxFuture to use async fn instead, which I'd ideally like to take on in the next major release. That's probably a few months down the road, though.

It looks like libsql-client-rs uses Tokio but remember that we do also support async-std still, so it might take some additional doing to make it work with that runtime. If it still needs to use a Tokio runtime internally, users might be able to accept that, but I'd prefer not to force them to have a Tokio runtime if we can.

I'm also not sure how robust libsql-client-rs is in regards to cancellation safety, that's something that's bitten us a number of times before so I'd recommend searching our issue tracker for issues regarding cancellation safety to see what went wrong and how to avoid it.

[praveenperera]

Whats the status on this? Turso guys need to support async-std and after the async fn refactor this could get in?

[MNThomson]

The lack of SQLx support is what's stopping me from switching to Turso currently. I'm happy to help; is there a defined plan/specific issues for getting libSQL to work with SQLx?

[codegod100]

It's frustrating that libsql isn't compatible with any mainstream database crate yet, so I'm stuck with using the lib raw instead of all the great tooling that's out there.

[abonander]

Since the driver crate refactor, it doesn't strictly need to be merged into this repo to be usable. The required traits just need to be implemented somewhere.

Macro support is the one missing piece. It's intended to be possible to define a new facade like the sqlx crate but with additional drivers, but that's a lot of work and maintenance burden for whoever takes it on.

However, after #3383 I have an idea on how to make the macros and sqlx-cli aware of additional drivers without needing to build a new facade. They could just be cargo installed instead.

[seanaye]

Turso is a great platform but I've had to drop it because of the lack of sqlx support. Every so often I come back to see if theres any updates on this issue. Let me know if I can help in any way to move this forward

[LucioFranco]

We've started some implementation work on this, though, we will likely wait for #3383 to land before we release since this basically will allow us to support all the features.