Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add proxy connection type #1881

Merged
merged 54 commits into from
Nov 19, 2023
Merged

feat: Add proxy connection type #1881

merged 54 commits into from
Nov 19, 2023

Conversation

langyo
Copy link
Contributor

@langyo langyo commented Sep 27, 2023

PR Info

New Features

  • Add the proxy connection type.

Changes

  • Add a new feature called proxy.
  • GIve a demo to help us create a proxy connection quickly.

@langyo langyo marked this pull request as ready for review October 9, 2023 09:13
@langyo
Copy link
Contributor Author

langyo commented Oct 9, 2023

Although the test samples and example code are not rich enough, I think it is time to start reviewing the code.

If it can be merged, I'll do some additional work, such as adding CI configuration and writing more examples.

@langyo
Copy link
Contributor Author

langyo commented Oct 9, 2023

cc @tyt2y3 😉

uuid = { version = "1", default-features = false, optional = true }
ouroboros = { version = "0.17", default-features = false }
url = { version = "2.2", default-features = false }
thiserror = { version = "1", default-features = false }

[target.'cfg(not(wasm))'.dependencies]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This position temporarily disables sqlx because rustls, the upstream dependency of sqlx, cannot compile to any wasm target for the time being. The mainly cause is that rustls-dependent rings need to cross-compile code from C language.

Recently, ring may support compilation of the wasm platform by compiling old cipher suites with BoringC instead, but this will take a long time.

Copy link
Contributor Author

@langyo langyo Oct 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@langyo
Copy link
Contributor Author

langyo commented Oct 23, 2023

Now it's ready for review.

Here are three examples that utilize a proxy connection.

The example that utilizes SurrealDB might be able to enhance its robustness by employing the SQL parser, but it doesn't actually matter.

The other example, which uses Wasmtime, relies on the unstable WASI preview support, so its ABI may change in the future.

The third example, which employs GlueSQL, is currently using the legacy version (0.13) to avoid conflicts with the chrono library. Once the conflict is resolved, we can upgrade to the latest version (0.14).

@tyt2y3 😉

@langyo
Copy link
Contributor Author

langyo commented Oct 28, 2023

I've written a demo for my own project that uses this fork branch, take a look.

@tyt2y3
Copy link
Member

tyt2y3 commented Nov 8, 2023

That's very interesting indeed. Can you explain how to implement a ProxyDb?

celestia-island/tairitsu@f819768#diff-0f845934140e261f65d6abe821a10b9c11c6f84b68d9ab05aa3e81325dfa2fbb

And indeed how does GlueSQL come into play? How many features of SeaORM is usuable with GlueSQL?

I think it'd be better if we include an example implementation of ProxyDb as well.

@langyo
Copy link
Contributor Author

langyo commented Nov 9, 2023

That's very interesting indeed. Can you explain how to implement a ProxyDb?

celestia-island/tairitsu@f819768#diff-0f845934140e261f65d6abe821a10b9c11c6f84b68d9ab05aa3e81325dfa2fbb

And indeed how does GlueSQL come into play? How many features of SeaORM is usuable with GlueSQL?

I think it'd be better if we include an example implementation of ProxyDb as well.

The best practices for this are all included in the examples I attached to this fork:

https://github.com/langyo/sea-orm/tree/proxy-conn/examples/proxy_gluesql_example

https://github.com/langyo/sea-orm/tree/proxy-conn/examples/proxy_surrealdb_example

https://github.com/langyo/sea-orm/tree/proxy-conn/examples/proxy_wasmtime_example

It was already done two weeks ago...

To implement a basic proxy database, you only need to prepare a class that implements sea_orm::ProxyDatabaseTrait, and at least implement the query and execute functions. When creating a database later, use the sea_orm::Database::connect_proxy function instead of sea_orm::Database::connect function creation.

By the way, don’t forget to enable the proxy feature.

Here is an example:

struct ProxyDb {}

impl ProxyDatabaseTrait for ProxyDb {
    fn query(&self, statement: Statement) -> Result<Vec<ProxyRow>, DbErr> {
        // ...

        Ok(vec![])
    }

    fn execute(&self, statement: Statement) -> Result<ProxyExecResult, DbErr> {
        // ...

        Ok(ProxyExecResult {
            last_insert_id: 1,
            rows_affected: 1,
        })
    }
}

fn main() -> anyhow::Result<()> {
    let db = Database::connect_proxy(
        DbBackend::Sqlite,
        Arc::new(Mutex::new(Box::new(ProxyDb { }))),
    )
    .await?;

    Ok(())
}

For database engines that are not adapted to the protocols of MySQL, PostgreSQL and Sqlite (such as GlueSQL and SurrealDB), it is necessary to manually re-parse and construct SQL statements. After all, the SQL statements produced by SQLX cannot be used directly...

I am actually planning to introduce a new mechanism to be able to choose other database back-end engines besides SQLX, which includes constructing one on site, but this obviously should be solved by opening another PR.

So...I would do this first. I will implement the feature of the proxy database first, and I will open a new issue to implement the rest later. The current examples for SurrealDB and GlueSQL focus on demonstrating the capabilities of using proxy databases, rather than putting them into production immediately.

@tyt2y3
Copy link
Member

tyt2y3 commented Nov 15, 2023

Thank you for your explanation! I will read again and then probably merge it as is, if you wouldn't mind becoming the PIC of this feature. TBH I am not 100% clear of your big picture, but I am sure you are making something interesting and great! Let's see how it goes.

@langyo
Copy link
Contributor Author

langyo commented Nov 16, 2023

Thank you for your explanation! I will read again and then probably merge it as is, if you wouldn't mind becoming the PIC of this feature. TBH I am not 100% clear of your big picture, but I am sure you are making something interesting and great! Let's see how it goes.

Certainly! I appreciate your trust, and I'm glad to continue maintaining the feature if I have free time.

@tyt2y3 tyt2y3 changed the base branch from master to proxy November 19, 2023 12:49
@tyt2y3 tyt2y3 merged commit 2844d03 into SeaQL:proxy Nov 19, 2023
34 checks passed
@tyt2y3
Copy link
Member

tyt2y3 commented Nov 19, 2023

I will make a few edits on the proxy branch

Copy link

🎉 Released In 0.12.10 🎉

Thank you everyone for the contribution!
This feature is now available in the latest release. Now is a good time to upgrade!
Your participation is what makes us unique; your adoption is what drives us forward.
You can support SeaQL 🌊 by starring our repos, sharing our libraries and becoming a sponsor ⭐.

maxdeviant referenced this pull request in zed-industries/zed Jul 27, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [sea-orm](https://www.sea-ql.org/SeaORM)
([source](https://togithub.com/SeaQL/sea-orm)) | dev-dependencies |
patch | `0.12.3` -> `0.12.15` |
| [sea-orm](https://www.sea-ql.org/SeaORM)
([source](https://togithub.com/SeaQL/sea-orm)) | dependencies | patch |
`0.12.3` -> `0.12.15` |

---

### Release Notes

<details>
<summary>SeaQL/sea-orm (sea-orm)</summary>

###
[`v0.12.15`](https://togithub.com/SeaQL/sea-orm/releases/tag/0.12.15)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.14...0.12.15)

##### Enhancements

- `DerivePartialModel` macro attribute `entity` now supports `syn::Type`
[https://github.com/SeaQL/sea-orm/pull/2137](https://togithub.com/SeaQL/sea-orm/pull/2137)

```rust

#[derive(DerivePartialModel)]
#[sea_orm(entity = "<entity::Model as ModelTrait>::Entity")]
struct EntityNameNotAIdent {
    #[sea_orm(from_col = "foo2")]
    _foo: i32,
    #[sea_orm(from_col = "bar2")]
    _bar: String,
}
```

- Added `RelationDef::from_alias()`
[https://github.com/SeaQL/sea-orm/pull/2146](https://togithub.com/SeaQL/sea-orm/pull/2146)

```rust
assert_eq!(
    cake::Entity::find()
        .join_as(
            JoinType::LeftJoin,
            cake_filling::Relation::Cake.def().rev(),
            cf.clone()
        )
        .join(
            JoinType::LeftJoin,
            cake_filling::Relation::Filling.def().from_alias(cf)
        )
        .build(DbBackend::MySql)
        .to_string(),
    [
        "SELECT `cake`.`id`, `cake`.`name` FROM `cake`",
        "LEFT JOIN `cake_filling` AS `cf` ON `cake`.`id` = `cf`.`cake_id`",
        "LEFT JOIN `filling` ON `cf`.`filling_id` = `filling`.`id`",
    ]
    .join(" ")
);
```

###
[`v0.12.14`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#01214---2024-02-05)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.12...0.12.14)

- Added feature flag `sqlite-use-returning-for-3_35` to use SQLite's
returning
[https://github.com/SeaQL/sea-orm/pull/2070](https://togithub.com/SeaQL/sea-orm/pull/2070)
- Added Loco example
[https://github.com/SeaQL/sea-orm/pull/2092](https://togithub.com/SeaQL/sea-orm/pull/2092)

###
[`v0.12.12`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#01212---2024-01-22)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.11...0.12.12)

##### Bug Fixes

- \[sea-orm-cli] Fix entity generation for non-alphanumeric enum
variants[https://github.com/SeaQL/sea-orm/pull/1821](https://togithub.com/SeaQL/sea-orm/pull/1821)1
- \[sea-orm-cli] Fix entity generation for relations with composite
keys[https://github.com/SeaQL/sea-orm/pull/2071](https://togithub.com/SeaQL/sea-orm/pull/2071)1

##### Enhancements

-   Added `ConnectOptions::test_before_acquire`

###
[`v0.12.11`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#01211---2024-01-14)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.10...0.12.11)

##### New Features

- Added `desc` to `Cursor` paginator
[https://github.com/SeaQL/sea-orm/pull/2037](https://togithub.com/SeaQL/sea-orm/pull/2037)

##### Enhancements

- Improve query performance of `Paginator`'s `COUNT` query
[https://github.com/SeaQL/sea-orm/pull/2030](https://togithub.com/SeaQL/sea-orm/pull/2030)
- Added SQLx slow statements logging to `ConnectOptions`
[https://github.com/SeaQL/sea-orm/pull/2055](https://togithub.com/SeaQL/sea-orm/pull/2055)
- Added `QuerySelect::lock_with_behavior`
[https://github.com/SeaQL/sea-orm/pull/1867](https://togithub.com/SeaQL/sea-orm/pull/1867)

##### Bug Fixes

- \[sea-orm-macro] Qualify types in `DeriveValueType` macro
[https://github.com/SeaQL/sea-orm/pull/2054](https://togithub.com/SeaQL/sea-orm/pull/2054)

##### House keeping

- Fix clippy warnings on 1.75
[https://github.com/SeaQL/sea-orm/pull/2057](https://togithub.com/SeaQL/sea-orm/pull/2057)

###
[`v0.12.10`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#01210---2023-12-14)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.9...0.12.10)

##### New Features

- \[sea-orm-macro] Comment attribute for Entity (`#[sea_orm(comment =
"action")]`); `create_table_from_entity` supports comment
[https://github.com/SeaQL/sea-orm/pull/2009](https://togithub.com/SeaQL/sea-orm/pull/2009)
- Added "proxy" (feature flag `proxy`) to database backend
[https://github.com/SeaQL/sea-orm/pull/1881](https://togithub.com/SeaQL/sea-orm/pull/1881),
[https://github.com/SeaQL/sea-orm/pull/2000](https://togithub.com/SeaQL/sea-orm/pull/2000)

##### Enhancements

- Cast enums in `is_in` and `is_not_in`
[https://github.com/SeaQL/sea-orm/pull/2002](https://togithub.com/SeaQL/sea-orm/pull/2002)

##### Upgrades

- Updated `sea-query` to `0.30.5`
https://github.com/SeaQL/sea-query/releases/tag/0.30.5

###
[`v0.12.9`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#0129---2023-12-08)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.8...0.12.9)

##### Enhancements

- Add source annotations to errors
[https://github.com/SeaQL/sea-orm/pull/1999](https://togithub.com/SeaQL/sea-orm/pull/1999)

##### Upgrades

- Updated `sea-query` to `0.30.4`
https://github.com/SeaQL/sea-query/releases/tag/0.30.4

###
[`v0.12.8`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#0128---2023-12-04)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.7...0.12.8)

##### Enhancements

- Implement `StatementBuilder` for `sea_query::WithQuery`
[https://github.com/SeaQL/sea-orm/issues/1960](https://togithub.com/SeaQL/sea-orm/issues/1960)

##### Upgrades

- Upgrade `axum` example to `0.7`
[https://github.com/SeaQL/sea-orm/pull/1984](https://togithub.com/SeaQL/sea-orm/pull/1984)

###
[`v0.12.7`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#0127---2023-11-22)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.6...0.12.7)

##### Enhancements

- Added method `expr_as_` that accepts `self`
[https://github.com/SeaQL/sea-orm/pull/1979](https://togithub.com/SeaQL/sea-orm/pull/1979)

##### Upgrades

- Updated `sea-query` to `0.30.3`
https://github.com/SeaQL/sea-query/releases/tag/0.30.3

###
[`v0.12.6`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#0126---2023-11-13)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.5...0.12.6)

##### New Features

- Added `#[sea_orm(skip)]` for `FromQueryResult` derive macro
[https://github.com/SeaQL/sea-orm/pull/1954](https://togithub.com/SeaQL/sea-orm/pull/1954)

###
[`v0.12.5`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#0125---2023-11-12)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.4...0.12.5)

##### Bug Fixes

- \[sea-orm-cli] Fix duplicated active enum use statements on generated
entities[https://github.com/SeaQL/sea-orm/pull/1953](https://togithub.com/SeaQL/sea-orm/pull/1953)3
- \[sea-orm-cli] Added `--enum-extra-derives`
[https://github.com/SeaQL/sea-orm/pull/1934](https://togithub.com/SeaQL/sea-orm/pull/1934)
- \[sea-orm-cli] Added `--enum-extra-attributes`
[https://github.com/SeaQL/sea-orm/pull/1952](https://togithub.com/SeaQL/sea-orm/pull/1952)

###
[`v0.12.4`](https://togithub.com/SeaQL/sea-orm/blob/HEAD/CHANGELOG.md#0124---2023-10-19)

[Compare
Source](https://togithub.com/SeaQL/sea-orm/compare/0.12.3...0.12.4)

##### New Features

- Add support for root JSON arrays
[https://github.com/SeaQL/sea-orm/pull/1898](https://togithub.com/SeaQL/sea-orm/pull/1898)
Now the following works (requires the `json-array` / `postgres-array`
feature)!

```rust

#[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)]
#[sea_orm(table_name = "json_struct_vec")]
pub struct Model {
    #[sea_orm(primary_key)]
    pub id: i32,
    #[sea_orm(column_type = "Json")]
    pub struct_vec: Vec<JsonColumn>,
}

#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, FromJsonQueryResult)]
pub struct JsonColumn {
    pub value: String,
}
```

##### Enhancements

- Loader: use `ValueTuple` as hash key
[https://github.com/SeaQL/sea-orm/pull/1868](https://togithub.com/SeaQL/sea-orm/pull/1868)

##### Upgrades

- Updated `sea-query` to `0.30.2`
https://github.com/SeaQL/sea-query/releases/tag/0.30.2

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "after 3pm on Wednesday" in timezone
America/New_York, Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these
updates again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

Release Notes:

- N/A

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MzguMCIsInVwZGF0ZWRJblZlciI6IjM3LjQzOC4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants