Using Rust and Polars with Postgres

[spirrello]

Research

• I have searched the above polars tags on StackOverflow for similar questions.

• I have asked my usage related question on StackOverflow.

Link to question on StackOverflow

https://stackoverflow.com/questions/73697218/how-to-use-polars-with-rust-and-postgres

Question about polars

Where can I find an easy to follow guide such as the following for Python? There doesn't seem to be anything for Rust.

https://pola-rs.github.io/polars-book/user-guide/howcani/io/postgres.html

[huang12zheng]

• read
  I just want show code for you:

# https://github.com/pola-rs/polars/blob/master/py-polars/polars/io.py#L1018
def read_sql(
# ...
if _WITH_CX:
        tbl = cx.read_sql(
            conn=connection_uri,
            query=sql,
            return_type="arrow2",  //<<<
            partition_on=partition_on,
            partition_range=partition_range,
            partition_num=partition_num,
            protocol=protocol,
        )
        return from_arrow(tbl)

That is to say, you need ConnectorX to import data from DB into DataFrame(arrow) high speed
I'm not sure about that

from_arrow in rust maybe is DataFrame:try_from :
// https://github.com/pola-rs/polars/blob/9991a0dd81c0e71a054b52e5f801579fc260e4c7/polars/polars-core/src/frame/from.rs

• write
  It should be consistent with DOC

[spirrello]

Full disclosure, I'm completely new to Polars and I'm still learning Rust. Perhaps I have this completely wrong but it sounds like it isn't possible to write something similar purely in Rust, correct? https://github.com/pola-rs/polars/blob/master/py-polars/polars/io.py#L1018

…

On Tue, Sep 13, 2022 at 4:28 AM huang12zheng ***@***.***> wrote: - read I just want show code for you: # https://github.com/pola-rs/polars/blob/master/py-polars/polars/io.py#L1018def read_sql(# ...if _WITH_CX: tbl = cx.read_sql( conn=connection_uri, query=sql, return_type="arrow2", //<<< partition_on=partition_on, partition_range=partition_range, partition_num=partition_num, protocol=protocol, ) return from_arrow(tbl) That is to say, you need ConnectorX to import data from DB into DataFrame(arrow) high speed I'm not sure about that from_arrow in rust maybe is DataFrame:try_from : // https://github.com/pola-rs/polars/blob/9991a0dd81c0e71a054b52e5f801579fc260e4c7/polars/polars-core/src/frame/from.rs - write It should be consistent with DOC — Reply to this email directly, view it on GitHub <#4842 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWXQ5CFFSSSQ2IOZEAGBEDV6A3JVANCNFSM6AAAAAAQLAWC3I> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[huang12zheng]

@spirrello
I looked again and found the Connector-X, which is write by rust
https://github.com/sfu-db/connector-x
So I think you can actually do that with Rust.

[huang12zheng]

what's more
https://github.com/sfu-db/connector-x/blob/main/connectorx/tests/test_polars.rs
let df: DataFrame = destination.polars().unwrap();
I think you can solve it directly with ConnectorX

and, some other issue Related to this one:
#3540
#2994

[spirrello]

@huang12zheng Thank you so much for sharing these. I'll look these over today or tomorrow.

[zundertj]

Closing as the question has been answered.