Add benchmark on Pandas DataFrame for Pandas, Polars, DuckDB, chDB

[auxten]

Add benchmark on Pandas DataFrame for Pandas, Polars, DuckDB, chDB

• Pandas: 2.2.2
• Polars: 1.6.0
• DuckDB: 1.0.0
• chDB : 2.0.3

1. Assume data is already in memory, time cost on reading parquet and converting it into Pandas DataFrame are not counted as load time. But as Polars needs to convert Pandas DataFrame into Polars DataFrame to make it work, the convertion time is recorded in the "load_time"
2. Pandas and Polars queries are using their own api to emulate the operation of SQL.
3. During query Polars crashed the Python process several times, I don't know how to make it work. So Q39, Q42 are marked failed

Polars Q39 crash message:

Crash with:
  thread '<unnamed>' panicked at crates/polars-time/src/windows/duration.rs:215:21:
  expected leading integer in the duration string, found m
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
lambda x: x.filter(
    (pl.col("CounterID") == 62)
    & (pl.col("EventDate") >= pl.datetime(2013, 7, 1))
    & (pl.col("EventDate") <= pl.datetime(2013, 7, 31))
    & (pl.col("IsRefresh") == 0)
)
.group_by(
    [
        "TraficSourceID",
        "SearchEngineID",
        "AdvEngineID",
        # pl.when(pl.col("SearchEngineID").eq(0) & pl.col("AdvEngineID").eq(0))
        # .then(pl.col("Referer"))
        # .otherwise("")
        # .alias("Src"),
        "URL",
    ]
)
.agg(pl.len().alias("PageViews"))
.sort("PageViews", descending=True)
.slice(1000, 10),

Polars Q42 crash message:

Crash with:
  thread '<unnamed>' panicked at crates/polars-time/src/windows/duration.rs:215:21:
  expected leading integer in the duration string, found m
lambda x: x.filter(
    (pl.col("CounterID") == 62)
    & (pl.col("EventDate") >= pl.datetime(2013, 7, 14))
    & (pl.col("EventDate") <= pl.datetime(2013, 7, 15))
    & (pl.col("IsRefresh") == 0)
    & (pl.col("DontCountHits") == 0)
)
.group_by(pl.col("EventTime").dt.truncate("minute"))
.agg(pl.len().alias("PageViews"))
.slice(1000, 10),

[auxten]

@rschu1ze @alexey-milovidov Please have a look

[rschu1ze]

I had to use pip install --break-system-packages here (as it complained without the flag). Interestingly, the pip install commands in other ClickBench scripts also don't use that flag, so I am okay with not having it (it might be some weirdness with my system).

[auxten]

I also encountered the same problem and checked how other Python pkgs worked. I think this should be the Ubuntu 24 introduced new problem.
But after that I decided to leave it identical with other packages. Maybe we could fix all of them with another patch.

[rschu1ze]

#223

[rschu1ze]

Double-checked the measurements on a c6a.metal box, and they reproduced nicely (with a few % deviation here and there but that is expected).

[qoega]

Is it meant to be added to the main report? https://github.com/ClickHouse/ClickBench/blob/main/index.html

Or you can have a separate one as we have for hardware and versions.

[auxten]

Is it meant to be added to the main report? https://github.com/ClickHouse/ClickBench/blob/main/index.html

Or you can have a separate one as we have for hardware and versions.

I added a "dataframe" type. You can check it with this link

[qoega]

Polars Q0 and Q29 - did we check the result? Did we check that it is not using query result caching for 29?

[qoega]

What shift does(link)? we just needed +1. +2 etc and not play with indices

[auxten]

You are right, both Pandas and Polars are incorrect. I'll fix them later.

[auxten]

Polars Q0 and Q29 - did we check the result? Did we check that it is not using query result caching for 29?

As I mentioned:

But as Polars needs to convert Pandas DataFrame into Polars DataFrame to make it work, the convertion time is recorded in the "load_time"

For Q0, I think Polars dataframe just keeps some statistic data to make Q0 super fast.