You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
FrostDB requires rows to be sorting for ingestion. To ingest arrow records we sorted them by writing them to a Parquet buffer, and then reading them back out into an arrow record. This reduced a lot of the CPU gains that we had obtained using Arrow record ingestion.
We should remove the conversion from series->parquet->arrow to just be series->arrow and sort that resulting record.
The arrow community seems to be working on a sorting implementation in the library: apache/arrow#34719
The text was updated successfully, but these errors were encountered:
FrostDB requires rows to be sorting for ingestion. To ingest arrow records we sorted them by writing them to a Parquet buffer, and then reading them back out into an arrow record. This reduced a lot of the CPU gains that we had obtained using Arrow record ingestion.
We should remove the conversion from series->parquet->arrow to just be series->arrow and sort that resulting record.
The arrow community seems to be working on a sorting implementation in the library: apache/arrow#34719
The text was updated successfully, but these errors were encountered: