CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

vortex-datafusion - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)
• specify the storage_dtype in ExtDType (#1007)

Other

• Filter pushdown over layouts (#1124)

vortex-serde - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)
• store min, max, null count, and true count in column metadata (#1164)

Other

• Filter pushdown over layouts (#1124)

vortex-zigzag - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

vortex-sampling-compressor - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

vortex-runend-bool - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

vortex-runend - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

vortex-roaring - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

Other

• deny missing_docs on vortex-dtype (#1182)

vortex-fsst - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

Other

• split CI into separate moldy jobs (#1192)

vortex-expr - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

Other

• Filter pushdown over layouts (#1124)

vortex-dict - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

vortex-datetime-parts - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)
• specify the storage_dtype in ExtDType (#1007)

vortex-bytebool - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

vortex-fastlanes - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

vortex-scalar - 0.14.0 - 2024-11-04

Added

• specify the storage_dtype in ExtDType (#1007)
• store min, max, null count, and true count in column metadata (#1164)

vortex-proto - 0.14.0 - 2024-11-04

Added

• specify the storage_dtype in ExtDType (#1007)

vortex-flatbuffers - 0.14.0 - 2024-11-04

Added

• specify the storage_dtype in ExtDType (#1007)

Other

• Filter pushdown over layouts (#1124)

vortex-error - 0.14.0 - 2024-11-04

Added

• docs for vortex-error crate (#1184)

vortex-dtype - 0.14.0 - 2024-11-04

Added

• specify the storage_dtype in ExtDType (#1007)

Other

• deny missing_docs on vortex-dtype (#1182)

vortex-datetime-dtype - 0.14.0 - 2024-11-04

Added

• specify the storage_dtype in ExtDType (#1007)

Other

• improve datetime-dtype unit tests (#1183)

vortex-buffer - 0.14.0 - 2024-11-04

Added

• store min, max, null count, and true count in column metadata (#1164)

vortex-array - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)
• specify the storage_dtype in ExtDType (#1007)
• store min, max, null count, and true count in column metadata (#1164)

Other

• deny missing_docs on vortex-dtype (#1182)

vortex-alp - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

vortex-all - 0.14.0 - 2024-11-04

Added

• it's a bird, it's a plane, it's vortex-all! (#1140)

Fixed

• update setup instructions (rye -> uv) (#1176)
• fix docs badge in readme (#753)

Other

• deny missing_docs on vortex-dtype (#1182)
• very small README.md fixes
• More README.md improvements (#1084)
• Update README.md (#1055)
• minor addition to README (#1030)
• updated README (#876)
• release to Test PyPI on each push to version tags (#760)
• Run ETE benchmarks with MiMalloc and leave a note encouraging its usage (#399)
• README updates (#394)
• Download flatc instead of building it from source (#374)
• Update README.md (#337)
• IPC Prototype (#181)
• Add note to readme about git submodules and zig version (#176)
• acknowledgments (#171)
• Update README.md (#168)
• More README updates (#140)
• Update README.md
• readme improvements (#137)
• README (#102)
• Root project is vortex-array (#67)
• Add minimal description to readme and fixup cargo metadata (#30)
• Add Readme

vortex-serde - 0.13.1 - 2024-10-31

Fixed

• specify features in vortex-serde & test default features (#1168)

vortex-zigzag - 0.13.0 - 2024-10-29

Added

• trim metadatas (part 2) (#1028)
• teach *Metadata and ScalarValue to Display (#975)

Other

• Even more docs (#1105)

vortex-runend-bool - 0.13.0 - 2024-10-29

Added

• teach *Metadata and ScalarValue to Display (#975)

Other

• Even more docs (#1105)
• Faster RunEndBool decompression, plus metadata cleanup (#981)

vortex-runend - 0.13.0 - 2024-10-29

Added

• trim metadatas (part 2) (#1028)
• teach *Metadata and ScalarValue to Display (#975)

Other

• Even more docs (#1105)
• RunEnd ends array max is length of the array (#1017)
• Clean up runend metadata & stats (#1011)

vortex-roaring - 0.13.0 - 2024-10-29

Added

• use hashbrown::hashmap everywhere (modestly faster decompress) (#1160)
• teach *Metadata and ScalarValue to Display (#975)

Fixed

• RoaringInt can_compress was erroneously always None (#1004)

Other

• Even more docs (#1105)
• RoaringBool bitmaps needs to manually append trailing false values when canonicalizing (#988)

vortex-fsst - 0.13.0 - 2024-10-29

Added

• FSSTArray::into_canonical directly build VarBinView (#1161)
• German strings, attempt 3 (#1082)
• trim metadatas (part 2) (#1028)
• teach *Metadata and ScalarValue to Display (#975)

Other

• Even more docs (#1105)

vortex-dict - 0.13.0 - 2024-10-29

Added

• use hashbrown::hashmap everywhere (modestly faster decompress) (#1160)
• specialized IntoCanonical for DictArray utf8/binary (#1146)
• German strings, attempt 3 (#1082)
• increase dict decompression throughput (#1032)
• trim metadatas (part 2) (#1028)
• faster Dict logical validity (#1034)
• improved objective function in sampling compressor (#1000)
• teach *Metadata and ScalarValue to Display (#975)

Other

• Even more docs (#1105)
• Implement filter for dict (#1099)
• Use foldhash in dict encoding (#980)

vortex-datetime-parts - 0.13.0 - 2024-10-29

Added

• teach *Metadata and ScalarValue to Display (#975)

Other

• Even more docs (#1105)

vortex-sampling-compressor - 0.13.0 - 2024-10-29

Added

• use hashbrown::hashmap everywhere (modestly faster decompress) (#1160)
• German strings, attempt 3 (#1082)
• vortex.dataset.Dataset: deep integration with Polars & DuckDB (#1089)
• trim metadatas (part 2) (#1028)
• add ChunkedCompressor which compresses chunk n+1 like chunk n (#996)
• improved objective function in sampling compressor (#1000)

Fixed

• dict compressor supports varbinview (#1118)
• disable roaring compressors (#1076)
• Vortex (de)compress benchmarks read/write a Layout (#1024)
• RoaringInt can_compress was erroneously always None (#1004)

Other

• More docs (#1104)
• Trim BitPackedMetadata to only required values (#1046)

vortex-serde - 0.13.0 - 2024-10-29

Added

• use hashbrown::hashmap everywhere (modestly faster decompress) (#1160)
• teach PyVortex to read from object storage (attempt two) (#1151)
• faster take for BitPackedArray and SparseArray (#1133)
• German strings, attempt 3 (#1082)
• vortex.dataset.Dataset: deep integration with Polars & DuckDB (#1089)
• Unify FlatLayout with other layouts and add metadata to their flatbuffer representation (#1077)
• improved objective function in sampling compressor (#1000)

Fixed

• RelativeLayoutCache must hold disk dtype (#1051)
• from_fields is fallible (#1054)

Other

• VortexRecordBatchReader is generic over runtime (#1120)
• Layouts have buffers and there's self describing schema layout (#1098)
• limit number of single char variables to 2 (#1066)
• Error when projecting projected dtype (#1063)
• Delegate dtype deserialization and projection to layouts (#1060)
• Correctly resolve filter column references when reading column layouts (#1058)
• Add Select expression and reorganize vortex-expr crate (#1049)
• make write logic slightly more simple (#1026)
• Fix writing of Chunked Struct arrays (#1020)

vortex-expr - 0.13.0 - 2024-10-29

Added

• use hashbrown::hashmap everywhere (modestly faster decompress) (#1160)

Other

• Add Select expression and reorganize vortex-expr crate (#1049)

vortex-datafusion - 0.13.0 - 2024-10-29

Added

• German strings, attempt 3 (#1082)
• teach *Metadata and ScalarValue to Display (#975)

Fixed

• from_fields is fallible (#1054)

Other

• move infer_schema and infer_data_type into vortex-dtype (#1081)
• Add Select expression and reorganize vortex-expr crate (#1049)

vortex-bytebool - 0.13.0 - 2024-10-29

Added

• trim metadatas (part 2) (#1028)
• teach *Metadata and ScalarValue to Display (#975)

Other

• Even more docs (#1105)

vortex-fastlanes - 0.13.0 - 2024-10-29

Added

• faster take for BitPackedArray and SparseArray (#1133)
• canonicalize indices in take if sufficiently large (#1036)
• trim metadatas (part 2) (#1028)
• improved objective function in sampling compressor (#1000)
• teach *Metadata and ScalarValue to Display (#975)

Other

• Resolve suppressed clippy warning in BitPacked::take (#1135)
• clean up stale comments (#1134)
• Even more docs (#1105)
• limit number of single char variables to 2 (#1066)
• Trim BitPackedMetadata to only required values (#1046)

vortex-scalar - 0.13.0 - 2024-10-29

Added

• German strings, attempt 3 (#1082)
• teach PyArray scalar_at (#1095)
• trim metadatas (part 2) (#1028)
• introduce ScalarType, the trait for scalar-y Rust types (#1008)
• proto matches serde: f16 serializes as an unsigned integer (#992)
• teach *Metadata and ScalarValue to Display (#975)

Fixed

• rely on ScalarValue Display in Scalar Display (#978)
• teach protobuf how to deserialize f16 (#991)

Other

• Even more docs (#1105)
• StructScalar stores borrowed arcs (#1073)
• limit number of single char variables to 2 (#1066)

vortex-flatbuffers - 0.13.0 - 2024-10-29

Added

• Unify FlatLayout with other layouts and add metadata to their flatbuffer representation (#1077)

Other

• Layouts have buffers and there's self describing schema layout (#1098)
• limit number of single char variables to 2 (#1066)

vortex-error - 0.13.0 - 2024-10-29

Added

• teach PyVortex to read from object storage (attempt two) (#1151)

vortex-dtype - 0.13.0 - 2024-10-29

Fixed

• RelativeLayoutCache must hold disk dtype (#1051)
• RoaringInt can_compress was erroneously always None (#1004)

Other

• More docs (#1104)
• limit number of single char variables to 2 (#1066)
• Delegate dtype deserialization and projection to layouts (#1060)

vortex-datetime-dtype - 0.13.0 - 2024-10-29

Other

• limit number of single char variables to 2 (#1066)

vortex-buffer - 0.13.0 - 2024-10-29

Added

• FSSTArray::into_canonical directly build VarBinView (#1161)

Fixed

• even empty slices must be aligned properly (#1112)

Other

• limit number of single char variables to 2 (#1066)

vortex-array - 0.13.0 - 2024-10-29

Added

• use hashbrown::hashmap everywhere (modestly faster decompress) (#1160)
• faster take for BitPackedArray and SparseArray (#1133)
• German strings, attempt 3 (#1082)
• print diagnostic for function implementations (#1103)
• BoolArray::take is faster (#1035)
• better error message when an array encoding id is unknown (#997)
• teach *Metadata and ScalarValue to Display (#975)

Fixed

• VarBinArray into_canonical dtype erasure (#1143) (#1145)
• support non-Primitive encodings for views (#1123)
• canonicalize null ConstantArray to VarBinViewArray (#1122)
• even empty slices must be aligned properly (#1112)
• from_fields is fallible (#1054)
• StructArray::try_from([]) is an error (#1053)
• RoaringInt can_compress was erroneously always None (#1004)
• teach pyvortex all our encodings (#998)
• BitWidthFreq must be u64/usize (#974)

Other

• VarBinViewArray take preserves nullability (#1157)
• :from_iter_bin creates non nullable array (#1150)
• Register VarBinView compare fn (#1130)
• Even more docs (#1105)
• More docs (#1104)
• move infer_schema and infer_data_type into vortex-dtype (#1081)
• limit number of single char variables to 2 (#1066)
• Register ConstantArray filter function (#1057)
• Use BooleanBuffer when canonicalizing Constant (#1056)
• Fix infinite loop in constant array compare (#1016)
• Slightly nicer unknown encodings error (#1002)
• Don't clone dtype on PrimitiveArray::cast (#972)

vortex-alp - 0.13.0 - 2024-10-29

Added

• use hashbrown::hashmap everywhere (modestly faster decompress) (#1160)
• print diagnostic for function implementations (#1103)
• improved objective function in sampling compressor (#1000)
• teach *Metadata and ScalarValue to Display (#975)

Other

• Even more docs (#1105)
• Trim BitPackedMetadata to only required values (#1046)

vortex-runend-bool - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)

Other

• Include README in Cargo.toml (#936)

vortex-bytebool - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)

Other

• Include README in Cargo.toml (#936)

vortex-zigzag - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)
• make PrimitiveArray cast properly handle validity/nullability (#968)

Other

• Include README in Cargo.toml (#936)

vortex-runend - 0.12.0 - 2024-10-03

Added

• SparseArray uses ScalarValue instead of Scalar (#955)
• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)
• make PrimitiveArray cast properly handle validity/nullability (#968)

Other

• Include README in Cargo.toml (#936)

vortex-roaring - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)

Other

• Include README in Cargo.toml (#936)

vortex-fsst - 0.12.0 - 2024-10-03

Added

• teach FSSTArray to compress the offsets of its codes (#952)
• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)
• make PrimitiveArray cast properly handle validity/nullability (#968)

Other

• Include README in Cargo.toml (#936)

vortex-dict - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)
• make PrimitiveArray cast properly handle validity/nullability (#968)

Other

• Include README in Cargo.toml (#936)

vortex-datetime-parts - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)
• cast error in compress_noci benchmark (#971)
• make PrimitiveArray cast properly handle validity/nullability (#968)

Other

• Include README in Cargo.toml (#936)

vortex-sampling-compressor - 0.12.0 - 2024-10-03

Added

• cost of for is ~0 (#967)
• implement ALP-RD compression (#947)
• teach FSSTArray to compress the offsets of its codes (#952)
• teach DeltaArray slice and scalar_at (#927)
• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)

Other

• Include README in Cargo.toml (#936)

vortex-serde - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• increase the measurement_time of benches of serde (#941)
• Include README in Cargo.toml (#936)

vortex-schema - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-expr - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-datafusion - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-fastlanes - 0.12.0 - 2024-10-03

Added

• implement ALP-RD compression (#947)
• SparseArray uses ScalarValue instead of Scalar (#955)
• teach DeltaArray slice and scalar_at (#927)
• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)
• make PrimitiveArray cast properly handle validity/nullability (#968)

Other

• Include README in Cargo.toml (#936)

vortex-scalar - 0.12.0 - 2024-10-03

Added

• teach ScalarValue and PValue is_instance_of (#958)
• SparseArray uses ScalarValue instead of Scalar (#955)
• slim down vortex-array metadata (#951)
• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-proto - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-flatbuffers - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-error - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-dtype - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-datetime-dtype - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-buffer - 0.12.0 - 2024-10-03

Added

• enable cargo lints (#948)

Other

• Include README in Cargo.toml (#936)

vortex-array - 0.12.0 - 2024-10-03

Added

• implement ALP-RD compression (#947)
• teach ScalarValue and PValue is_instance_of (#958)
• SparseArray uses ScalarValue instead of Scalar (#955)
• BoolMetadata stores bit offset in 8 bits instead of 64 (#956)
• slim down vortex-array metadata (#951)
• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)
• cast error in compress_noci benchmark (#971)
• make PrimitiveArray cast properly handle validity/nullability (#968)

Other

• Include README in Cargo.toml (#936)
• Update README.md in vortex-array
• Include README.md files in crates and link top level readme into vortex-array crate (#934)
• Converting Arrow to Vortex should create Array and not ArrayData (#931)

vortex-alp - 0.12.0 - 2024-10-03

Added

• implement ALP-RD compression (#947)
• SparseArray uses ScalarValue instead of Scalar (#955)
• enable cargo lints (#948)

Fixed

• trim array metadatas + fix validity handling (part 1) (#966)
• make PrimitiveArray cast properly handle validity/nullability (#968)
• edge case in filling ALP encoded child on patches (#939)

Other

• Include README in Cargo.toml (#936)

vortex-runend-bool - 0.11.0 - 2024-09-26

Added

• ArrayView::child will throw if encoding not found (#886)

vortex-bytebool - 0.11.0 - 2024-09-26

Added

• ArrayView::child will throw if encoding not found (#886)

vortex-runend - 0.11.0 - 2024-09-26

Added

• ArrayView::child will throw if encoding not found (#886)

vortex-roaring - 0.11.0 - 2024-09-26

Other

• Update croaring-sys to 4.1.4 and remove workarounds for croaring/660 (#898)

vortex-sampling-compressor - 0.11.0 - 2024-09-26

Added

• sampling compressor is now seeded (#917)

vortex-fastlanes - 0.11.0 - 2024-09-26

Added

• ArrayView::child will throw if encoding not found (#886)

Fixed

• BitPackedArray must be unsigned (#930)

Other

• Refactoring some IO-related code (#846)

vortex-serde - 0.11.0 - 2024-09-26

Added

• update IPC format to hold buffer_index (#903)

Other

• Naive interleaved filtering and data reading (#918)
• Refactoring some IO-related code (#846)

vortex-schema - 0.11.0 - 2024-09-26

Other

• Refactoring some IO-related code (#846)

vortex-expr - 0.11.0 - 2024-09-26

Other

• Refactoring some IO-related code (#846)

vortex-datafusion - 0.11.0 - 2024-09-26

Added

• ArrayView::child will throw if encoding not found (#886)

Other

• VortexScanExec stats are computed only once (#914)
• Refactoring some IO-related code (#846)
• VortexScanExec reports statistics to datafusion (#909)

vortex-scalar - 0.11.0 - 2024-09-26

Other

• Teach StructTrait how to project fields (#910)

vortex-flatbuffers - 0.11.0 - 2024-09-26

Added

• update IPC format to hold buffer_index (#903)

vortex-dtype - 0.11.0 - 2024-09-26

Other

• Naive interleaved filtering and data reading (#918)
• Refactoring some IO-related code (#846)

vortex-array - 0.11.0 - 2024-09-26

Added

• update IPC format to hold buffer_index (#903)
• ArrayView::child will throw if encoding not found (#886)

Other

• Refactoring some IO-related code (#846)
• Teach StructTrait how to project fields (#910)

vortex-alp - 0.11.0 - 2024-09-26

Added

• improve ALP exponent selection (#921)
• ArrayView::child will throw if encoding not found (#886)

Other

• faster ALP encode (#924)

vortex-serde - 0.10.1 - 2024-09-20

Added

• track compressed size & compare to parquet(zstd)? & canonical (#882)

vortex-runend-bool - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• Make entry point compute functions accept generic arguments (#861)

vortex-bytebool - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

vortex-zigzag - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• Make entry point compute functions accept generic arguments (#861)

vortex-runend - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• Make entry point compute functions accept generic arguments (#861)
• Fix take on sliced RunEnd array (#859)

vortex-roaring - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• Compute stats for RoaringBoolArray (#874)

vortex-fsst - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• make miri tests fast again (take 2) (#884)
• also run the compress benchmarks (#841)
• Use sliced_bytes of VarBinArray when iterating over bytes() (#867)
• Make entry point compute functions accept generic arguments (#861)

vortex-dict - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• Make entry point compute functions accept generic arguments (#861)

vortex-datetime-parts - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• Make entry point compute functions accept generic arguments (#861)

vortex-sampling-compressor - 0.10.0 - 2024-09-20

Added

• use Buffer for BitPackedArray (#862)

vortex-fastlanes - 0.10.0 - 2024-09-20

Added

• add back ptype check for BitPackedArray (#872)
• use Buffer for BitPackedArray (#862)

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• Make entry point compute functions accept generic arguments (#861)

vortex-datafusion - 0.10.0 - 2024-09-20

Other

• Make entry point compute functions accept generic arguments (#861)

vortex-buffer - 0.10.0 - 2024-09-20

Added

• use Buffer for BitPackedArray (#862)

vortex-array - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)
• teach compute_as_cast and get_as_cast to handle null-only arrays (#881)

Other

• Remove clone when creating ArrayData to run validation (#888)
• Don't validate offset buffers when converting them to arrow (#887)
• Add doc to bytes and sliced_bytes methods of VarBinArray (#869)
• Use sliced_bytes of VarBinArray when iterating over bytes() (#867)
• Make entry point compute functions accept generic arguments (#861)

vortex-alp - 0.10.0 - 2024-09-20

Fixed

• ID collision between vortex.ext and fastlanes.delta (#878)

Other

• Make entry point compute functions accept generic arguments (#861)

vortex-serde - 0.9.0 - 2024-09-17

Added

• more Results, fewer panics, always have backtraces (#761)

Fixed

• vortex-serde benchmarks depend on the ipc feature in arrow (#849)

Other

• Simplify/idiomize the way arrays return &Array (#826)
• Reorder row filters (#825)
• Introduce a new vortex-schema crate (#819)
• Convert pruning filters to express whether the block should be pruned and not whether it should stay (#800)
• Fix ChunkedArray find_chunk_idx for empty chunks (#802)
• More explicit API for converting search sorted results into indices (#777)
• overload the name 'Footer' a bit less (#773)

vortex-expr - 0.9.0 - 2024-09-17

Added

• more Results, fewer panics, always have backtraces (#761)

Other

• Reorder row filters (#825)

vortex-scalar - 0.9.0 - 2024-09-17

Added

• more Results, fewer panics, always have backtraces (#761)

Other

• Define consistent float ordering (#808)
• Actually fuzz Struct and Chunked Arrays (#805)
• Fuzz Chunked and Struct arrays (#801)
• Fuzzer performs multiple operations on the underlying array instead of just one (#766)

vortex-flatbuffers - 0.9.0 - 2024-09-17

Added

• more Results, fewer panics, always have backtraces (#761)

vortex-error - 0.9.0 - 2024-09-17

Added

• more Results, fewer panics, always have backtraces (#761)

vortex-dtype - 0.9.0 - 2024-09-17

Added

• more Results, fewer panics, always have backtraces (#761)

Other

• Add description to new vortex-schema crate (#829)
• Define consistent float ordering (#808)
• Actually fuzz Struct and Chunked Arrays (#805)

vortex-datetime-dtype - 0.9.0 - 2024-09-17

Added

• more Results, fewer panics, always have backtraces (#761)

Other

• release to Test PyPI on each push to version tags (#760)

vortex-array - 0.9.0 - 2024-09-17

Added

• implement search_sorted_many (#840)
• more Results, fewer panics, always have backtraces (#761)

Other

• Update to rust 1.81 binary_search algorithm (#851)
• Fix chunked filter handling of set slices spanning multiple chunks (#842)
• Handle empty filters when filtering empty structs (#834)
• Handle filtering empty struct arrays (#827)
• Simplify/idiomize the way arrays return &Array (#826)
• Define consistent float ordering (#808)
• Actually fuzz Struct and Chunked Arrays (#805)
• Add is_encoding to array and fix cases of redundant encoding id checks (#796)
• implement FilterFn for ChunkedArray (#794)
• Fix ChunkedArray find_chunk_idx for empty chunks (#802)
• Fuzz Chunked and Struct arrays (#801)
• implement FilterFn for SparseArray (#799)
• Better scalar compare using collect_bool (#792)
• greedily combine chunks before compressing (#783)
• Introduce MaybeCompareFn trait to allow for partial compare specializations (#768)
• More explicit API for converting search sorted results into indices (#777)
• Fix slicing already sliced SparseArray (#780)
• Fix SearchSorted for SparseArray when searching from Right (#770)
• Fix StructArray::filter length calculation (#769)
• Fuzzer performs multiple operations on the underlying array instead of just one (#766)
• Filter struct arrays (#767)
• Fix unary/binary fn on PrimitiveArray (#764)
• Fix benchmarks (#762)
• Better implementation for Validity::and (#758)

vortex-alp - 0.9.0 - 2024-09-17

Added

• more Results, fewer panics, always have backtraces (#761)

Other

• Simplify/idiomize the way arrays return &Array (#826)
• Introduce MaybeCompareFn trait to allow for partial compare specializations (#768)

vortex-runend-bool - 0.8.0 - 2024-09-05

Other

• Add scalar_at_unchecked (#666)
• Fix RunEnd take and scalar_at compute functions (#588)

vortex-bytebool - 0.8.0 - 2024-09-05

Other

• Fix issues discovered by fuzzer (#707)
• Add scalar_at_unchecked (#666)
• Move expression filters out of datafusion (#638)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-zigzag - 0.8.0 - 2024-09-05

Other

• Add scalar_at_unchecked (#666)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-runend - 0.8.0 - 2024-09-05

Other

• Teach RunEnd take to respect its own validity (#691)
• Add scalar_at_unchecked (#666)
• Assert expected row count in tpch_benchmark binary (#620)
• RunEnd array scalar_at respects validity (#608)
• Fix RunEnd take and scalar_at compute functions (#588)

vortex-roaring - 0.8.0 - 2024-09-05

Other

• Add scalar_at_unchecked (#666)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-fsst - 0.8.0 - 2024-09-05

Other

• FSSTCompressor (#664)
• Fix issues discovered by fuzzer (#707)
• Add scalar_at_unchecked (#666)

vortex-dict - 0.8.0 - 2024-09-05

Other

• Add scalar_at_unchecked (#666)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-datetime-parts - 0.8.0 - 2024-09-05

Other

• Add scalar_at_unchecked (#666)
• Move expression filters out of datafusion (#638)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-sampling-compressor - 0.8.0 - 2024-09-05

Other

• Add fuzzing for Take and SearchSorted functions (#724)
• FSSTCompressor (#664)
• Move expression filters out of datafusion (#638)
• FoR compressor handles nullable arrays (#617)
• Use then vs then_some for values that have to be lazy (#599)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-fastlanes - 0.8.0 - 2024-09-05

Other

• Fix search_sorted for FoRArray, BitPacked and PrimitiveArray (#732)
• Fix issues discovered by fuzzer (#707)
• FoR decompression happens in place if possible (#699)
• Remove length of patches from ALP and BitPacked array (#688)
• Add scalar_at_unchecked (#666)
• Bitpacking validity is checked first when getting a scalar (#630)
• Fix FoRArray decompression with non 0 shift (#625)
• FoR compressor handles nullable arrays (#617)
• Basic fuzzing for compression and slicing functions (#600)
• Use then vs then_some for values that have to be lazy (#599)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-serde - 0.8.0 - 2024-09-05

Other

• Upgrade rust nightly toolchain & MSRV (#745)
• directly implement VortexReadAt on File (#738)
• Teach schema dtype() and into_dtype() (#714)
• Add method for converting VortexExpr into equivalent pruning expression (#701)
• Primitive and Bool array roundtrip serialization (#704)
• Move flatbuffer schema project functions around (#680)
• Deduplicate filter projection with result projection (#668)
• Push filter schema manipulation into layout reader and reuse ipc message writer in file writer (#651)
• Bring back ability to convert ArrayView to ArrayData (#626)
• Move expression filters out of datafusion (#638)
• Assert expected row count in tpch_benchmark binary (#620)
• ArrayBufferReader assumes flatbuffers are validated (#610)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-expr - 0.8.0 - 2024-09-05

Other

• Add method for converting VortexExpr into equivalent pruning expression (#701)
• Fix Operator::swap (#672)
• Push filter schema manipulation into layout reader and reuse ipc message writer in file writer (#651)
• Support Temporal scalar conversion between datafusion and arrow (#657)
• cargo-sort related maintenance (#650)
• Move expression filters out of datafusion (#638)
• Generate more structured inputs for fuzzing (#635)
• Fix bug where operations were negated instead of swapped when lhs/rhs were flipped (#619)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-datafusion - 0.8.0 - 2024-09-05

Other

• Push filter schema manipulation into layout reader and reuse ipc message writer in file writer (#651)
• Support Temporal scalar conversion between datafusion and arrow (#657)
• Move expression filters out of datafusion (#638)
• Remove dead code after disk and in memory table provider unification (#633)
• Unify expression evaluation for both Table Providers (#632)
• Exact support for more expressions (#628)
• Assert expected row count in tpch_benchmark binary (#620)
• Fix a bug in vortex in-memory predicate pushdown (#618)
• Nulls as false respects original array nullability (#606)
• Fix a bug in the handling the conversion physical expression (#601)
• Vortex physical expressions support for on-disk data (#581)
• (deps) update datafusion to v41 (major) (#595)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-scalar - 0.8.0 - 2024-09-05

Other

• Fix search_sorted for FoRArray, BitPacked and PrimitiveArray (#732)
• Add fuzzing for Take and SearchSorted functions (#724)
• impl Display for Time, Date, and Timestamp (#683)
• impl Display for StructValue (#682)
• impl Display for Utf8Scalar and BinaryScalar (#678)
• Add doc to pvalue typed accessor methods (#658)
• Support Temporal scalar conversion between datafusion and arrow (#657)
• Move expression filters out of datafusion (#638)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-proto - 0.8.0 - 2024-09-05

Other

• Push filter schema manipulation into layout reader and reuse ipc message writer in file writer (#651)
• cargo-sort related maintenance (#650)

vortex-flatbuffers - 0.8.0 - 2024-09-05

Other

• Push filter schema manipulation into layout reader and reuse ipc message writer in file writer (#651)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-error - 0.8.0 - 2024-09-05

Other

• impl Display for Time, Date, and Timestamp (#683)

vortex-dtype - 0.8.0 - 2024-09-05

Other

• Move flatbuffer schema project functions around (#680)
• DType serde project requires flatbuffers feature (#679)
• Push filter schema manipulation into layout reader and reuse ipc message writer in file writer (#651)
• Support Temporal scalar conversion between datafusion and arrow (#657)
• Get beyond the immediate fuzzing failures (#611)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-datetime-dtype - 0.8.0 - 2024-09-05

Other

• impl Display for Time, Date, and Timestamp (#683)
• Support Temporal scalar conversion between datafusion and arrow (#657)
• cargo-sort related maintenance (#650)

vortex-buffer - 0.8.0 - 2024-09-05

Other

• Primitive Iterator API (#689)
• Support Temporal scalar conversion between datafusion and arrow (#657)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-array - 0.8.0 - 2024-09-05

Other

• PyVortex (#729)
• Upgrade rust nightly toolchain & MSRV (#745)
• Fix search_sorted for FoRArray, BitPacked and PrimitiveArray (#732)
• Unary and Binary functions trait (#726)
• Add fuzzing for Take and SearchSorted functions (#724)
• Fix issues discovered by fuzzer (#707)
• Primitive Iterator API (#689)
• StructArray roundtrips arrow conversion (#705)
• Primitive and Bool array roundtrip serialization (#704)
• Fix pack_varbin (#674)
• Slightly faster iter of LogicalValidity to Validity (#673)
• Fix Operator::swap (#672)
• Add scalar_at_unchecked (#666)
• Push filter schema manipulation into layout reader and reuse ipc message writer in file writer (#651)
• Faster canonicalization (#663)
• Fix slicing of ChunkedArray if end index == array length (#660)
• Implement LogicalValidity for ChunkedArray (#661)
• Support Temporal scalar conversion between datafusion and arrow (#657)
• Bring back ability to convert ArrayView to ArrayData (#626)
• Improve Primitive array comparison (#644)
• Let chunked arrays use specialized compare implementations (#640)
• Expand fuzzing space (#639)
• Move expression filters out of datafusion (#638)
• Exact support for more expressions (#628)
• Fix bug where operations were negated instead of swapped when lhs/rhs were flipped (#619)
• Get beyond the immediate fuzzing failures (#611)
• Basic fuzzing for compression and slicing functions (#600)
• Vortex physical expressions support for on-disk data (#581)
• Use then vs then_some for values that have to be lazy (#599)
• Child assert includes index and encoding id (#598)
• Clippy deny unwrap & panic in functions that return Result (#578)
• Add tests to sparse array slicing + extra length validation (#590)

vortex-alp - 0.8.0 - 2024-09-05

Other

• ALP compressor is better at roundtripping values (#736)
• Fix issues discovered by fuzzer (#707)
• Primitive Iterator API (#689)
• ALP decompress in place (#700)
• Remove length of patches from ALP and BitPacked array (#688)
• Add scalar_at_unchecked (#666)
• Fix alp null handling (#623)
• Clippy deny unwrap & panic in functions that return Result (#578)

vortex-array - 0.7.0 - 2024-08-09

Other

• Fix REE slicing with end being equal to array len (#586)
• Fix vortex compressed benchmarks (#577)

vortex-serde - 0.6.0 - 2024-08-09

Other

• Only deserialize the required dtypes by projection from the footer (#569)

vortex-buffer - 0.6.0 - 2024-08-09

Other

• enforce docstrings in vortex-buffer (#575)

vortex-array - 0.6.0 - 2024-08-09

Other

• Remove to_present_null_buffer from LogicalValidity (#579)

vortex-runend-bool - 0.5.0 - 2024-08-08

Other

• Re-import array types (#559)
• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Added bool iterators index and slice and filtering across some array types (#505)
• Fix out ouf bounds when taking from run end arrays (#501)
• Change codes for runendbool so it doesn't conflict with datetimeparts (#498)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)

vortex-bytebool - 0.5.0 - 2024-08-08

Other

• Refactor specialized conversion traits into From and Into (#560)
• Re-import array types (#559)
• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Simpler ByteBool slice (#527)
• Added bool iterators index and slice and filtering across some array types (#505)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)

vortex-serde - 0.5.0 - 2024-08-08

Other

• Push column projections down to the file IO layer (#568)
• Lots of things to try and get publishing working (#557)
• Support dynamic layouts with io batching (#533)
• Re-import array types (#559)
• File-based table provider for Datafusion (#546)
• build-vortex -> vortex-build (#552)
• Use versioned workspace deps (#551)
• Add identity projection to the file reader (#532)
• Support reading unaligned chunks across columns (#531)
• Initial version of simple FileReader/Writer (#516)

vortex-datafusion - 0.5.0 - 2024-08-08

Other

• Hook on-disk vortex files into benchmarking (#565)

vortex-error - 0.5.0 - 2024-08-08

Other

• Lots of things to try and get publishing working (#557)

vortex-array - 0.5.0 - 2024-08-08

Other

• Lots of things to try and get publishing working (#557)
• Hook on-disk vortex files into benchmarking (#565)

vortex-runend-bool - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Added bool iterators index and slice and filtering across some array types (#505)
• Fix out ouf bounds when taking from run end arrays (#501)
• Change codes for runendbool so it doesn't conflict with datetimeparts (#498)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)

vortex-bytebool - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Simpler ByteBool slice (#527)
• Added bool iterators index and slice and filtering across some array types (#505)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)

vortex-serde - 0.2.0 - 2024-08-05

Other

• build-vortex -> vortex-build (#552)
• Use versioned workspace deps (#551)
• Add identity projection to the file reader (#532)
• Support reading unaligned chunks across columns (#531)
• Initial version of simple FileReader/Writer (#516)

vortex-sampling-compressor - 0.2.0 - 2024-08-05

Fixed

• fix UB and run tests with miri (#517)

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• FoR will compress signed array when min == 0 now (#511)
• Smoketest for SamplingCompressor, fix bug in varbin stats (#510)
• Make unary functions nicer to use (#493)
• Remove LocalDateTimeArray, introduce TemporalArray (#480)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Array Length (#445)
• Split compression from encodings (#422)

vortex-runend - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Fix out ouf bounds when taking from run end arrays (#501)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• demo module level imports granularity (#485)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Use shorthand canonicalize methods (#460)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• ArrayData can contain child Arrays instead of just ArrayData (#391)
• Rename typed_data to maybe_null_slice (#386)
• Move encodings into directory (#379)

vortex-datafusion - 0.2.0 - 2024-08-05

Fixed

• fix UB and run tests with miri (#517)

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Follow up for 537 (#538)
• Rename the pushdown config into a positive boolean value (#537)
• Ignore tests that miri can't run (#514)
• Add and/or compute functions (#481)
• demo module level imports granularity (#485)
• Remove LocalDateTimeArray, introduce TemporalArray (#480)
• Expand pushdown support with more comparison and logical operations (#478)
• Debug compilation caching (#475)
• Basic predicate pushdown support for Datafusion (#472)
• Slightly nicer use statements for compute functions (#466)
• Run tpch_benchmark queries single-threaded in rayon pool (#463)
• Update datafusion to v40 (major) (#455)
• Make into_arrow truly zero-copy, rewrite DataFusion operators (#451)
• Setup TPC-H benchmark infra (#444)
• v0 Datafusion with late materialization (#414)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• DataFusion TableProvider for memory arrays (#384)

vortex-scalar - 0.2.0 - 2024-08-05

Other

• build-vortex -> vortex-build (#552)
• Use versioned workspace deps (#551)
• Initial version of simple FileReader/Writer (#516)
• More specialized compare functions (#488)
• demo module level imports granularity (#485)
• Basic predicate pushdown support for Datafusion (#472)
• DType variant traits (#473)
• FoR encoding doesn't panic if array min > i64::MAX (#406)
• Add ByteBoolArray type and fixe a bug in BoolArray (#383)
• FoR array holds encoded values as unsinged (#401)
• DataFusion expr conversion (#349)
• Fix FOR bug, also fix bench to compile (#341)
• Implement StructValue proto serde without google.protobuf.Value (#343)
• Random access benchmarks are runnable again (#330)
• define ScalarValue in VortexScalar protobuf (#323)
• Proto Refactor (#325)
• IPC Bench (#319)
• Static ArrayView (#310)
• StatsView2 (#305)
• Add ScalarView (#301)
• DType Serialization (#298)
• OwnedBuffer (#300)
• Add validity to Struct arrays (#289)
• Extension Array (#287)
• Remove composite and decimal (#285)
• Add convenience stats retrieval functions and avoid needless copy when unwrapping stat value (#279)
• Scalar subtraction (#270)
• Add ExtDType (#281)
• Refactor for DType::Primitive (#276)
• Extract a vortex-scalar crate (#275)

vortex-runend-bool - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Added bool iterators index and slice and filtering across some array types (#505)
• Fix out ouf bounds when taking from run end arrays (#501)
• Change codes for runendbool so it doesn't conflict with datetimeparts (#498)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)

vortex-bytebool - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Simpler ByteBool slice (#527)
• Added bool iterators index and slice and filtering across some array types (#505)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)

vortex-serde - 0.2.0 - 2024-08-05

Other

• build-vortex -> vortex-build (#552)
• Use versioned workspace deps (#551)
• Add identity projection to the file reader (#532)
• Support reading unaligned chunks across columns (#531)
• Initial version of simple FileReader/Writer (#516)

vortex-zigzag - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• ArrayData can contain child Arrays instead of just ArrayData (#391)
• Rename typed_data to maybe_null_slice (#386)
• Fastlanez -> Fastlanes (#381)
• Move encodings into directory (#379)

vortex-sampling-compressor - 0.2.0 - 2024-08-05

Fixed

• fix UB and run tests with miri (#517)

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• FoR will compress signed array when min == 0 now (#511)
• Smoketest for SamplingCompressor, fix bug in varbin stats (#510)
• Make unary functions nicer to use (#493)
• Remove LocalDateTimeArray, introduce TemporalArray (#480)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Array Length (#445)
• Split compression from encodings (#422)

vortex-runend - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Fix out ouf bounds when taking from run end arrays (#501)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• demo module level imports granularity (#485)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Use shorthand canonicalize methods (#460)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• ArrayData can contain child Arrays instead of just ArrayData (#391)
• Rename typed_data to maybe_null_slice (#386)
• Move encodings into directory (#379)

vortex-roaring - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Ignore tests that miri can't run (#514)
• Added bool iterators index and slice and filtering across some array types (#505)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• demo module level imports granularity (#485)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• Rename typed_data to maybe_null_slice (#386)
• Move encodings into directory (#379)

vortex-fastlanes - 0.2.0 - 2024-08-05

Fixed

• fix UB and run tests with miri (#517)

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• demo module level imports granularity (#485)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Use shorthand canonicalize methods (#460)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Fix semantic conflict between searching and slicing sparse and bitpacked arrays (#412)
• Fix Slice and SearchSorted for BitPackedArray (#410)
• FoR encoding doesn't panic if array min > i64::MAX (#406)
• Add search_sorted for FOR, Bitpacked and Sparse arrays (#398)
• FoR array holds encoded values as unsinged (#401)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• ArrayData can contain child Arrays instead of just ArrayData (#391)
• Rename typed_data to maybe_null_slice (#386)
• Fastlanez -> Fastlanes (#381)
• Move encodings into directory (#379)

vortex-dict - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• demo module level imports granularity (#485)
• Basic predicate pushdown support for Datafusion (#472)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Run cargo doc at CI time (#469)
• Use shorthand canonicalize methods (#460)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• ArrayData can contain child Arrays instead of just ArrayData (#391)
• Rename typed_data to maybe_null_slice (#386)
• Move encodings into directory (#379)

vortex-datetime-parts - 0.2.0 - 2024-08-05

Fixed

• canonicalization of chunked ExtensionArray (#499)

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Add license check to CI (#518)
• Fix SparseArray validity logic and give DateTimeParts unique code (#495)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• demo module level imports granularity (#485)
• Remove LocalDateTimeArray, introduce TemporalArray (#480)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Use shorthand canonicalize methods (#460)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• ArrayData can contain child Arrays instead of just ArrayData (#391)
• Rename typed_data to maybe_null_slice (#386)
• Move encodings into directory (#379)

vortex-datafusion - 0.2.0 - 2024-08-05

Fixed

• fix UB and run tests with miri (#517)

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Follow up for 537 (#538)
• Rename the pushdown config into a positive boolean value (#537)
• Ignore tests that miri can't run (#514)
• Add and/or compute functions (#481)
• demo module level imports granularity (#485)
• Remove LocalDateTimeArray, introduce TemporalArray (#480)
• Expand pushdown support with more comparison and logical operations (#478)
• Debug compilation caching (#475)
• Basic predicate pushdown support for Datafusion (#472)
• Slightly nicer use statements for compute functions (#466)
• Run tpch_benchmark queries single-threaded in rayon pool (#463)
• Update datafusion to v40 (major) (#455)
• Make into_arrow truly zero-copy, rewrite DataFusion operators (#451)
• Setup TPC-H benchmark infra (#444)
• v0 Datafusion with late materialization (#414)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• DataFusion TableProvider for memory arrays (#384)

vortex-scalar - 0.2.0 - 2024-08-05

Other

• build-vortex -> vortex-build (#552)
• Use versioned workspace deps (#551)
• Initial version of simple FileReader/Writer (#516)
• More specialized compare functions (#488)
• demo module level imports granularity (#485)
• Basic predicate pushdown support for Datafusion (#472)
• DType variant traits (#473)
• FoR encoding doesn't panic if array min > i64::MAX (#406)
• Add ByteBoolArray type and fixe a bug in BoolArray (#383)
• FoR array holds encoded values as unsinged (#401)
• DataFusion expr conversion (#349)
• Fix FOR bug, also fix bench to compile (#341)
• Implement StructValue proto serde without google.protobuf.Value (#343)
• Random access benchmarks are runnable again (#330)
• define ScalarValue in VortexScalar protobuf (#323)
• Proto Refactor (#325)
• IPC Bench (#319)
• Static ArrayView (#310)
• StatsView2 (#305)
• Add ScalarView (#301)
• DType Serialization (#298)
• OwnedBuffer (#300)
• Add validity to Struct arrays (#289)
• Extension Array (#287)
• Remove composite and decimal (#285)
• Add convenience stats retrieval functions and avoid needless copy when unwrapping stat value (#279)
• Scalar subtraction (#270)
• Add ExtDType (#281)
• Refactor for DType::Primitive (#276)
• Extract a vortex-scalar crate (#275)

vortex-expr - 0.2.0 - 2024-08-05

Other

• build-vortex -> vortex-build (#552)
• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Clean up fields / field paths (#353)
• Expression proto serde (#351)
• DataFusion expr conversion (#349)
• FilterIndices compute function (#326)
• Introduce FieldPath abstraction, restrict predicates to Field, Op, (Field|Scalar) (#324)
• Minimal expressions API for vortex (#318)

vortex-flatbuffers - 0.2.0 - 2024-08-05

Other

• Run cargo-sort on the whole workspace (#550)
• Add ScalarView (#301)
• Remove unused dependencies (and bump lance) (#286)
• Add ExtDType (#281)
• IPC Terminator (#267)
• Refactor (#237)
• Constant (#230)
• Format imports (#184)
• IPC Prototype (#181)

vortex-error - 0.2.0 - 2024-08-05

Other

• Run cargo-sort on the whole workspace (#550)
• Minimal support for reading vortex with object_store (#427)
• IPC Bench (#319)
• More Async IPC (#313)
• Add ScalarView (#301)
• Extension Array (#283)
• Struct Array (#217)
• Array Data + View (#210)
• IPC Prototype (#181)
• Reduce number of distinct error types and capture tracebacks (#175)
• Random Access Benchmark (#149)
• Vortex Error (#133)

vortex-dtype - 0.2.0 - 2024-08-05

Other

• build-vortex -> vortex-build (#552)
• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• Initial version of simple FileReader/Writer (#516)
• Add and/or compute functions (#481)
• demo module level imports granularity (#485)
• Avoid dtype comparison failure in take -- upcast indices in take_strict_sorted (#464)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• DataFusion TableProvider for memory arrays (#384)
• Clean up fields / field paths (#353)
• Expression proto serde (#351)
• DataFusion expr conversion (#349)
• FilterIndices compute function (#326)
• Proto Refactor (#325)
• Introduce FieldPath abstraction, restrict predicates to Field, Op, (Field|Scalar) (#324)
• Minimal expressions API for vortex (#318)
• IPC Bench (#319)
• Remove buffer -> dtype dependency (#309)
• Add ScalarView (#301)
• DType Serialization (#298)
• Add validity to Struct arrays (#289)
• Remove unused dependencies (and bump lance) (#286)
• Remove composite and decimal (#285)
• Extension Array (#283)
• Add convenience stats retrieval functions and avoid needless copy when unwrapping stat value (#279)
• Scalar subtraction (#270)
• Add ExtDType (#281)
• Refactor for DType::Primitive (#276)
• Move PType into vortex-dtype (#274)
• Vortex Schema -> Vortex DType (#273)

vortex-buffer - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• setup automated releases with release-plz (#547)
• Make into_arrow truly zero-copy, rewrite DataFusion operators (#451)
• DataFusion TableProvider for memory arrays (#384)
• Buffer into_vec respects value alignment (#387)
• More IPC Refactorings (#329)
• IPC Bench (#319)
• More Async IPC (#313)
• Async IPC (#307)
• Remove buffer -> dtype dependency (#309)
• Add ScalarView (#301)
• OwnedBuffer (#300)
• Vortex Buffer Crate (#299)

vortex-array - 0.2.0 - 2024-08-05

Fixed

• fix UB and run tests with miri (#517)
• canonicalization of chunked ExtensionArray (#499)
• fix comment on TemporalArray::new_time (#482)

Other

• build-vortex -> vortex-build (#552)
• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• setup automated releases with release-plz (#547)
• Initial version of simple FileReader/Writer (#516)
• Use Arrow's varbin builder (#519)
• Smoketest for SamplingCompressor, fix bug in varbin stats (#510)
• Added bool iterators index and slice and filtering across some array types (#505)
• Fix remaining copies (#504)
• Remove some vortex mem allocations in "zero-copy" memory transformations (#503)
• Lazy deserialize metadata from ArrayData and ArrayView (#502)
• Fix out ouf bounds when taking from run end arrays (#501)
• More specialized compare functions (#488)
• Fix SparseArray validity logic and give DateTimeParts unique code (#495)
• Add and/or compute functions (#481)
• Implement CastFn for chunkedarray (#497)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• demo module level imports granularity (#485)
• Remove LocalDateTimeArray, introduce TemporalArray (#480)
• Debug compilation caching (#475)
• Basic predicate pushdown support for Datafusion (#472)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Avoid dtype comparison failure in take -- upcast indices in take_strict_sorted (#464)
• Use shorthand canonicalize methods (#460)
• Add FilterFn trait + default implementation (#458)
• Make into_arrow truly zero-copy, rewrite DataFusion operators (#451)
• Completely remove scalar_buffer method (#448)
• Chunked take (#447)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Buffer chunks to read when taking rows (#419)
• v0 Datafusion with late materialization (#414)
• Add SearchSortedFn for ConstantArray (#417)
• Add SliceFn implementation for ConstantArray (#416)
• Fix Slice and SearchSorted for BitPackedArray (#410)
• Fix SearchSorted on sliced sparse array (#411)
• Add ByteBoolArray type and fixe a bug in BoolArray (#383)
• Add search_sorted for FOR, Bitpacked and Sparse arrays (#398)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• DataFusion TableProvider for memory arrays (#384)
• Use ChunkedArrayReader in random access benchmark (#393)
• ArrayData can contain child Arrays instead of just ArrayData (#391)
• Buffer into_vec respects value alignment (#387)
• Rename typed_data to maybe_null_slice (#386)
• Fastlanez -> Fastlanes (#381)
• Use IntoArrayData when we have owned arrays (#376)
• Clean up fields / field paths (#353)
• Use new search-sorted for finding chunk index (#342)
• NullArray + statsset cleanup (#350)
• Expression proto serde (#351)
• DataFusion expr conversion (#349)
• Fix FOR bug, also fix bench to compile (#341)
• Array comparison compute function (#336)
• FilterIndices compute function (#326)
• Take Rows Chunked Array (#331)
• Random access benchmarks are runnable again (#330)
• ChunkedArray is not a flat encoding (#332)
• More IPC Refactorings (#329)
• Add ArrayIterator and ArrayStream (#327)
• Stats don't allocate errors on missing stats (#320)
• IPC Bench (#319)
• Remove flatbuffers build.rs (#316)
• BoolArray stats respect nulls (#314)
• Remove array lifetimes (#312)
• Static ArrayView (#310)
• Async IPC (#307)
• Remove buffer -> dtype dependency (#309)
• Fix chunked array stat merging (#303)
• Include stats in IPC messages (#302)
• StatsView2 (#305)
• Add ScalarView (#301)
• DType Serialization (#298)
• OwnedBuffer (#300)
• Vortex Buffer Crate (#299)
• Support WASM (#297)
• Add Context and remove linkme (#295)
• Add validity to Struct arrays (#289)
• IPC take returns an iterator instead of ChunkedArray (#271)
• Extension Array (#287)
• Remove unused dependencies (and bump lance) (#286)
• Remove composite and decimal (#285)
• DateTimeParts (#284)
• Extension Array (#283)
• Add convenience stats retrieval functions and avoid needless copy when unwrapping stat value (#279)
• Scalar subtraction (#270)
• Add ExtDType (#281)
• Bring back slice for ChunkedArray (#280)
• Refactor for DType::Primitive (#276)
• Extract a vortex-scalar crate (#275)
• Move PType into vortex-dtype (#274)
• Vortex Schema -> Vortex DType (#273)
• Implement take for StreamArrayReader (#266)
• Don't skip first element in stats calculation (#268)
• Enable sparse compression (#262)
• Logical validity from stats (#264)
• Refactor (#237)
• Comparison artifacts & analysis (#247)
• Fix binary stats for arrays containing null bytes and match stats behaviour between varbin and primitive arrays (#233)
• Address comments from varbin enhancement pr (#231)
• SearchSorted can return whether search resulted in exact match (#226)
• Convert slice to compute function (#227)
• Constant (#230)
• Array2 compute (#224)
• Better iterators for VarBin/VarBinView that don't always copy (#221)
• Try to inline WithCompute calls (#223)
• Struct Array (#217)
• Optimize bitpacked take (#192)
• SparseArray TakeFn returns results in the requested order (#212)
• Add TakeFn for SparseArray (#206)
• Slightly simplified SparseArray FlattenFn (#205)
• Don't zero memory when reading a buffer (#208)
• Move validity into a trait (#198)
• Patching Bitpacked and ALP arrays doesn't require multiple copies (#189)
• Implement Validity for SparseArray, make scalar_at for bitpacked array respect patches (#194)
• Simplify chunk end searching in ChunkedArray (#199)
• Compute with a primitive trait (#191)
• Skip codecs where can_compress on sample is null (#188)
• Accessor lifetime (#186)
• Validity array (#185)
• Format imports (#184)
• IPC Prototype (#181)
• Use wrapping arithmetic for Frame of Reference (#178)
• Reduce number of distinct error types and capture tracebacks (#175)
• Implement generic search sorted using scalar_at (#167)
• Add Take for Bitpacked array (#161)
• Scalar_at for FORArray (#159)
• Random Access Benchmark (#149)
• Remove unknown (#156)
• Nullable scalars (#152)
• Implement Flatten for DictArray (#143)
• Implement take for BoolArray (#146)
• Chunked Take (#141)
• Fix dict encoding validity (#138)
• Add Validity enum (#136)
• Vortex Error (#133)
• Fastlanes delta (#57)
• Fix encoding discovery (#132)
• Upgrade arrow-rs to 51.0.0 and extract common dependencies to top level (#127)
• Make EncodingID Copy (#131)
• Noah's Arc (#130)
• Use flatbuffers to serialize dtypes (#126)
• DateTime encoding (#90)
• Split vortex-schema from main crate (#124)
• flatten ALP arrays (#123)
• Composite Arrays (#122)
• Rename Typed to Composite (#120)
• Replace iter arrow with flatten (#109)
• Decompress to Arrow (#106)
• Add ability to define composite dtypes, i.e. dtypes redefining meaning (#103)
• Serde errors (#105)
• Trim down arrow dependency (#98)
• Add bit shifting to FOR (#89)
• remove dead polars code (#95)
• Add sizeof tests (#94)
• Scalars are an enum (#93)
• Search sorted (#91)
• More Compression (#87)
• Cleanup Dict encoding (#82)
• Compression Updates (#84)
• Array display (#83)
• Compressor recursion (#73)
• Rust ALP (#72)
• Remove PrimitiveArray::from_vec in favour of PrimitiveArray::from (#70)
• Fill forward compute function (#69)
• Root project is vortex-array (#67)

vortex-alp - 0.2.0 - 2024-08-05

Other

• Use versioned workspace deps (#551)
• Run cargo-sort on the whole workspace (#550)
• setup automated releases with release-plz (#547)
• Make unary functions nicer to use (#493)
• use FQDNs in impl_encoding macro (#490)
• demo module level imports granularity (#485)
• DType variant traits (#473)
• Slightly nicer use statements for compute functions (#466)
• Array Length (#445)
• Remove ViewContext and assign stable ids to encodings (#433)
• Split compression from encodings (#422)
• Add search_sorted for FOR, Bitpacked and Sparse arrays (#398)
• Rename flatten -> canonicalize + bugfix + a secret third thing (#402)
• ArrayData can contain child Arrays instead of just ArrayData (#391)
• Rename typed_data to maybe_null_slice (#386)
• Fastlanez -> Fastlanes (#381)
• Move encodings into directory (#379)