Skip to content

Latest commit

 

History

History
104 lines (62 loc) · 5.84 KB

CHANGELOG.md

File metadata and controls

104 lines (62 loc) · 5.84 KB
Raw

Changelog

v0.3.1 (2024-09-10)

Full Changelog

API Changes

Additions

  1. Two new binary expressions: In and NotIn, as well as a new Scalar::Array variant to represent arrays in the expression framework #270 NOTE: exact API for these expressions is still evolving.

Implemented enhancements:

  • Enabled more golden table tests #301

Fixed bugs:

  • Allow kernel to read tables with invalid _last_checkpoint #311
  • List log files with checkpoint hint when constructing latest snapshot (when version requested is None) #312
  • Fix incorrect offset value when computing list offsets #327
  • Fix metadata string conversion in default engine arrow conversion #328

v0.3.0 (2024-08-07)

Full Changelog

API Changes

Breaking

  1. delta_kernel::column_mapping module moved to delta_kernel::features::column_mapping #222

Additions

  1. New deletion vector API row_indexes (and accompanying FFI) to get row indexes instead of seletion vector of deleted rows. This can be more efficient for sparse DVs. #215
  2. Typed table features: ReaderFeatures, WriterFeatures enums and has_reader_feature/has_writer_feature API #222

Implemented enhancements:

  • Add --limit option to example read-table-multi-threaded #297
  • FFI now built with cmake. Move to using the read-test example as an ffi-test. And building on macos. #288
  • Golden table tests migrated from delta-spark/delta-kernel java #295
  • Code coverage implemented via cargo-llvm-cov and reported with codecov #287
  • All tests enabled to run in CI #284
  • Updated DAT to 0.3 #290

Fixed bugs:

  • Evaluate timestamps as "UTC" instead of "+00:00" for timezone #295
  • Make Map arrow type field naming consistent with parquet field naming #299

v0.2.0 (2024-07-17)

Full Changelog

API Changes

Breaking

  1. The scan callback if using visit_scan_files now takes an extra Option<Stats> argument, holding top level stats for associated scan file. You will need to add this argument to your callback.

    Likewise, the callback in the ffi code also needs to take a new argument which is a pointer to a Stats struct, and which can be null if no stats are present.

Additions

  1. You can call scan_builder() directly on a snapshot, for more convenience.
  2. You can pass a URL starting with "hdfs" or "viewfs" to the default client to read using hdfs_native_store

Implemented enhancements:

  • Handle nested structs in schemaString (allows reading iceberg compat tables) #257
  • Expose top level stats in scans #227
  • Hugely expanded C-FFI example #203
  • Add scan_builder function to Snapshot #273
  • Add hdfs_native_store support #273
  • Proper reading of Parquet files, including only reading requested leaves, type casting, and reordering #271
  • Allow building the package if you are behind an https proxy #282

Fixed bugs:

  • Don't error if more fields exist than expected in a struct expression #267
  • Handle cases where the deletion vector length is less than the total number of rows in the chunk #276
  • Fix partition map indexing if column mapping is in effect #278

v0.1.1 (2024-06-03)

Full Changelog

Implemented enhancements:

  • Support unary NOT and IsNull for data skipping #231
  • Add unary visitors to c ffi #247
  • Minor other QOL improvements

v0.1.0 (2024-06-12)

Initial public release