2024-04-04 kernel meeting notes

[zachschuermann]

summary

major prios: FFI PR, do a crates.io release.

action items

some PRs in flight:

• start implementing FFI #91 MERGING!
• [WIP] Implement extern EngineData #152
• Use a proc macro for being able to derive schemas for our action structs #129 (resurrected and merged below)
• Schema derive macro (reborn) #156
• Add backtrace option to errors #151
• Read and verify dat data + initial bit of timestamp_ntz #154

attendees

@nicklan @zachschuermann @hntd187 @rtyler @tdas

notes stream

• FFI PR start implementing FFI #91 to merge soon
• tyler: want to do a release on crates.io!
  • then we can start integrating types
• planning on 0.0.x release on crates.io
• txn action support
• tyler: need timeline on crates release. blocking a PR: feat: implement transaction identifiers delta-io/delta-rs#2327
• @nicklan talking with @tdas before committing! but soon as possible!!
• crates.io release context: we lock down a name+version+snapshot of code
• @tdas says LGTM
• planning on crates release tomorrow 🥳
• need to work with david to pull some changes to kernel side (@nicklan and @zachschuermann and @hntd187 volunteer)
• does delta-rs need a more complex scan API? tyler: confident the current scans aren't simple. in general would probably need complex due to datafusion
• nick: simple and complex API for both use cases