Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Kernel] Add JsonMetadataDomain and RowTrackingMetadataDomain #3893

Open
wants to merge 36 commits into
base: master
Choose a base branch
from
Open

[Kernel] Add JsonMetadataDomain and RowTrackingMetadataDomain #3893

wants to merge 36 commits into from

Commits on Oct 31, 2024

  1. Added Domain Metadata support to Delta Kernel

    @qiyuandong-db
    qiyuandong-db committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    94260ef View commit details
    Browse the repository at this point in the history

  2. Lazily load domain metadata during log replay.

    @qiyuandong-db
    qiyuandong-db committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    7612613 View commit details
    Browse the repository at this point in the history

  3. Use an iterator to wrap the data action iterator for DM duplicate det… 

    …ection
    @qiyuandong-db
    qiyuandong-db committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    2ac3985 View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2024

  1. Update kernel/kernel-api/src/main/java/io/delta/kernel/internal/Delta… 

    …Errors.java

Co-authored-by: Johan Lasperas <[email protected]>
    @qiyuandong-db @johanl-db
    qiyuandong-db and johanl-db committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    807c896 View commit details
    Browse the repository at this point in the history

  2. Improve error message 

    Co-authored-by: Johan Lasperas <[email protected]>
    @qiyuandong-db @johanl-db
    qiyuandong-db and johanl-db committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    84d4ae6 View commit details
    Browse the repository at this point in the history

  3. Rename a unit test 

    Co-authored-by: Johan Lasperas <[email protected]>
    @qiyuandong-db @johanl-db
    qiyuandong-db and johanl-db committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    509bc27 View commit details
    Browse the repository at this point in the history

  4. Don't allow duplicate DMs when reading actions from the winning txn d… 

    …uring conflict resolution
    @qiyuandong-db
    qiyuandong-db committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    83f3b1e View commit details
    Browse the repository at this point in the history

  5. Update error messages in the tests

    @qiyuandong-db
    qiyuandong-db committed Nov 1, 2024
    Configuration menu
    Copy the full SHA
    7d7032e View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2024

  1. Fix typos in comments

    @qiyuandong-db
    qiyuandong-db committed Nov 3, 2024
    Configuration menu
    Copy the full SHA
    dadc5a6 View commit details
    Browse the repository at this point in the history

Commits on Nov 4, 2024

  1. Add an integration test with spark

    @qiyuandong-db
    qiyuandong-db committed Nov 4, 2024
    Configuration menu
    Copy the full SHA
    0c188c7 View commit details
    Browse the repository at this point in the history

  2. Fix the JavadocGenerationFailed error in the Delta Kernel CI job.

    @qiyuandong-db
    qiyuandong-db committed Nov 4, 2024
    Configuration menu
    Copy the full SHA
    7948fdf View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2024

  1. Resolve PR comments.

    @qiyuandong-db
    qiyuandong-db committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    6195e7f View commit details
    Browse the repository at this point in the history

  2. Update util method extractDomainMetadataMap.

    @qiyuandong-db
    qiyuandong-db committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    fd06f6d View commit details
    Browse the repository at this point in the history

  3. Remove blank lines.

    @qiyuandong-db
    qiyuandong-db committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    3e30a41 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. Address PR comments

    @qiyuandong-db
    qiyuandong-db committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    cec85cf View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. Address PR comments

    @qiyuandong-db
    qiyuandong-db committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    8edcc9a View commit details
    Browse the repository at this point in the history

  2. Move domain metadata actions out of dataActions

    @qiyuandong-db
    qiyuandong-db committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    baca1c5 View commit details
    Browse the repository at this point in the history

  3. Fix javafmt

    @qiyuandong-db
    qiyuandong-db committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    7e0a172 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2024

  1. Use a set to check for unsupported writer features

    @qiyuandong-db
    qiyuandong-db committed Nov 10, 2024
    Configuration menu
    Copy the full SHA
    127de8c View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. Resolve PR comments

    @qiyuandong-db
    qiyuandong-db committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    c5f3672 View commit details
    Browse the repository at this point in the history

  2. Move golden table to kernel tests.

    @qiyuandong-db
    qiyuandong-db committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    b2d6546 View commit details
    Browse the repository at this point in the history

  3. Use getTestResourceFilePath in test to get golden table path.

    @qiyuandong-db
    qiyuandong-db committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    cd7ddeb View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. Resolve git comments

    @qiyuandong-db
    qiyuandong-db committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    68c77d8 View commit details
    Browse the repository at this point in the history

  2. Move resolveDomainMetadataConflict into handleDomainMetadata

    @qiyuandong-db
    qiyuandong-db committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    5afec36 View commit details
    Browse the repository at this point in the history

  3. Move addDomainMetadata from TransactionImpl to TransactionBuilderImpl

    @qiyuandong-db
    qiyuandong-db committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    e358878 View commit details
    Browse the repository at this point in the history

  4. Use SUPPORTED_WRITER_FEATURES in validateWriteSupportedTable

    @qiyuandong-db
    qiyuandong-db committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    4f56a2f View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2024

  1. Remove the duplicate check when reading winning txn's DM. Change extr… 

    …actDomainMetadataMap to fillDomainMetadataMap.
    @qiyuandong-db
    qiyuandong-db committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    189ec75 View commit details
    Browse the repository at this point in the history

  2. Fix nit

    @qiyuandong-db
    qiyuandong-db committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    a4e3104 View commit details
    Browse the repository at this point in the history

  3. Rename populateDomainMetadataMap

    @qiyuandong-db
    qiyuandong-db committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    b0e4a65 View commit details
    Browse the repository at this point in the history

  4. Remove unused imports

    @qiyuandong-db
    qiyuandong-db committed Nov 13, 2024
    Configuration menu
    Copy the full SHA
    f89199c View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2024

  1. Resolve PR comments

    @qiyuandong-db
    qiyuandong-db committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    a5c5726 View commit details
    Browse the repository at this point in the history

  2. Create the integration test table using Spark

    @qiyuandong-db
    qiyuandong-db committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    a893fa0 View commit details
    Browse the repository at this point in the history

Commits on Nov 20, 2024

  1. Move addDomainMetadata from TransactionBuilderImpl to TransactionImpl

    @qiyuandong-db
    qiyuandong-db committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    73faf8f View commit details
    Browse the repository at this point in the history

  2. Update domainMetadataUnsupported error message

    @qiyuandong-db
    qiyuandong-db committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    9ce9b56 View commit details
    Browse the repository at this point in the history

  3. Update the error message in the test

    @qiyuandong-db
    qiyuandong-db committed Nov 20, 2024
    Configuration menu
    Copy the full SHA
    60a29dd View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. Add JsonMetadataDomain & RowTrackingMetadataDomain

    @qiyuandong-db
    qiyuandong-db committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    b32c56a View commit details
    Browse the repository at this point in the history