Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Resumable IterableDataset] Add IterableDataset state_dict #6658

Merged
merged 29 commits into from
Jun 3, 2024
Merged

[Resumable IterableDataset] Add IterableDataset state_dict #6658

merged 29 commits into from
Jun 3, 2024

Commits on Feb 11, 2024

  1. add ex_iterable state_dict() and load_state_dict()

    @lhoestq
    lhoestq committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    8987126 View commit details
    Browse the repository at this point in the history

  2. style

    @lhoestq
    lhoestq committed Feb 11, 2024
    Configuration menu
    Copy the full SHA
    db26933 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. resuming + IterableDataset state_dict() + load_state_dict()

    @lhoestq
    lhoestq committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    367be53 View commit details
    Browse the repository at this point in the history

  2. minor

    @lhoestq
    lhoestq committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    d77b0a3 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2024

  1. Merge branch 'main' into iterable-dataset-state-dict

    @lhoestq
    lhoestq committed May 27, 2024
    Configuration menu
    Copy the full SHA
    48aab0b View commit details
    Browse the repository at this point in the history

  2. fix tests

    @lhoestq
    lhoestq committed May 27, 2024
    Configuration menu
    Copy the full SHA
    bcbf9ed View commit details
    Browse the repository at this point in the history

  3. fix one more test

    @lhoestq
    lhoestq committed May 27, 2024
    Configuration menu
    Copy the full SHA
    30d0cf2 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. implement CyclingMultiSourcesExamplesIterable resuming

    @lhoestq
    lhoestq committed May 28, 2024
    Configuration menu
    Copy the full SHA
    ca2ab81 View commit details
    Browse the repository at this point in the history

  2. remove unused code

    @lhoestq
    lhoestq committed May 28, 2024
    Configuration menu
    Copy the full SHA
    a84ffe0 View commit details
    Browse the repository at this point in the history

  3. fix spark

    @lhoestq
    lhoestq committed May 28, 2024
    Configuration menu
    Copy the full SHA
    254ff57 View commit details
    Browse the repository at this point in the history

Commits on May 29, 2024

  1. fix spark tests

    @lhoestq
    lhoestq committed May 29, 2024
    Configuration menu
    Copy the full SHA
    cfd4875 View commit details
    Browse the repository at this point in the history

  2. add test dependency

    @lhoestq
    lhoestq committed May 29, 2024
    Configuration menu
    Copy the full SHA
    d620071 View commit details
    Browse the repository at this point in the history

  3. enable long paths for git

    @lhoestq
    lhoestq committed May 29, 2024
    Configuration menu
    Copy the full SHA
    14b3d92 View commit details
    Browse the repository at this point in the history

  4. fix git command

    @lhoestq
    lhoestq committed May 29, 2024
    Configuration menu
    Copy the full SHA
    fc0f892 View commit details
    Browse the repository at this point in the history

  5. no additional deps for windows

    @lhoestq
    lhoestq committed May 29, 2024
    Configuration menu
    Copy the full SHA
    307a8d3 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Merge branch 'main' into iterable-dataset-state-dict

    @lhoestq
    lhoestq committed May 30, 2024
    Configuration menu
    Copy the full SHA
    c77c7aa View commit details
    Browse the repository at this point in the history

  2. fix tests

    @lhoestq
    lhoestq committed May 30, 2024
    Configuration menu
    Copy the full SHA
    f08082d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into iterable-dataset-state-dict

    @lhoestq
    lhoestq committed May 30, 2024
    Configuration menu
    Copy the full SHA
    8b03dd0 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into iterable-dataset-state-dict

    @lhoestq
    lhoestq committed May 30, 2024
    Configuration menu
    Copy the full SHA
    c323af0 View commit details
    Browse the repository at this point in the history

  5. fix iter_arrow resuming

    @lhoestq
    lhoestq committed May 30, 2024
    Configuration menu
    Copy the full SHA
    c35a036 View commit details
    Browse the repository at this point in the history

  6. tests

    @lhoestq
    lhoestq committed May 30, 2024
    Configuration menu
    Copy the full SHA
    bca57c5 View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. fix map and filter resuming

    @lhoestq
    lhoestq committed May 31, 2024
    Configuration menu
    Copy the full SHA
    defc676 View commit details
    Browse the repository at this point in the history

  2. fix spark

    @lhoestq
    lhoestq committed May 31, 2024
    Configuration menu
    Copy the full SHA
    5e96474 View commit details
    Browse the repository at this point in the history

  3. mark spark resuming as experimental

    @lhoestq
    lhoestq committed May 31, 2024
    Configuration menu
    Copy the full SHA
    d6d83f1 View commit details
    Browse the repository at this point in the history

Commits on Jun 3, 2024

  1. docs

    @lhoestq
    lhoestq committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    11e4778 View commit details
    Browse the repository at this point in the history

  2. fix docs

    @lhoestq
    lhoestq committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    3060d39 View commit details
    Browse the repository at this point in the history

  3. docs

    @lhoestq
    lhoestq committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    cc3cf8b View commit details
    Browse the repository at this point in the history

  4. add note

    @lhoestq
    lhoestq committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    efa101a View commit details
    Browse the repository at this point in the history

  5. add to docs too

    @lhoestq
    lhoestq committed Jun 3, 2024
    Configuration menu
    Copy the full SHA
    698585d View commit details
    Browse the repository at this point in the history