Weekly patch release v1.7.2 (#14126)

* update version and changelog for 1.7.2 release * Reset all results on epoch end (#14061) Co-authored-by: Carlos Mocholí <[email protected]> * Skip ddp fork tests on windows (#14121) * Fix device placement when `.cuda()` called without specifying index (#14128) * Convert subprocess test to standalone test (#14101) * Fix entry point test for Python 3.10 (#14154) * Fix flaky test caused by weak reference (#14157) * Fix saving hyperparameters in a composition where parent is not a LM or LDM (#14151) Co-authored-by: Rohit Gupta <[email protected]> * Remove DeepSpeed version restriction from Lite (#13967) * Configure the check-group app (#14165) Co-authored-by: Jirka <[email protected]> * Update onnxruntime requirement from <=1.12.0 to <1.13.0 in /requirements (#14083) Updates the requirements on [onnxruntime](https://github.com/microsoft/onnxruntime) to permit the latest version. - [Release notes](https://github.com/microsoft/onnxruntime/releases) - [Changelog](https://github.com/microsoft/onnxruntime/blob/master/docs/ReleaseManagement.md) - [Commits](microsoft/onnxruntime@v0.1.4...v1.12.1) --- updated-dependencies: - dependency-name: onnxruntime dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update gcsfs requirement from <2022.6.0,>=2021.5.0 to >=2021.5.0,<2022.8.0 in /requirements (#14079) Update gcsfs requirement in /requirements Updates the requirements on [gcsfs](https://github.com/fsspec/gcsfs) to permit the latest version. - [Release notes](https://github.com/fsspec/gcsfs/releases) - [Commits](fsspec/gcsfs@2021.05.0...2022.7.1) --- updated-dependencies: - dependency-name: gcsfs dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix a bug that caused spurious `AttributeError` when multiple `DataLoader` classes are imported (#14117) fix * CI: Replace `_` of in GHA workflow filenames with `-` (#13917) * Rename workflow files * Update docs * Fix azure badges * Update the main readme * bad rebase * Update doc * CI: Update Windows version from 2019 to 2022 (#14129) Update windows * CI/CD: Add CUDA version to docker image tags (#13831) * append cuda version to tags * revertme: push to hub * Update docker readme * Build base-conda-py3.9-torch1.12-cuda11.3.1 * Use new images in conda tests * revertme: push to hub * Revert "revertme: push to hub" This reverts commit 0f7d534. * Revert "revertme: push to hub" This reverts commit 46a05fc. * Run conda if workflow edited * Run gpu testing if workflow edited * Use new tags in release/Dockerfile * Build base-cuda and PL release images with all combinations * Update release docker * Update conda from py3.9-torch1.12 to py3.10-torch.1.12 * Fix ubuntu version * Revert conda * revertme: push to hub * Don't build Python 3.10 for now... * Fix pl release builder * updating version contribute to the error? docker/buildx#456 * Update actions' versions * Update slack user to notify * Don't use 11.6.0 to avoid bagua incompatibility * Don't use 11.1, and use 11.1.1 * Update .github/workflows/ci-pytorch_test-conda.yml Co-authored-by: Luca Medeiros <[email protected]> * Update trigger * Ignore artfacts from tutorials * Trim docker images to distribute * Add an image for tutorials * Update conda image 3.8x1.10 * Try different conda variants * No need to set cuda for conda jobs * Update who to notify ipu failure * Don't push * update filenaem Co-authored-by: Luca Medeiros <[email protected]> * Avoid entry_points deprecation warning (#14052) Co-authored-by: Adam J. Stewart <[email protected]> Co-authored-by: Akihiro Nitta <[email protected]> * Configure the check-group app (#14165) Co-authored-by: Jirka <[email protected]> * Profile batch transfer and gradient clipping hooks (#14069) Co-authored-by: Rohit Gupta <[email protected]> * Avoid false positive warning about using `sync_dist` when using torchmetrics (#14143) Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: Rohit Gupta <[email protected]> * Avoid raising the sampler warning if num_replicas=1 (#14097) Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: Rohit Gupta <[email protected]> Co-authored-by: otaj <[email protected]> * Remove skipping logic in favor of path filtering (#14170) * Support checkpoint save and load with Stochastic Weight Averaging (#9938) Co-authored-by: thomas chaton <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Carlos Mocholi <[email protected]> Co-authored-by: Kushashwa Ravi Shrimali <[email protected]> Co-authored-by: Jirka <[email protected]> Co-authored-by: Rohit Gupta <[email protected]> * Use fsdp module to initialize precision scalar for fsdp native (#14092) Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: Laverne Henderson <[email protected]> Co-authored-by: Rohit Gupta <[email protected]> * add more issues types (#14174) * add more issues types * Update .github/ISSUE_TEMPLATE/config.yml Co-authored-by: Mansy <[email protected]> * typo Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Kaushik B <[email protected]> Co-authored-by: Mansy <[email protected]> Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: Laverne Henderson <[email protected]> Co-authored-by: Akihiro Nitta <[email protected]> * CI: clean building docs (#14216) * CI: clean building docs * group * . * CI: docker focus on PL only (#14246) * CI: docker focus on PL only * group * Allowed setting attributes on `DataLoader` and `BatchSampler` when instantiated inside `*_dataloader` hooks (#14212) Co-authored-by: otaj <[email protected]> * Revert "Remove skipping logic in favor of path filtering (#14170)" (#14244) * Update defaults for WandbLogger's run name and project name (#14145) Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: Rohit Gupta <[email protected]> Co-authored-by: Jirka <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Akihiro Nitta <[email protected]> Co-authored-by: Luca Medeiros <[email protected]> Co-authored-by: Adam J. Stewart <[email protected]> Co-authored-by: otaj <[email protected]> Co-authored-by: Adam Reeve <[email protected]> Co-authored-by: thomas chaton <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Kushashwa Ravi Shrimali <[email protected]> Co-authored-by: Laverne Henderson <[email protected]> Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Kaushik B <[email protected]> Co-authored-by: Mansy <[email protected]>
Lightning-AI · Aug 17, 2022 · 4fae327 · akihironitta · Aug 24, 2022 · awaelchli
1 parent be58159
commit 4fae327
Show file tree

Hide file tree

Showing 75 changed files with 1,212 additions and 583 deletions.
diff --git a/.actions/setup_tools.py b/.actions/setup_tools.py
@@ -94,11 +94,10 @@ def load_readme_description(path_dir: str, homepage: str, version: str) -> str:
     text = text.replace("pytorch-lightning.readthedocs.io/en/stable/", f"pytorch-lightning.readthedocs.io/en/{version}")
     # codecov badge
     text = text.replace("/branch/master/graph/badge.svg", f"/release/{version}/graph/badge.svg")
-    # replace github badges for release ones
+    # github actions badge
     text = text.replace("badge.svg?branch=master&event=push", f"badge.svg?tag={version}")
-    # Azure...
+    # azure pipelines badge
     text = text.replace("?branchName=master", f"?branchName=refs%2Ftags%2F{version}")
-    text = re.sub(r"\?definitionId=\d+&branchName=master", f"?definitionId=2&branchName=refs%2Ftags%2F{version}", text)
 
     skip_begin = r"<!-- following section will be skipped from PyPI description -->"
     skip_end = r"<!-- end skipping PyPI description -->"

diff --git a/.azure/gpu-benchmark.yml b/.azure/gpu-benchmark.yml
@@ -28,7 +28,7 @@ jobs:
     cancelTimeoutInMinutes: "2"
     pool: azure-jirka-spot
     container:
-      image: "pytorchlightning/pytorch_lightning:base-cuda-py3.9-torch1.12"
+      image: "pytorchlightning/pytorch_lightning:base-cuda-py3.9-torch1.12-cuda11.3.1"
       options: "--runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all --shm-size=32g"
     workspace:
       clean: all

diff --git a/.azure/gpu-tests.yml b/.azure/gpu-tests.yml
@@ -26,7 +26,7 @@ jobs:
     strategy:
       matrix:
         'PyTorch - stable':
-          image: "pytorchlightning/pytorch_lightning:base-cuda-py3.9-torch1.12"
+          image: "pytorchlightning/pytorch_lightning:base-cuda-py3.9-torch1.12-cuda11.3.1"
     # how long to run the job before automatically cancelling
     timeoutInMinutes: "80"
     # how much time to give 'run always even if cancelled tasks' before stopping them
@@ -44,7 +44,7 @@ jobs:
 
     - bash: |
         CHANGED_FILES=$(git diff --name-status origin/master -- . | awk  '{print $2}')
-        FILTER='src/pytorch_lightning|requirements/pytorch|tests/tests_pytorch|examples/pl_*|.azure/*'
+        FILTER='src/pytorch_lightning|requirements/pytorch|tests/tests_pytorch|examples/pl_*'
         echo $CHANGED_FILES > changed_files.txt
         MATCHES=$(cat changed_files.txt | grep -E $FILTER)
         echo $MATCHES
@@ -75,7 +75,7 @@ jobs:
         CUDA_VERSION_MM=$(python -c "import torch ; print(''.join(map(str, torch.version.cuda.split('.')[:2])))")
         pip install "bagua-cuda$CUDA_VERSION_MM>=0.9.0"
         pip install -e .[strategies]
-        pip install deepspeed>0.6.4  # TODO: remove when docker images are upgraded
+        pip install -U deepspeed  # TODO: remove when docker images are upgraded
         pip install --requirement requirements/pytorch/devel.txt
         pip list
       env:
@@ -119,15 +119,6 @@ jobs:
       timeoutInMinutes: "35"
       condition: eq(variables['continue'], '1')
 
-    - bash: bash run_standalone_tasks.sh
-      workingDirectory: tests/tests_pytorch
-      env:
-        PL_USE_MOCKED_MNIST: "1"
-        PL_RUN_CUDA_TESTS: "1"
-      displayName: 'Testing: PyTorch standalone tasks'
-      timeoutInMinutes: "10"
-      condition: eq(variables['continue'], '1')
-
     - bash: |
         python -m coverage report
         python -m coverage xml

diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml
@@ -1,8 +1,14 @@
 blank_issues_enabled: false
 contact_links:
-  - name: Ask a Question
+  - name: ❓ Ask a Question
     url: https://github.com/Lightning-AI/lightning/discussions/new
-    about: Ask and answer Lightning related questions
-  - name: 💬 Slack
+    about: Ask and answer Lightning related questions.
+  - name: 💬 Chat with us
     url: https://www.pytorchlightning.ai/community
-    about: Chat with our community
+    about: Live chat with experts, engineers, and users in our Slack community.
+  - name: 📖 Read the documentation
+    url: https://lightning.ai/lightning-docs/
+    about: Please consult the documentation before opening any issues!
+  - name: 🙋 Contact us about professional services
+    url: https://lightning.ai
+    about: Contact the Lightning.ai sales team for paid support.
diff --git a/.github/checkgroup.yml b/.github/checkgroup.yml
@@ -0,0 +1,173 @@
+custom_service_name: "Lightning CI required checker"
+subprojects:
+  - id: "CI: CircleCI"
+    paths:
+      - ".circleci/**"
+    checks:
+      - "test-on-tpus"
+
+  - id: "CI: Azure"
+    paths:
+      - ".azure/**"
+    checks:
+      - "pytorch-lightning (GPUs)"
+      - "pytorch-lightning (GPUs) (testing PyTorch - stable)"
+      - "pytorch-lightning (HPUs)"
+      - "pytorch-lightning (IPUs)"
+
+  - id: "pytorch_lightning"
+    paths:
+      # all examples don't need to be added because they aren't used in CI, but these are
+      - "examples/run_ddp_examples.sh"
+      - "examples/convert_from_pt_to_pl/**"
+      - "examples/run_pl_examples.sh"
+      - "examples/pl_basics/backbone_image_classifier.py"
+      - "examples/pl_basics/autoencoder.py"
+      - "examples/pl_loops/mnist_lite.py"
+      - "examples/pl_fault_tolerant/automatic.py"
+      - "examples/test_pl_examples.py"
+      - "examples/pl_integrations/dali_image_classifier.py"
+      - "requirements/pytorch/**"
+      - "src/pytorch_lightning/**"
+      - "tests/tests_pytorch/**"
+      - "setup.cfg"  # includes pytest config
+      - ".github/workflows/ci-pytorch*.yml"
+      - ".github/workflows/docs-*.yml"
+    checks:
+      - "conda (3.8, 1.10)"
+      - "conda (3.8, 1.9)"
+      - "conda (3.9, 1.11)"
+      - "conda (3.9, 1.12)"
+      - "cpu (macOS-11, 3.10, latest, stable)"
+      - "cpu (macOS-11, 3.7, latest, stable)"
+      - "cpu (macOS-11, 3.7, oldest, stable)"
+      - "cpu (ubuntu-20.04, 3.10, latest, stable)"
+      - "cpu (ubuntu-20.04, 3.7, latest, stable)"
+      - "cpu (ubuntu-20.04, 3.7, oldest, stable)"
+      - "cpu (windows-2022, 3.10, latest, stable)"
+      - "cpu (windows-2022, 3.7, latest, stable)"
+      - "cpu (windows-2022, 3.7, oldest, stable)"
+      - "doctest (pytorch)"
+      - "make-docs (pytorch)"
+      - "mypy"
+      - "PR Gatekeeper (pytorch)"
+      - "pytorch-lightning (GPUs)"
+      - "pytorch-lightning (GPUs) (testing PyTorch - stable)"
+      - "pytorch-lightning (HPUs)"
+      - "pytorch-lightning (IPUs)"
+      - "slow (macOS-11, 3.7, 1.11)"
+      - "slow (ubuntu-20.04, 3.7, 1.11)"
+      - "slow (windows-2022, 3.7, 1.11)"
+      - "test-on-tpus"
+
+  - id: "pytorch_lightning: Docs"
+    paths:
+      - "docs/source-pytorch/**"
+      - ".github/workflows/docs-*.yml"
+      - "requirements/docs.txt"
+      - "requirements/pytorch/**"
+    checks:
+      - "doctest (pytorch)"
+      - "make-docs (pytorch)"
+
+  - id: "pytorch_lightning: Docker"
+    paths:
+      - "dockers/**"
+      - "!dockers/README.md"
+      - "requirements.txt"
+      - "requirements/*.txt"
+      - "requirements/pytorch/*"
+      - "environment.yml"
+      - ".github/workflows/*docker*.yml"
+      - "setup.py"
+    checks:
+      - "build-conda (3.8, 1.10)"
+      - "build-conda (3.8, 1.9)"
+      - "build-conda (3.9, 1.11)"
+      - "build-conda (3.9, 1.12)"
+      - "build-cuda (3.8, 1.9, 11.1.1)"
+      - "build-cuda (3.9, 1.10, 11.3.1)"
+      - "build-cuda (3.9, 1.11, 11.3.1)"
+      - "build-cuda (3.9, 1.12, 11.3.1)"
+      - "build-cuda (3.9, 1.9, 11.1.1)"
+      - "build-hpu (1.5.0, 1.11.0)"
+      - "build-ipu (3.9, 1.9)"
+      - "build-NGC"
+      - "build-pl (3.9, 1.10, 11.3.1)"
+      - "build-pl (3.9, 1.11, 11.3.1)"
+      - "build-pl (3.9, 1.12, 11.3.1)"
+      - "build-pl (3.9, 1.9, 11.1.1)"
+      - "build-xla (3.7, 1.12)"
+
+  - id: "pytorch_lightning: mypy"
+    paths:
+      - ".github/workflows/code-checks.yml"
+      - "pyproject.toml"  # includes mypy config
+    checks:
+      - "mypy"
+
+  - id: "lightning_app"
+    paths:
+      - ".github/workflows/ci-app*.yml"
+      - "requirements/app/**"
+      - "src/lightning_app/**"
+      - "tests/tests_app/**"
+      - "tests/tests_app_examples/**"
+      - "tests/tests_clusters/**"
+      # the examples are used in the app CI
+      - "examples/app_*"
+    checks:
+      - "Cloud Test (boring_app)"
+      - "Cloud Test (collect_failures)"
+      - "Cloud Test (commands_and_api)"
+      - "Cloud Test (custom_work_dependencies)"
+      - "Cloud Test (drive)"
+      - "Cloud Test (idle_timeout)"
+      - "Cloud Test (payload)"
+      - "Cloud Test (template_jupyterlab)"
+      - "Cloud Test (template_react_ui)"
+      - "Cloud Test (template_streamlit_ui)"
+      - "Cloud Test (v0_app)"
+      - "doctest (app)"
+      - "make-docs (app)"
+      - "pytest (macOS-11, 3.8, latest)"
+      - "pytest (macOS-11, 3.8, oldest)"
+      - "pytest (ubuntu-20.04, 3.8, latest)"
+      - "pytest (ubuntu-20.04, 3.8, oldest)"
+      - "pytest (windows-2022, 3.8, latest)"
+      - "pytest (windows-2022, 3.8, oldest)"
+
+  - id: "lightning_app: Docs"
+    paths:
+      - "docs/source-app/**"
+      - ".github/workflows/docs-*.yml"
+      - "requirements/docs.txt"
+      - "requirements/app/**"
+    checks:
+      - "doctest (app)"
+      - "make-docs (app)"
+
+  - id: "install"
+    paths:
+      - ".actions/setup_tools.py"
+      - ".github/workflows/ci-pkg-install.yml"
+      - "setup.py"
+      - "src/lightning/**"
+      # all __about__, __version__, __setup__
+      - "src/*/__*.py"
+    checks:
+      - "install-meta-pypi (macOS-11, 3.8)"
+      - "install-meta-pypi (ubuntu-20.04, 3.8)"
+      - "install-meta-pypi (windows-2022, 3.8)"
+      - "install-meta-src (macOS-11, 3.8)"
+      - "install-meta-src (macOS-11, lightning, 3.8)"
+      - "install-meta-src (ubuntu-20.04, 3.8)"
+      - "install-meta-src (ubuntu-20.04, lightning, 3.8)"
+      - "install-meta-src (windows-2022, 3.8)"
+      - "install-meta-src (windows-2022, lightning, 3.8)"
+      - "install-standalone (macOS-11, app, 3.8)"
+      - "install-standalone (macOS-11, pytorch, 3.8)"
+      - "install-standalone (ubuntu-20.04, app, 3.8)"
+      - "install-standalone (ubuntu-20.04, pytorch, 3.8)"
+      - "install-standalone (windows-2022, app, 3.8)"
+      - "install-standalone (windows-2022, pytorch, 3.8)"