CI: enable testing with coming PT 2.2 #19289

Borda · 2024-01-15T19:08:56Z

What does this PR do?

building images for new PT
upload the new images
use these images in CI
if all pass ad it to GC

Before submitting

Was this discussed/agreed via a GitHub issue? (not for typos and docs)
Did you read the contributor guideline, Pull Request section?
Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

Is this pull request ready for review? (if not, please submit in draft mode)
Check that all items from Before submitting are resolved
Make sure the title is self-explanatory and the description concisely explains the PR
Add labels and milestones (and optionally projects) to the PR so it can be classified

📚 Documentation preview 📚: https://pytorch-lightning--19289.org.readthedocs.build/en/19289/

cc @Borda @carmocca @justusschock @awaelchli

github-actions · 2024-01-15T19:09:16Z

⚡ Required checks status: All passing 🟢

Groups summary

🟢 pytorch_lightning: Tests workflow

Check ID	Status
pl-cpu (macOS-11, lightning, 3.8, 1.12, oldest)	success	✅
pl-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
pl-cpu (macOS-11, lightning, 3.10, 2.1)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.8, 1.12, oldest)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
pl-cpu (ubuntu-20.04, lightning, 3.10, 2.1)	success	✅
pl-cpu (windows-2022, lightning, 3.8, 1.12, oldest)	success	✅
pl-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
pl-cpu (windows-2022, lightning, 3.10, 2.1)	success	✅
pl-cpu (macOS-11, pytorch, 3.8, 1.13)	success	✅
pl-cpu (ubuntu-20.04, pytorch, 3.8, 1.13)	success	✅
pl-cpu (windows-2022, pytorch, 3.8, 1.13)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.0)	success	✅
pl-cpu (macOS-12, pytorch, 3.11, 2.1)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.0)	success	✅
pl-cpu (ubuntu-22.04, pytorch, 3.11, 2.1)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.0)	success	✅
pl-cpu (windows-2022, pytorch, 3.11, 2.1)	success	✅

These checks are required after the changes to .github/workflows/ci-tests-pytorch.yml, requirements/fabric/base.txt, requirements/pytorch/base.txt, requirements/pytorch/extra.txt, tests/tests_pytorch/callbacks/test_spike.py, tests/tests_pytorch/loops/test_prediction_loop.py, tests/tests_pytorch/models/test_amp.py, tests/tests_pytorch/serve/test_servable_module_validator.py, tests/tests_pytorch/strategies/launchers/test_multiprocessing.py, tests/tests_pytorch/strategies/test_fsdp.py, tests/tests_pytorch/trainer/connectors/test_data_connector.py, tests/tests_pytorch/trainer/logging_/test_train_loop_logging.py.

🟢 pytorch_lightning: Azure GPU

Check ID	Status
pytorch-lightning (GPUs) (testing Lightning \| latest)	success	✅
pytorch-lightning (GPUs) (testing PyTorch \| latest)	success	✅

These checks are required after the changes to .azure/gpu-tests-pytorch.yml, requirements/pytorch/base.txt, requirements/pytorch/extra.txt, tests/tests_pytorch/callbacks/test_spike.py, tests/tests_pytorch/loops/test_prediction_loop.py, tests/tests_pytorch/models/test_amp.py, tests/tests_pytorch/serve/test_servable_module_validator.py, tests/tests_pytorch/strategies/launchers/test_multiprocessing.py, tests/tests_pytorch/strategies/test_fsdp.py, tests/tests_pytorch/trainer/connectors/test_data_connector.py, tests/tests_pytorch/trainer/logging_/test_train_loop_logging.py, requirements/fabric/base.txt.

🟢 pytorch_lightning: Benchmarks

Check ID	Status
lightning.Benchmarks	success	✅

These checks are required after the changes to requirements/fabric/base.txt, requirements/pytorch/base.txt, requirements/pytorch/extra.txt.

🟢 fabric: Docs

Check ID	Status
docs-make (fabric, doctest)	success	✅
docs-make (fabric, html)	success	✅

These checks are required after the changes to requirements/fabric/base.txt.

🟢 pytorch_lightning: Docs

Check ID	Status
docs-make (pytorch, doctest)	success	✅
docs-make (pytorch, html)	success	✅

These checks are required after the changes to requirements/pytorch/base.txt, requirements/pytorch/extra.txt.

🟢 pytorch_lightning: Docker

Check ID	Status
build-cuda (3.9, 1.12, 11.7.1)	success	✅
build-cuda (3.9, 1.13, 11.8.0)	success	✅
build-cuda (3.9, 1.13, 12.0.1)	success	✅
build-cuda (3.10, 2.0, 11.8.0)	success	✅
build-cuda (3.10, 2.1, 12.1.0)	success	✅
build-pl (3.9, 1.12, 11.7.1)	success	✅
build-pl (3.9, 1.13, 11.8.0)	success	✅
build-pl (3.9, 1.13, 12.0.1)	success	✅
build-pl (3.10, 2.0, 11.8.0)	success	✅
build-pl (3.10, 2.1, 12.1.0)	success	✅

These checks are required after the changes to .github/workflows/docker-build.yml, dockers/base-cuda/Dockerfile, requirements/pytorch/base.txt, requirements/pytorch/extra.txt, requirements/fabric/base.txt.

🟢 lightning_fabric: CPU workflow

Check ID	Status
fabric-cpu (macOS-11, lightning, 3.8, 1.12, oldest)	success	✅
fabric-cpu (macOS-11, lightning, 3.9, 1.12)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 1.13)	success	✅
fabric-cpu (macOS-11, lightning, 3.10, 2.0)	success	✅
fabric-cpu (macOS-11, lightning, 3.11, 2.1)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.8, 1.12, oldest)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.9, 1.12)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.10, 2.0)	success	✅
fabric-cpu (ubuntu-20.04, lightning, 3.11, 2.1)	success	✅
fabric-cpu (windows-2022, lightning, 3.8, 1.12, oldest)	success	✅
fabric-cpu (windows-2022, lightning, 3.9, 1.12)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 1.13)	success	✅
fabric-cpu (windows-2022, lightning, 3.10, 2.0)	success	✅
fabric-cpu (windows-2022, lightning, 3.11, 2.1)	success	✅
fabric-cpu (macOS-11, fabric, 3.8, 1.13)	success	✅
fabric-cpu (ubuntu-20.04, fabric, 3.8, 1.13)	success	✅
fabric-cpu (windows-2022, fabric, 3.8, 1.13)	success	✅
fabric-cpu (macOS-12, fabric, 3.11, 2.0)	success	✅
fabric-cpu (macOS-12, fabric, 3.11, 2.1)	success	✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.0)	success	✅
fabric-cpu (ubuntu-22.04, fabric, 3.11, 2.1)	success	✅
fabric-cpu (windows-2022, fabric, 3.11, 2.0)	success	✅
fabric-cpu (windows-2022, fabric, 3.11, 2.1)	success	✅

These checks are required after the changes to requirements/fabric/base.txt, tests/tests_fabric/plugins/precision/test_amp_integration.py, tests/tests_fabric/strategies/launchers/test_multiprocessing_integration.py, tests/tests_fabric/strategies/test_ddp_integration.py, tests/tests_fabric/strategies/test_fsdp_integration.py, tests/tests_fabric/utilities/test_distributed.py, tests/tests_fabric/utilities/test_spike.py, .github/workflows/ci-tests-fabric.yml.

🟢 lightning_fabric: Azure GPU

Check ID	Status
lightning-fabric (GPUs) (testing Fabric \| latest)	success	✅
lightning-fabric (GPUs) (testing Lightning \| latest)	success	✅

These checks are required after the changes to .azure/gpu-tests-fabric.yml, requirements/fabric/base.txt, tests/tests_fabric/plugins/precision/test_amp_integration.py, tests/tests_fabric/strategies/launchers/test_multiprocessing_integration.py, tests/tests_fabric/strategies/test_ddp_integration.py, tests/tests_fabric/strategies/test_fsdp_integration.py, tests/tests_fabric/utilities/test_distributed.py, tests/tests_fabric/utilities/test_spike.py.

🟢 lightning_app: Tests workflow

Check ID	Status
app-pytest (macOS-11, lightning, 3.8, latest)	success	✅
app-pytest (macOS-11, lightning, 3.8, oldest)	success	✅
app-pytest (macOS-11, app, 3.9, latest)	success	✅
app-pytest (macOS-12, app, 3.11, latest)	success	✅
app-pytest (ubuntu-20.04, lightning, 3.8, latest)	success	✅
app-pytest (ubuntu-20.04, lightning, 3.8, oldest)	success	✅
app-pytest (ubuntu-20.04, app, 3.9, latest)	success	✅
app-pytest (ubuntu-22.04, app, 3.11, latest)	success	✅
app-pytest (windows-2022, lightning, 3.8, latest)	success	✅
app-pytest (windows-2022, lightning, 3.8, oldest)	success	✅
app-pytest (windows-2022, app, 3.8, latest)	success	✅
app-pytest (windows-2022, app, 3.11, latest)	success	✅

These checks are required after the changes to requirements/app/app.txt.

🟢 lightning_app: Examples

Check ID	Status
app-examples (macOS-11, lightning, 3.9, latest)	success	✅
app-examples (macOS-11, lightning, 3.9, oldest)	success	✅
app-examples (macOS-11, app, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, lightning, 3.9, latest)	success	✅
app-examples (ubuntu-20.04, lightning, 3.9, oldest)	success	✅
app-examples (ubuntu-20.04, app, 3.9, latest)	success	✅
app-examples (windows-2022, lightning, 3.9, latest)	success	✅
app-examples (windows-2022, lightning, 3.9, oldest)	success	✅
app-examples (windows-2022, app, 3.9, latest)	success	✅

These checks are required after the changes to requirements/app/app.txt.

🟢 lightning_app: Docs

Check ID	Status
docs-make (app, doctest)	success	✅
docs-make (app, html)	success	✅

These checks are required after the changes to requirements/app/app.txt.

🟢 mypy

Check ID	Status
mypy	success	✅

These checks are required after the changes to requirements/app/app.txt, requirements/fabric/base.txt, requirements/pytorch/base.txt, requirements/pytorch/extra.txt.

🟢 install

Check ID	Status
install-pkg (ubuntu-22.04, app, 3.8)	success	✅
install-pkg (ubuntu-22.04, app, 3.11)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.8)	success	✅
install-pkg (ubuntu-22.04, fabric, 3.11)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.8)	success	✅
install-pkg (ubuntu-22.04, pytorch, 3.11)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.8)	success	✅
install-pkg (ubuntu-22.04, lightning, 3.11)	success	✅
install-pkg (ubuntu-22.04, notset, 3.8)	success	✅
install-pkg (ubuntu-22.04, notset, 3.11)	success	✅
install-pkg (macOS-12, app, 3.8)	success	✅
install-pkg (macOS-12, app, 3.11)	success	✅
install-pkg (macOS-12, fabric, 3.8)	success	✅
install-pkg (macOS-12, fabric, 3.11)	success	✅
install-pkg (macOS-12, pytorch, 3.8)	success	✅
install-pkg (macOS-12, pytorch, 3.11)	success	✅
install-pkg (macOS-12, lightning, 3.8)	success	✅
install-pkg (macOS-12, lightning, 3.11)	success	✅
install-pkg (macOS-12, notset, 3.8)	success	✅
install-pkg (macOS-12, notset, 3.11)	success	✅
install-pkg (windows-2022, app, 3.8)	success	✅
install-pkg (windows-2022, app, 3.11)	success	✅
install-pkg (windows-2022, fabric, 3.8)	success	✅
install-pkg (windows-2022, fabric, 3.11)	success	✅
install-pkg (windows-2022, pytorch, 3.8)	success	✅
install-pkg (windows-2022, pytorch, 3.11)	success	✅
install-pkg (windows-2022, lightning, 3.8)	success	✅
install-pkg (windows-2022, lightning, 3.11)	success	✅
install-pkg (windows-2022, notset, 3.8)	success	✅
install-pkg (windows-2022, notset, 3.11)	success	✅

These checks are required after the changes to requirements/app/app.txt, requirements/fabric/base.txt, requirements/pytorch/base.txt, requirements/pytorch/extra.txt.

Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

codecov · 2024-01-15T22:46:48Z

Codecov Report

Merging #19289 (de66d77) into master (ee9f17e) will decrease coverage by 7%.
The diff coverage is n/a.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #19289     +/-   ##
=========================================
- Coverage      83%      77%     -7%     
=========================================
  Files         448      448             
  Lines       37877    37877             
=========================================
- Hits        31620    29106   -2514     
- Misses       6257     8771   +2514

gitguardian · 2024-01-16T09:24:20Z

⚠️ GitGuardian has uncovered 4 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	Secret	Commit	Filename
5685611	Generic High Entropy Secret	`ab6ad5a`	tests/tests_data/streaming/test_resolver.py	View secret
5685611	Generic High Entropy Secret	`ab6ad5a`	tests/tests_data/streaming/test_resolver.py	View secret
5685611	Generic High Entropy Secret	`ab6ad5a`	tests/tests_data/streaming/test_resolver.py	View secret
5685611	Generic High Entropy Secret	`ab6ad5a`	tests/tests_data/streaming/test_resolver.py	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secret safely. Learn here the best practices.
Revoke and rotate this secret.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!}

Borda · 2024-01-16T10:25:43Z

@awaelchli what is your suggestion about

E       ValueError: Does not validate against any of the Union subtypes
E       Subtypes: (<class 'torch.optim.lr_scheduler.StepLR'>, <class 'torch.optim.lr_scheduler.ExponentialLR'>)
E       Errors:
E         - Problem with given class_path 'torch.optim.lr_scheduler.StepLR':
E             Parser key "verbose":
E               Expected a <class 'bool'>. Got value: deprecated
E         - Import path torch.optim.lr_scheduler.StepLR does not correspond to a subclass of <class 'torch.optim.lr_scheduler.ExponentialLR'>
E       Given value type: <class 'jsonargparse._namespace.Namespace'>
E       Given value: Namespace(class_path='torch.optim.lr_scheduler.StepLR', init_args=Namespace(step_size=50))

awaelchli · 2024-01-25T14:44:18Z

I'm also lost. What's the difference local vs here in the CI 🤔 Tested in clean env with Python 3.10, PyTorch 2.2 and jsonargparse from main, no issues.

carmocca · 2024-01-25T15:18:01Z

Could it be test interaction? Did you run all tests?

You can SSH into the workflow by adding this

    - name: Setup tmate session
      uses: mxschmitt/action-tmate@v3

to try things out in CI directly

awaelchli · 2024-01-25T22:41:24Z

Ok this seems to be just a stupid installation problem. I did what @carmocca said and logged in via SSH into the job. Running the tests there failed, but when I manually cloned jsonargparse and ran pip install -e .[signatures] then the tests started passing. So I think as soon as @mauvilsa will release the next version, we will be fine.

.github/workflows/ci-tests-pytorch.yml

mauvilsa · 2024-01-26T10:01:53Z

jsonargparse 4.27.3 is now released.

for more information, see https://pre-commit.ci

awaelchli · 2024-01-26T12:25:51Z

Thank you very much @mauvilsa! Everything is working now 🎉

tests/tests_fabric/plugins/precision/test_amp_integration.py

ci: build dockers for PT 2.2

5c95b0c

Borda added this to the 2.1.x milestone Jan 15, 2024

Borda requested review from carmocca, ethanwharris and justusschock as code owners January 15, 2024 19:08

github-actions bot added ci Continuous Integration dockers labels Jan 15, 2024

Borda added 5 commits January 15, 2024 20:20

py3.12

4f04497

--pre

83bf784

--extra-index-url

f416bad

test

9752202

typing-extensions

309fcc4

Borda requested review from lantiga, tchaton and awaelchli as code owners January 15, 2024 22:31

github-actions bot added fabric lightning.fabric.Fabric app (removed) Generic label for Lightning App package pl Generic label for PyTorch Lightning package dependencies Pull requests that update a dependency file labels Jan 15, 2024

push

93a70df

Borda added 6 commits January 16, 2024 00:44

ci

fd3d7eb

ci

99d2e5d

ci

aeb7d91

ci

32d3f2f

push

e3d3fd5

ci

c122ef1

debug

27b6910

debug

84965f0

awaelchli reviewed Jan 25, 2024

View reviewed changes

.github/workflows/ci-tests-pytorch.yml Show resolved Hide resolved

awaelchli added 2 commits January 26, 2024 12:17

install latest jsonargparse

4a51997

Add windows skips for Fabric

4f73593

mergify bot added the has conflicts label Jan 26, 2024

Merge branch 'master' into ci/pt-2.2

0d0e3db

mergify bot removed the has conflicts label Jan 26, 2024

pre-commit-ci bot and others added 5 commits January 26, 2024 11:43

[pre-commit.ci] auto fixes from pre-commit.com hooks

2ce6ee1

for more information, see https://pre-commit.ci

convert to xfail

b1e74db

add pytorch skips

06a95d0

[pre-commit.ci] auto fixes from pre-commit.com hooks

b0f6868

for more information, see https://pre-commit.ci

skip checkpoint consolidation test

482f056

awaelchli approved these changes Jan 26, 2024

View reviewed changes

set max torch

de66d77

Borda commented Jan 26, 2024

View reviewed changes

tests/tests_fabric/plugins/precision/test_amp_integration.py Show resolved Hide resolved

awaelchli mentioned this pull request Jan 26, 2024

Support consolidating sharded checkpoints in PyTorch >= 2.2 #19351

Closed

carmocca approved these changes Jan 26, 2024

View reviewed changes

mergify bot added the ready PRs ready to be merged label Jan 26, 2024

Borda merged commit 3bd133b into master Jan 26, 2024
163 checks passed

Borda deleted the ci/pt-2.2 branch January 26, 2024 15:42

awaelchli modified the milestones: 2.1.x, 2.2 Jan 31, 2024

awaelchli mentioned this pull request Mar 28, 2024

Add PyTorch 2.3 to CI matrix #19708

Merged

awaelchli mentioned this pull request Jun 24, 2024

Add testing for PyTorch 2.4 (Trainer) #20010

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: enable testing with coming PT 2.2 #19289

CI: enable testing with coming PT 2.2 #19289

Borda commented Jan 15, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Jan 15, 2024 •

edited

Loading

codecov bot commented Jan 15, 2024 •

edited

Loading

gitguardian bot commented Jan 16, 2024 •

edited

Loading

Borda commented Jan 16, 2024

awaelchli commented Jan 25, 2024

carmocca commented Jan 25, 2024

awaelchli commented Jan 25, 2024 •

edited

Loading

mauvilsa commented Jan 26, 2024

awaelchli commented Jan 26, 2024

CI: enable testing with coming PT 2.2 #19289

CI: enable testing with coming PT 2.2 #19289

Conversation

Borda commented Jan 15, 2024 • edited by github-actions bot Loading

What does this PR do?

PR review

github-actions bot commented Jan 15, 2024 • edited Loading

⚡ Required checks status: All passing 🟢

Groups summary

codecov bot commented Jan 15, 2024 • edited Loading

Codecov Report

gitguardian bot commented Jan 16, 2024 • edited Loading

⚠️ GitGuardian has uncovered 4 secrets following the scan of your pull request.

Borda commented Jan 16, 2024

awaelchli commented Jan 25, 2024

carmocca commented Jan 25, 2024

awaelchli commented Jan 25, 2024 • edited Loading

mauvilsa commented Jan 26, 2024

awaelchli commented Jan 26, 2024

Borda commented Jan 15, 2024 •

edited by github-actions bot

Loading

github-actions bot commented Jan 15, 2024 •

edited

Loading

codecov bot commented Jan 15, 2024 •

edited

Loading

gitguardian bot commented Jan 16, 2024 •

edited

Loading

awaelchli commented Jan 25, 2024 •

edited

Loading