MPS (Mac M1) device support #13102

carmocca · 2022-05-18T16:00:22Z

🚀 Feature

https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/
Docs: https://pytorch.org/docs/master/notes/mps.html

If you enjoy Lightning, check out our other projects! ⚡

Metrics: Machine learning metrics for distributed, scalable PyTorch applications.
Lite: enables pure PyTorch users to scale their existing code on any kind of device while retaining full control over their own loops and optimization logic.
Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, fine-tuning, and solving problems with deep learning.
Bolts: Pretrained SOTA Deep Learning models, callbacks, and more for research and production with PyTorch Lightning and PyTorch.
Lightning Transformers: Flexible interface for high-performance research using SOTA Transformers leveraging Pytorch Lightning, Transformers, and Hydra.

cc @Borda @akihironitta @rohitgr7 @justusschock

carmocca · 2022-05-18T16:41:09Z

I heard @awaelchli just got an M1 😈

awaelchli · 2022-05-18T16:51:22Z

@justusschock also has one if I remember correctly 😃

justusschock · 2022-05-18T17:05:49Z

I'll investigate on the weekend

carmocca · 2022-05-18T17:19:00Z

After a bit of offline discussion, we thought about this API:

The MPSAccelerator is introduced.
The GPUAccelerator is deprecated and renamed in favor of CUDAAccelerator
2 new accelerator options: Trainer(accelerator="cuda"|"mps") to explicitly choose one of the above.
Trainer(accelerator="gpu") now chooses from the 2 accelerators above based on the available hardware. (side note: this is how we would also support ROCm).
Trainer(accelerator="auto") is different in that it also considers IPUs, TPUs, HPUs...

This shouldn't be a problem because a machine cannot have both accelerators available.

eubinecto · 2022-05-18T23:43:17Z

Just when I thought I wanted to raise an issue! I'm so rooting for this.

ananthsub · 2022-05-19T00:34:44Z

I'm glad we did the accelerator refactor to make supporting new features like this much easier :)
GPUs is ambiguous, whereas the torch.device type is not: #10410 (comment)

scalastic · 2022-05-29T07:59:50Z

Note that, on M1 max, with a basic example like LitAutoEncoder of MNIST (like the one described on PytorchLightning homepage) I get better results without mpssettings:

results

With accelerator="mps" :

GPU available: True (mps), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
...
Epoch 0: | 7936/55000 [01:22<08:06, **96.77it/s**, loss=0.00155, v_num=0]

Without "mps" settings:

GPU available: True (mps), used: False
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
...
Epoch 0:   | 26081/55000 [00:52<00:57, **498.76it/s**, loss=0.0416, v_num=1]

env

My configuration is (numpy is installed with blas=*=*accelerate* in both cases):

% conda list
# packages in environment at /opt/homebrew/Caskroom/miniforge/base/envs/pytorch-lightning-m1:
#
# Name                    Version                   Build  Channel
absl-py                   1.0.0                    pypi_0    pypi
aiohttp                   3.8.1                    pypi_0    pypi
aiosignal                 1.2.0                    pypi_0    pypi
async-timeout             4.0.2                    pypi_0    pypi
attrs                     21.4.0                   pypi_0    pypi
blas                      2.114                accelerate    conda-forge
blas-devel                3.9.0           14_osxarm64_accelerate    conda-forge
bzip2                     1.0.8                h3422bc3_4    conda-forge
ca-certificates           2022.5.18.1          h4653dfc_0    conda-forge
cachetools                5.1.0                    pypi_0    pypi
certifi                   2022.5.18.1              pypi_0    pypi
charset-normalizer        2.0.12                   pypi_0    pypi
frozenlist                1.3.0                    pypi_0    pypi
fsspec                    2022.2.0                 pypi_0    pypi
google-auth               2.6.6                    pypi_0    pypi
google-auth-oauthlib      0.4.6                    pypi_0    pypi
grpcio                    1.46.3                   pypi_0    pypi
idna                      3.3                      pypi_0    pypi
importlib-metadata        4.11.4                   pypi_0    pypi
libblas                   3.9.0           14_osxarm64_accelerate    conda-forge
libcblas                  3.9.0           14_osxarm64_accelerate    conda-forge
libcxx                    14.0.3               h6a5c8ee_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0.dev0      11_0_1_hf114ba7_23    conda-forge
libgfortran5              11.0.1.dev0         hf114ba7_23    conda-forge
liblapack                 3.9.0           14_osxarm64_accelerate    conda-forge
liblapacke                3.9.0           14_osxarm64_accelerate    conda-forge
libprotobuf               3.20.1               h332123e_0    conda-forge
libzlib                   1.2.11            h90dfc92_1014    conda-forge
llvm-openmp               14.0.4               hd125106_0    conda-forge
markdown                  3.3.7                    pypi_0    pypi
multidict                 6.0.2                    pypi_0    pypi
ncurses                   6.3                  h07bb92c_1    conda-forge
numpy                     1.22.3                   pypi_0    pypi
oauthlib                  3.2.0                    pypi_0    pypi
openssl                   3.0.3                ha287fd2_0    conda-forge
packaging                 21.3                     pypi_0    pypi
pandas                    1.4.2                    pypi_0    pypi
pillow                    9.1.1                    pypi_0    pypi
pip                       22.1.1             pyhd8ed1ab_0    conda-forge
protobuf                  4.21.0                   pypi_0    pypi
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pydeprecate               0.3.2                    pypi_0    pypi
pyparsing                 3.0.9                    pypi_0    pypi
python                    3.8.13          hd3575e6_0_cpython    conda-forge
python-dateutil           2.8.2                    pypi_0    pypi
python_abi                3.8                      2_cp38    conda-forge
pytorch-lightning         1.7.0.dev0                dev_0    <develop>
pytz                      2022.1                   pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
readline                  8.1                  hedafd6a_0    conda-forge
requests                  2.27.1                   pypi_0    pypi
requests-oauthlib         1.3.1                    pypi_0    pypi
rsa                       4.8                      pypi_0    pypi
setuptools                62.3.2           py38h10201cd_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sqlite                    3.38.5               h40dfcc0_0    conda-forge
tensorboard               2.8.0                    pypi_0    pypi
tensorboard-data-server   0.6.1                    pypi_0    pypi
tensorboard-plugin-wit    1.8.1                    pypi_0    pypi
tk                        8.6.12               he1e0b03_0    conda-forge
torch                     1.13.0.dev20220525          pypi_0    pypi
torch-tb-profiler         0.4.0                    pypi_0    pypi
torchmetrics              0.7.2                    pypi_0    pypi
torchvision               0.12.0                   pypi_0    pypi
tqdm                      4.63.0                   pypi_0    pypi
typing-extensions         4.1.1                    pypi_0    pypi
urllib3                   1.26.9                   pypi_0    pypi
werkzeug                  2.1.2                    pypi_0    pypi
wheel                     0.37.1             pyhd8ed1ab_0    conda-forge
xz                        5.2.5                h642e427_1    conda-forge
yarl                      1.7.2                    pypi_0    pypi
zipp                      3.8.0                    pypi_0    pypi
zlib                      1.2.11            h90dfc92_1014    conda-forge

awaelchli · 2022-05-29T16:28:35Z

We could think about adding M1 in GitHub actions / self-host it.

paantya · 2022-05-29T16:53:00Z

Is training With accelerator="mps" slower than without it?

scalastic · 2022-05-29T20:35:10Z

@paantya In my case, yes it is.

With mps accelerator: 96.77it/s, GPU working at 100%, CPU at 30% approximately
Without mps: 498.76it/s, GPU 25%, CPU 30% also approximately

babaniyi · 2022-05-31T17:00:30Z

@carmocca How do I resolve pytorch_lightning not working on Mac M1 but torch and torchvision are working?
After installing via import pytorch_lightning as pl I get the below error:

TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

awaelchli · 2022-05-31T17:19:55Z

@babaniyi I think you need to downgrade protobuf to < 4.21.0, e.g., 3.20

scalastic · 2022-05-31T18:14:32Z

@babaniyi Here are the commands I use to configure a PyTorchLightning env for Apple Silicon.

Run at the root level of the PyTorchLightning repository on mps_accelerator branch:

% conda create -n pytorch-lightning-m1
% conda activate pytorch-lightning-m1
% conda install python=3.8
% pip install --upgrade --force-reinstall --pre torch --extra-index-url https://download.pytorch.org/whl/nightly/cpu
% pip install -r requirements.txt
% pip install -e .
% pip install --pre torchvision --extra-index-url https://download.pytorch.org/whl/nightly/cpu --force-reinstall
% conda install -c conda-forge protobuf=3.20

Hope this helps

babaniyi · 2022-05-31T19:12:17Z

@awaelchli @scalastic
Thanks for your suggestions, they both work.

carmocca added feature Is an improvement or enhancement accelerator labels May 18, 2022

carmocca added this to the 1.7 milestone May 18, 2022

johnnynunez mentioned this issue May 18, 2022

Support M1 ARM ashleve/lightning-hydra-template#302

Closed

johnnynunez mentioned this issue May 18, 2022

Support M1 ARM facebookresearch/hydra#2220

Closed

carmocca assigned justusschock May 18, 2022

eubinecto mentioned this issue May 18, 2022

Pytorch - m1 가속을 지원한다! the-clean-rnns에 적용해볼 수 있을까? eubinecto/train-of-thoughts#34

Closed

stweil mentioned this issue May 20, 2022

[feature request] Training with Metal Performance Shaders mittagessen/kraken#358

Open

ananthsub mentioned this issue May 20, 2022

PyTorch supports Metal Performance Shader as an accelerator now, can pytorch-lightning also add support for that? #13119

Closed

This was referenced May 21, 2022

MPS Accelerator #13123

Merged

Rename min_gpus to min_cuda_gpus #13133

Merged

This comment was marked as duplicate.

Sign in to view

justusschock mentioned this issue Jun 14, 2022

MPS Inf/Nan Loss #13285

Closed

This was referenced Jul 13, 2022

Rename GPUAccelerator to CUDAAccelerator #13636

Merged

Merge different gpu backends with accelerator='gpu' #13642

Merged

justusschock closed this as completed in #13642 Jul 25, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MPS (Mac M1) device support #13102

MPS (Mac M1) device support #13102

carmocca commented May 18, 2022 •

edited

Loading

carmocca commented May 18, 2022 •

edited

Loading

awaelchli commented May 18, 2022

justusschock commented May 18, 2022

carmocca commented May 18, 2022

eubinecto commented May 18, 2022

ananthsub commented May 19, 2022

scalastic commented May 29, 2022 •

edited by akihironitta

Loading

awaelchli commented May 29, 2022

paantya commented May 29, 2022

scalastic commented May 29, 2022

babaniyi commented May 31, 2022 •

edited

Loading

awaelchli commented May 31, 2022

scalastic commented May 31, 2022

babaniyi commented May 31, 2022

This comment was marked as duplicate.

MPS (Mac M1) device support #13102

MPS (Mac M1) device support #13102

Comments

carmocca commented May 18, 2022 • edited Loading

🚀 Feature

If you enjoy Lightning, check out our other projects! ⚡

carmocca commented May 18, 2022 • edited Loading

awaelchli commented May 18, 2022

justusschock commented May 18, 2022

carmocca commented May 18, 2022

eubinecto commented May 18, 2022

ananthsub commented May 19, 2022

scalastic commented May 29, 2022 • edited by akihironitta Loading

awaelchli commented May 29, 2022

paantya commented May 29, 2022

scalastic commented May 29, 2022

babaniyi commented May 31, 2022 • edited Loading

awaelchli commented May 31, 2022

scalastic commented May 31, 2022

babaniyi commented May 31, 2022

This comment was marked as duplicate.

carmocca commented May 18, 2022 •

edited

Loading

carmocca commented May 18, 2022 •

edited

Loading

scalastic commented May 29, 2022 •

edited by akihironitta

Loading

babaniyi commented May 31, 2022 •

edited

Loading