Option to install only the apps part of the lightning framework

[KogaiIrina]

🚀 Feature

Option to install only the apps part of the lightning framework. Something along the lines of pip install lightning[apps].

Motivation

I have an unresolvable dependency conflict between the protobuf version required by tensorboard required by lightning and my other dependency.

Pitch

It would be great to have the option to install only the apps part when I don't need the other parts with their dependencies.

Alternatives

N/A

---

If you enjoy Lightning, check out our other projects! ⚡

• Metrics: Machine learning metrics for distributed, scalable PyTorch applications.

• Lite: enables pure PyTorch users to scale their existing code on any kind of device while retaining full control over their own loops and optimization logic.

• Flash: The fastest way to get a Lightning baseline! A collection of tasks for fast prototyping, baselining, fine-tuning, and solving problems with deep learning.

• Bolts: Pretrained SOTA Deep Learning models, callbacks, and more for research and production with PyTorch Lightning and PyTorch.

• Lightning Transformers: Flexible interface for high-performance research using SOTA Transformers leveraging PyTorch Lightning, Transformers, and Hydra.

cc @Borda @Borda @carmocca @otaj @justusschock

[awaelchli]

@KogayIrina Thanks for checking out Lightning Apps!

Afaik we are actually working on this.
You can already install only the apps part today, by installing the package lightning-app from PyPi.

I believe the the lightning[apps], lightning[pytorch] etc syntax does not work yet but we will be supported it in the future as this was the plan all along. Please correct me if I'm wrong @Borda @carmocca.

[KogaiIrina]

@awaelchli Hi. Thank you for the response.
lightning-app looks what I want, but I installed lightning-app instead of lightning in a fresh conda environment and got this error:

$ lightning run app app.py 
Traceback (most recent call last):
  File "/home/irina/miniconda3/envs/masterpiece-creator/bin/lightning", line 5, in <module>
    from lightning_app.cli.lightning_cli import main
  File "/home/irina/miniconda3/envs/masterpiece-creator/lib/python3.8/site-packages/lightning_app/cli/lightning_cli.py", line 30, in <module>
    from lightning_app.utilities.app_logs import _app_logs_reader
  File "/home/irina/miniconda3/envs/masterpiece-creator/lib/python3.8/site-packages/lightning_app/utilities/app_logs.py", line 11, in <module>
    from lightning_app.utilities.logs_socket_api import _LightningLogsSocketAPI
  File "/home/irina/miniconda3/envs/masterpiece-creator/lib/python3.8/site-packages/lightning_app/utilities/logs_socket_api.py", line 8, in <module>
    from lightning.app.core import constants
ModuleNotFoundError: No module named 'lightning'

[tchaton]

Hey there,

Yes, you would need to use from lightning_app in your code instead of lightning.app.

Best
T.C

[KogaiIrina]

Hi, @tchaton

File "/home/irina/miniconda3/envs/masterpiece-creator/lib/python3.8/site-packages/lightning_app/utilities/logs_socket_api.py", line 8, in
from lightning.app.core import constants

It looks like the error comes not from my code but from the lightning internals.

[rohitgr7]

hey @KogayIrina this was fixed herE: #15124
must be in the recent release. Mind try that out?

[KogaiIrina]

Hi, @rohitgr7 ^_^

I am using the latest lightning-app version available on pypi, 0.7.0.

$ lightning run app app.py
Traceback (most recent call last):
  File "/home/irina/miniconda3/envs/masterpiece-creator-test-app/bin/lightning", line 5, in <module>
    from lightning_app.cli.lightning_cli import main
  File "/home/irina/miniconda3/envs/masterpiece-creator-test-app/lib/python3.8/site-packages/lightning_app/cli/lightning_cli.py", line 30, in <module>
    from lightning_app.utilities.app_logs import _app_logs_reader
  File "/home/irina/miniconda3/envs/masterpiece-creator-test-app/lib/python3.8/site-packages/lightning_app/utilities/app_logs.py", line 11, in <module>
    from lightning_app.utilities.logs_socket_api import _LightningLogsSocketAPI
  File "/home/irina/miniconda3/envs/masterpiece-creator-test-app/lib/python3.8/site-packages/lightning_app/utilities/logs_socket_api.py", line 8, in <module>
    from lightning.app.core import constants
ModuleNotFoundError: No module named 'lightning'

$ pip list | grep lightning
lightning-app          0.7.0
lightning-cloud        0.5.7
lightning-utilities    0.3.0

Here is the lightning-app version of the code I am running: https://github.com/KogayIrina/masterpiece-creator/tree/test-lightning-app (I used a fresh conda environment for the test)

[awaelchli]

That is expected and we are aware of it. I believe the installation through lightning[app] would solve this problem so that one can have the imports in the standardized lightning way.

cc @Borda @carmocca @otaj @justusschock

[KogaiIrina]

Hi, @awaelchli
So, the lightning-app package is unusable right now, right?

To me, from the stack trace, it looks like the error comes from the lightning_app module code (it contains the lightning import), not my code. My code contains correct lightning_app imports.

[rohitgr7]

looks like that PR wasn't included in patch release. This will work once v1.8 is released in few days :)

[KogaiIrina]

@rohitgr7 thank you for the update :)

[Borda]

lets proceed it together with #15542