Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

misc: Add multiple machine example #757

Merged
merged 7 commits into from
Feb 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions examples/multi-machine/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# GitHub syntax highlighting
pixi.lock linguist-language=YAML
2 changes: 2 additions & 0 deletions examples/multi-machine/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# pixi environments
.pixi
8,658 changes: 8,658 additions & 0 deletions examples/multi-machine/pixi.lock

Large diffs are not rendered by default.

47 changes: 47 additions & 0 deletions examples/multi-machine/pixi.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
[project]
name = "multi-machine"
description = "A mock project that does ML stuff"
channels = ["conda-forge", "pytorch"]
# All platforms that are supported by the project as the features will take the intersection of the platforms defined there.
platforms = ["win-64", "linux-64", "osx-64", "osx-arm64"]

[tasks]
train = "python train.py"
test = "python test.py"
start = {depends_on = ["train", "test"]}

[dependencies]
python = "3.11.*"
pytorch = {version = ">=2.0.1", channel = "pytorch"}
torchvision = {version = ">=0.15", channel = "pytorch"}
polars = ">=0.20,<0.21"
matplotlib-base = ">=3.8.2,<3.9"
ipykernel = ">=6.28.0,<6.29"

[feature.cuda]
platforms = ["win-64", "linux-64"]
channels = ["nvidia", {channel = "pytorch", priority = -1}]
system-requirements = {cuda = "12.1"}

[feature.cuda.tasks]
train = "python train.py --cuda"
test = "python test.py --cuda"

[feature.cuda.dependencies]
pytorch-cuda = {version = "12.1.*", channel = "pytorch"}

[feature.mlx]
platforms = ["osx-arm64"]
system-requirements = {macos = "13.3"}

[feature.mlx.tasks]
train = "python train.py --mlx"
test = "python test.py --mlx"

[feature.mlx.dependencies]
mlx = "*"


[environments]
cuda = ["cuda"]
mlx = ["mlx"]
8 changes: 8 additions & 0 deletions examples/multi-machine/test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import os
import sys

print("Hello from test.py!")
print("Environment you are running on:")
print(os.environ["PIXI_ENVIRONMENT_NAME"])
print("Arguments given to the script:")
print(sys.argv[1:])
24 changes: 24 additions & 0 deletions examples/multi-machine/train.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import os
import sys

if os.environ["PIXI_ENVIRONMENT_NAME"] == "mlx":
import mlx.core as mx
a = mx.array([1, 2, 3, 4])
print(a.shape)
print("MLX is available, in mlx environment as expected")

if os.environ["PIXI_ENVIRONMENT_NAME"] == "cuda":
import torch
assert torch.cuda.is_available(), "CUDA is not available"
print("CUDA is available, in cuda environment as expected")

if os.environ["PIXI_ENVIRONMENT_NAME"] == "default":
import torch
assert not torch.cuda.is_available(), "CUDA is available, in default environment"
print("CUDA is not available, in default environment as expected")

print("\nHello from train.py!")
print("Environment you are running on:")
print(os.environ["PIXI_ENVIRONMENT_NAME"])
print("Arguments given to the script:")
print(sys.argv[1:])
19 changes: 14 additions & 5 deletions examples/opencv/webcam_capture.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,25 @@ def capture_and_grayscale():
# Load the cascade for face detection
face_cascade = cv2.CascadeClassifier(filename)

# Connect to the webcam
cap = cv2.VideoCapture(0)
# Search for available webcams
working_cam = None
for index in range(3):
cap = cv2.VideoCapture(index)
if not cap.read()[0]:
cap.release()
continue
else:
working_cam = cap
break


# Check if the webcam is opened correctly
if not cap.isOpened():
if not working_cam.isOpened():
raise IOError("Cannot open webcam")

while True:
# Read the current frame from the webcam
ret, frame = cap.read()
ret, frame = working_cam.read()

if not ret:
break
Expand All @@ -57,7 +66,7 @@ def capture_and_grayscale():
break

# Release the VideoCapture object
cap.release()
working_cam.release()

# Destroy all windows
cv2.destroyAllWindows()
Expand Down
Loading