A self‐guided introduction to Pytorch

Self-Guided Introductory Session on PyTorch

---

1. Advantages of Using PyTorch for Deep Learning Tasks:

• Dynamic Computational Graphs: PyTorch uses dynamic computation graphs, allowing for flexibility in model building and debugging.
• Pythonic Nature: PyTorch integrates seamlessly with Python, making it easy to learn and use, especially for those familiar with Python.
• Strong Community and Ecosystem: A large, active community with extensive resources, tutorials, and libraries that enhance productivity.
• Interoperability with Other Libraries: PyTorch works well with other libraries, like NumPy and sci-kit-learn, enabling smooth integration of various tools.
• GPU Acceleration: PyTorch has built-in support for CUDA, allowing models to run on GPUs for faster computation.
• Extensive Model Zoo: Access to a wide variety of pre-trained models that can be easily fine-tuned for specific tasks.

2. Sequential Gradual List of Short PyTorch Examples:

Step 1: Installing PyTorch

pip install torch torchvision torchaudio

Step 2: Importing PyTorch and Checking for GPU Availability

import torch

# Check for GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f'Using device: {device}')

Step 3: Creating Tensors

# Create a 2x3 tensor
tensor = torch.tensor([[1, 2, 3], [4, 5, 6]])
print(tensor)

# Creating a random tensor with GPU support
tensor_gpu = torch.randn((3, 3), device=device)
print(tensor_gpu)

Step 4: Basic Operations with Tensors

# Addition
tensor_sum = tensor + tensor
print(tensor_sum)

# Element-wise multiplication
tensor_mul = tensor * tensor
print(tensor_mul)

# Matrix multiplication
tensor_matmul = torch.matmul(tensor, tensor.T)
print(tensor_matmul)

Step 5: Building a Simple Neural Network

import torch.nn as nn

# Define a simple feedforward neural network
class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(3, 2)
        self.fc2 = nn.Linear(2, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = torch.sigmoid(self.fc2(x))
        return x

# Initialize the model, loss function, and optimizer
model = SimpleNN().to(device)
loss_fn = nn.BCELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

Step 6: Training the Neural Network

# Dummy input and target
input_data = torch.tensor([[1.0, 2.0, 3.0]], device=device)
target = torch.tensor([[1.0]], device=device)

# Forward pass
output = model(input_data)
loss = loss_fn(output, target)

# Backward pass and optimization
optimizer.zero_grad()
loss.backward()
optimizer.step()

print(f'Output: {output.item()}, Loss: {loss.item()}')

Step 7: Saving and Loading the Model

# Saving the model
torch.save(model.state_dict(), 'simple_nn.pth')

# Loading the model
model_loaded = SimpleNN()
model_loaded.load_state_dict(torch.load('simple_nn.pth'))
model_loaded.to(device)

3. PyTorch Learning Resources:

• Books:
  • "Deep Learning with PyTorch" by Eli Stevens, Luca Antiga, and Thomas Viehmann.
  • "Programming PyTorch for Deep Learning" by Ian Pointer.
  • "Machine Learning with PyTorch and Scikit-Learn" by Sebastian Raschka and Yuxi (Hayden) Liu.
• Online Courses:
  • PyTorch: Deep Learning and Artificial Intelligence – A comprehensive course on Udemy.
  • Fast.ai's Practical Deep Learning for Coders – An in-depth course that uses PyTorch.
  • Deep Learning with PyTorch: A 60 Minute Blitz – A quick, hands-on introduction.
• Documentation:
  • Official PyTorch Documentation – The primary resource for all things PyTorch.
  • PyTorch Tutorials – A variety of tutorials ranging from beginner to advanced topics.
  • PyTorch Lightning
• Tutorials and Blogs:
  • Primers: PyTorch. Aman.AI.
  • Deep Learning with PyTorch: A 60 Minute Blitz. PyTorch.
  • PyTorch Blog – Updates, tutorials, and case studies related to PyTorch.
  • Towards Data Science – Numerous articles and tutorials on PyTorch and deep learning.

4. General References:

• Paszke, A., et al. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. NeurIPS.
• He, K., et al. (2016). Deep Residual Learning for Image Recognition. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).
• Zhang, Aston and Lipton, Zachary C. and Li, Mu and Smola, Alexander J., et al. (2023). Dive into Deep Learning. Cambridge University Press.
• Goodfellow, I., et al. (2016). Deep Learning. MIT Press.

---

Created: 08/18/2024 (C. Lizárraga)

Updated: 09/11/2024 (C. Lizárraga)

DataLab, Data Science Institute, University of Arizona.

CC BY-NC-SA 4.0