Introduction to PyTorch and Example CNN Training on CIFAR-10

PyTorch is a popular open‑source machine learning library developed by Facebook's AI research lab, now a core framework for deep learning in the Python ecosystem, offering powerful numerical computation, a flexible dynamic computation graph, and automatic differentiation.

Dynamic Computation Graph: PyTorch allows building and modifying computation graphs on the fly, enabling rapid experimentation and prototyping.

Tensors: The core data structure torch.Tensor behaves like a NumPy array but can reside on CPU or GPU and supports extensive mathematical operations.

Automatic Differentiation (Autograd): Autograd records operation history and automatically computes gradients during back‑propagation, essential for training neural networks.

Deep Learning Modules: Built‑in modules such as nn.Conv2d, nn.Linear, nn.ReLU, and loss functions like nn.CrossEntropyLoss simplify constructing complex models.

Optimizers: A variety of optimization algorithms (e.g., SGD, Adam, RMSprop) are provided to update model parameters.

Data Loading and Preprocessing: The torch.utils.data module offers Dataset and DataLoader classes for efficient data loading, batching, and multi‑process preprocessing.

Distributed Training: PyTorch supports multi‑machine, multi‑GPU training via the torch.distributed package.

TorchScript: Models can be converted to TorchScript, a serialized, optimized bytecode format suitable for deployment in non‑Python environments or on mobile devices.

Visualization Tools: Libraries like torchviz or torchsummary help visualize computation graphs and model architectures.

Example Code – Simple CNN Definition:

import torch
import torch.nn as nn
import torch.optim as optim
# Define a simple convolutional neural network model
class SimpleCNN(nn.Module):
def __init__(self, num_classes=10):
super(SimpleCNN, self).__init__()
# Convolutional layers
self.conv_layers = nn.Sequential(
nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3, padding=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2),
nn.Conv2d(16, 32, 3, padding=1),
nn.ReLU(),
nn.MaxPool2d(2, 2),
)
# Fully‑connected layers
self.fc_layers = nn.Sequential(
nn.Linear(32 * 8 * 8, 512),  # assuming feature map size 8x8 after pooling
nn.ReLU(),
nn.Dropout(p=0.5),  # prevent overfitting
nn.Linear(512, num_classes),  # output layer
)
def forward(self, x):
x = self.conv_layers(x)
x = x.view(x.size(0), -1)  # flatten
x = self.fc_layers(x)
return x
# Instantiate the model
model = SimpleCNN()
# Define loss function and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)
# Training loop (data loading omitted for brevity)

Example Code – CIFAR‑10 Data Loading and Training Loop:

import torchvision
import torchvision.transforms as transforms
from torch.utils.data import DataLoader
# Data preprocessing
transform = transforms.Compose([
transforms.RandomHorizontalFlip(),  # data augmentation
transforms.RandomCrop(32, padding=4),
transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))  # normalization
])
# Load training and test datasets
trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
trainloader = DataLoader(trainset, batch_size=100, shuffle=True, num_workers=2)
testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)
testloader = DataLoader(testset, batch_size=100, shuffle=False, num_workers=2)
# Set device
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = model.to(device)
num_epochs = 25
for epoch in range(num_epochs):
running_loss = 0.0
for i, data in enumerate(trainloader, 0):
inputs, labels = data[0].to(device), data[1].to(device)
optimizer.zero_grad()   # clear previous gradients
outputs = model(inputs)  # forward pass
loss = criterion(outputs, labels)  # compute loss
loss.backward()  # backward pass
optimizer.step()  # update weights
running_loss += loss.item()
print(f'Epoch {epoch + 1}, Loss: {running_loss / (i + 1)}')
# Validation
correct = 0
total = 0
with torch.no_grad():
for data in testloader:
images, labels = data[0].to(device), data[1].to(device)
outputs = model(images)
_, predicted = torch.max(outputs.data, 1)
total += labels.size(0)
correct += (predicted == labels).sum().item()
print(f'Accuracy of the network on the 10000 test images: {100 * correct / total}%')