Introduction to PyTorch and Example CNN Training on CIFAR-10

PyTorch is a popular open‑source machine learning library developed by Facebook's AI research lab, now a core framework for deep learning in the Python ecosystem, offering powerful numerical computation, a flexible dynamic computation graph, and automatic differentiation.

Dynamic Computation Graph: PyTorch allows building and modifying computation graphs on the fly, enabling rapid experimentation and prototyping.

Tensors: The core data structure torch.Tensor behaves like a NumPy array but can reside on CPU or GPU and supports extensive mathematical operations.

Automatic Differentiation (Autograd): Autograd records operation history and automatically computes gradients during back‑propagation, essential for training neural networks.

Deep Learning Modules: Built‑in modules such as nn.Conv2d, nn.Linear, nn.ReLU, and loss functions like nn.CrossEntropyLoss simplify constructing complex models.

Optimizers: A variety of optimization algorithms (e.g., SGD, Adam, RMSprop) are provided to update model parameters.

Data Loading and Preprocessing: The torch.utils.data module offers Dataset and DataLoader classes for efficient data loading, batching, and multi‑process preprocessing.

Distributed Training: PyTorch supports multi‑machine, multi‑GPU training via the torch.distributed package.

TorchScript: Models can be converted to TorchScript, a serialized, optimized bytecode format suitable for deployment in non‑Python environments or on mobile devices.

Visualization Tools: Libraries like torchviz or torchsummary help visualize computation graphs and model architectures.

Example Code – Simple CNN Definition:

import torch</code>
<code>import torch.nn as nn</code>
<code>import torch.optim as optim</code>
<code># Define a simple convolutional neural network model</code>
<code>class SimpleCNN(nn.Module):</code>
<code>    def __init__(self, num_classes=10):</code>
<code>        super(SimpleCNN, self).__init__()</code>
<code>        # Convolutional layers</code>
<code>        self.conv_layers = nn.Sequential(
<code>            nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3, padding=1),</code>
<code>            nn.ReLU(),</code>
<code>            nn.MaxPool2d(kernel_size=2, stride=2),</code>
<code>            nn.Conv2d(16, 32, 3, padding=1),</code>
<code>            nn.ReLU(),
<code>            nn.MaxPool2d(2, 2),
<code>        )</code>
<code>        # Fully‑connected layers</code>
<code>        self.fc_layers = nn.Sequential(
<code>            nn.Linear(32 * 8 * 8, 512),  # assuming feature map size 8x8 after pooling</code>
<code>            nn.ReLU(),
<code>            nn.Dropout(p=0.5),  # prevent overfitting</code>
<code>            nn.Linear(512, num_classes),  # output layer</code>
<code>        )</code>
<code>    def forward(self, x):</code>
<code>        x = self.conv_layers(x)</code>
<code>        x = x.view(x.size(0), -1)  # flatten</code>
<code>        x = self.fc_layers(x)</code>
<code>        return x</code>
<code># Instantiate the model</code>
<code>model = SimpleCNN()</code>
<code># Define loss function and optimizer</code>
<code>criterion = nn.CrossEntropyLoss()</code>
<code>optimizer = optim.SGD(model.parameters(), lr=0.001, momentum=0.9)</code>
<code># Training loop (data loading omitted for brevity)

Example Code – CIFAR‑10 Data Loading and Training Loop:

import torchvision</code>
<code>import torchvision.transforms as transforms</code>
<code>from torch.utils.data import DataLoader</code>
<code># Data preprocessing</code>
<code>transform = transforms.Compose([
<code>    transforms.RandomHorizontalFlip(),  # data augmentation
<code>    transforms.RandomCrop(32, padding=4),
<code>    transforms.ToTensor(),
<code>    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))  # normalization
<code>])</code>
<code># Load training and test datasets</code>
<code>trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)</code>
<code>trainloader = DataLoader(trainset, batch_size=100, shuffle=True, num_workers=2)</code>
<code>testset = torchvision.datasets.CIFAR10(root='./data', train=False, download=True, transform=transform)</code>
<code>testloader = DataLoader(testset, batch_size=100, shuffle=False, num_workers=2)</code>
<code># Set device</code>
<code>device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")</code>
<code>model = model.to(device)</code>
<code>num_epochs = 25</code>
<code>for epoch in range(num_epochs):</code>
<code>    running_loss = 0.0</code>
<code>    for i, data in enumerate(trainloader, 0):</code>
<code>        inputs, labels = data[0].to(device), data[1].to(device)</code>
<code>        optimizer.zero_grad()   # clear previous gradients</code>
<code>        outputs = model(inputs)  # forward pass</code>
<code>        loss = criterion(outputs, labels)  # compute loss</code>
<code>        loss.backward()  # backward pass</code>
<code>        optimizer.step()  # update weights</code>
<code>        running_loss += loss.item()</code>
<code>    print(f'Epoch {epoch + 1}, Loss: {running_loss / (i + 1)}')</code>
<code># Validation</code>
<code>correct = 0</code>
<code>total = 0</code>
<code>with torch.no_grad():</code>
<code>    for data in testloader:</code>
<code>        images, labels = data[0].to(device), data[1].to(device)</code>
<code>        outputs = model(images)</code>
<code>        _, predicted = torch.max(outputs.data, 1)</code>
<code>        total += labels.size(0)</code>
<code>        correct += (predicted == labels).sum().item()</code>
<code>print(f'Accuracy of the network on the 10000 test images: {100 * correct / total}%')