Extracting and Initializing Parameters in a PyTorch CNN Model

In PyTorch practice, sometimes the extracted layer structure is insufficient and you need to initialize the parameters inside the layers; this guide shows how to extract a network's parameters and initialize them.

The nn.Module class provides two important attributes for parameters: named_parameters() , which yields an iterator of (name, parameter) pairs, and parameters() , which yields an iterator of all parameters.

Example code defining a simple CNN model and printing each parameter's name:

import os
import torch
import torch.nn as nn
import torch.optim as optim
import torch.backends.cudnn as cudnn
import torch.nn.init as init
import argparse
import torch.autograd.variable as variable

class SimpleCNN(nn.Module):
    def __init__(self):
        super(SimpleCNN,self).__init__()
        # b,3,32,32
        layer1=nn.Sequential()
        layer1.add_module('conv1',nn.Conv2d(in_channels=3,out_channels=32,kernel_size=3,stride=1,padding=1))
        # b,32,32,32
        layer1.add_module('relu1',nn.ReLU(True))
        layer1.add_module('pool1',nn.MaxPool2d(2,2))
        # b,32,16,16
        self.layer1=layer1
        layer2=nn.Sequential()
        layer1.add_module('conv2',nn.Conv2d(in_channels=32,out_channels=64,kernel_size=3,stride=1,padding=1))
        # b,64,16,16
        layer2.add_module('relu2',nn.ReLU(True))
        layer2.add_module('pool2',nn.MaxPool2d(2,2))
        # b,64,8,8
        self.layer2=layer2
        
        layer3=nn.Sequential()
        layer3.add_module('conv3', nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3 ,stride=1, padding =1))
        # b,128,8,8
        layer3.add_module('relu3', nn.ReLU(True))
        layer3.add_module('poo13', nn.MaxPool2d(2, 2))#b,128,4,4
        self.layer3=layer3
        
        layer4 =nn.Sequential()
        layer4.add_module('fc1',nn.Linear(in_features=2048, out_features=512 ))
        layer4.add_module('fc_relu1', nn.ReLU(True))
        layer4.add_module('fc2 ', nn.Linear(in_features=512, out_features=64 ))
        layer4.add_module('fc_relu2', nn.ReLU(True))
        layer4.add_module('fc3', nn.Linear(64, 10))
        self.layer4 = layer4
    
    def forward(self,x):
        conv1=self.layer1(x)
        conv2=self.layer2(conv1)
        conv3=self.layer3(conv2)
        fc_input=conv3.view(conv3.size(0),-1)
        fc_output=self.layer4(fc_input)
        return fc_output
model=SimpleCNN()
for param in model.named_parameters():
    print(param[0])

The above loop prints each layer's parameter name.

To initialize weights, you can access the data attribute of each parameter (a Variable ) and apply the desired initialization method.

Typical initialization code for convolutional and linear layers:

for m in model.modules():
    if isinstance(m,nn.Conv2d):
        init.normal(m.weight.data)  # fill with normal distribution
        init.xavier_normal(m.weight.data)  # Xavier uniform (from 2010 paper)
        init.kaiming_normal(m.weight.data)  # Kaiming normal (from 2015 paper)
        m.bias.data.fill_(0)
    elif isinstance(m,nn.Linear):
        m.weight.data.normal_()

By applying these methods, you can initialize convolutional layer weights using any of PyTorch's built‑in functions or your own custom initialization logic.

For more details, refer to the original article linked at the end of the source.