Understanding Faster R-CNN: Architecture, Training, and Experimental Results

Object detection is a core problem in computer vision, and Faster R‑CNN has become a landmark framework that dramatically improves detection speed and accuracy by introducing a Region Proposal Network (RPN). This article reviews the background of object detection, the evolution from Selective Search to R‑CNN, Fast R‑CNN, and finally Faster R‑CNN.

Background

Traditional detection pipelines relied on external region proposal methods such as Selective Search, which were computationally expensive. Fast R‑CNN shared convolutional features but still depended on these costly proposals. Faster R‑CNN solves this bottleneck by embedding a fully convolutional RPN that generates high‑quality proposals directly from the shared feature map.

End‑to‑end training : RPN and Fast R‑CNN are trained jointly.

Feature sharing : Both modules use the same convolutional backbone, reducing computation.

Multi‑scale anchors : RPN employs anchors of various scales and aspect ratios to cover objects of different sizes.

Related Work

Region Proposal Methods

Methods such as Selective Search, CPMC, EdgeBoxes, and sliding‑window approaches generate candidate boxes to reduce the search space for downstream classifiers.

Deep Learning Detection Networks

R‑CNN, OverFeat, MultiBox, and SPP‑net introduced convolutional feature sharing, leading to faster and more accurate detectors.

Faster R‑CNN Model Analysis

Faster R‑CNN consists of two main components:

Region Proposal Network (RPN) : Generates object proposals.

Fast R‑CNN Detector : Classifies and refines the proposals.

Algorithm Principles

1. RPN Training : A small sliding window network predicts k anchors and objectness scores for each location on the feature map.

2. Anchors : Default settings use 3 scales (128², 256², 512²) and 3 aspect ratios (1:1, 1:2, 2:1) to cover diverse object shapes.

3. Multi‑task Loss : Combines classification loss (binary cross‑entropy) and bounding‑box regression loss (smooth L1).

4. Shared Features : RPN and Fast R‑CNN share the backbone convolutional layers, so feature extraction is performed only once.

5. Alternating Training : The network is trained in stages—independent RPN training, Fast R‑CNN training on RPN proposals, joint fine‑tuning of shared layers, and final Fast R‑CNN head refinement.

Experiments

1. Experimental Procedure

Datasets: PASCAL VOC 2007/2012 (20 classes) and MS COCO (80 classes). Pre‑trained ImageNet models are used as backbones. RPN is first trained with SGD, followed by Fast R‑CNN training on the generated proposals. Feature sharing is enforced via alternating optimization. During inference, non‑maximum suppression (NMS) filters overlapping boxes.

2. Results

PASCAL VOC : Faster R‑CNN achieves 73.2% mAP, with an inference time of ~198 ms per image at a detection threshold of 0.6.

MS COCO : The model reaches 42.1% [email protected] and 21.5% mAP@[0.5,0.95] using a VGG‑16 backbone.

Code Reproduction

The following PyTorch‑style code illustrates the main components of Faster R‑CNN (RPN, Fast R‑CNN head, and the combined model). Users can extend it with actual layer dimensions and loss functions.

import torch
import torch.nn as nn
import torchvision.models as models

class RegionProposalNetwork(nn.Module):
    def __init__(self, feature_map_size, anchor_sizes, anchor_ratios):
        super(RegionProposalNetwork, self).__init__()
        # Convolutional layer for feature extraction
        self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding)
        # Classification head
        self.cls_score = nn.Linear(fc_size, 2 * num_anchors)
        # Bounding‑box regression head
        self.bbox_pred = nn.Linear(fc_size, 4 * num_anchors)

    def forward(self, x):
        h = self.conv1(x)
        cls_score = self.cls_score(h)
        bbox_pred = self.bbox_pred(h)
        return cls_score, bbox_pred

class FastRCNN(nn.Module):
    def __init__(self, num_classes):
        super(FastRCNN, self).__init__()
        self.vgg = models.vgg16(pretrained=True)
        self.roi_pool = RoIPooling(output_size)
        self.detection_head = nn.Sequential(
            nn.Linear(vgg.fc.in_features, fc_size),
            nn.ReLU(),
            nn.Linear(fc_size, num_classes + 1),  # +1 for background
            nn.Sigmoid()
        )
        self.bbox_reg = nn.Linear(vgg.fc.in_features, 4 * (num_classes + 1))

    def forward(self, x, rois):
        pool = self.roi_pool(x, rois)
        cls_score = self.detection_head(pool)
        bbox_pred = self.bbox_reg(pool)
        return cls_score, bbox_pred

class FasterRCNN(nn.Module):
    def __init__(self, num_classes, feature_map_size, anchor_sizes, anchor_ratios):
        super(FasterRCNN, self).__init__()
        self.rpn = RegionProposalNetwork(feature_map_size, anchor_sizes, anchor_ratios)
        self.fast_rcnn = FastRCNN(num_classes)

    def forward(self, images, targets=None):
        features = self.fast_rcnn.vgg(images)
        cls_score, bbox_pred = self.rpn(features)
        if self.training and targets is not None:
            rpn_loss_cls, rpn_loss_bbox = compute_rpn_loss(cls_score, bbox_pred, targets)
            return rpn_loss_cls, rpn_loss_bbox
        proposals = generate_proposals(cls_score, bbox_pred)
        det_boxes, det_probs = nms(proposals)
        if not self.training:
            pool = self.fast_rcnn.roi_pool(features, det_boxes)
            cls_score, bbox_pred = self.fast_rcnn(pool)
            return cls_score, bbox_pred

Experimental results demonstrate that Faster R‑CNN delivers high mAP on both PASCAL VOC and MS COCO, confirming its strong generalization and accuracy. Its success stems from the innovative RPN design and the effective sharing of deep features, influencing many subsequent object‑detection models.

Note : Detailed implementation and model hyper‑parameters can be obtained from the author for further research.