Unlocking AI Model Choices: From CNNs to Transformers and Fine‑Tuning Strategies

Following the development trajectory of AI, this series introduces key model families—CNN, RNN, Transformer, GAN, and GNN—explaining their principles, typical variants, and suitable application scenarios.

Model and Model Architecture

In deep learning, a model architecture (the blueprint) defines the structure and computation logic (e.g., CNN, Transformer), while a trained model is a concrete instance with learned weights (e.g., a specific GPT model).

Convolutional Neural Networks (CNNs)

CNNs efficiently extract local spatial features and excel at grid‑like data such as images.

LeNet‑5 : early CNN for handwritten digit recognition.

AlexNet : boosted ImageNet performance and spurred deep learning.

VGGNet : deeper variants (VGG16, VGG19) improve accuracy.

ResNet : residual connections mitigate gradient vanishing.

EfficientNet : balances model scale and performance via compound scaling.

YOLO : real‑time object detection.

Recurrent Neural Networks (RNNs)

RNNs process sequential data by retaining temporal dependencies.

Standard RNN : basic recurrent network, suffers from gradient vanishing.

LSTM : gated cells address vanishing gradients.

GRU : simpler gated unit with comparable performance.

Bi‑directional RNN : captures context from both directions.

Transformer Models

Transformers rely on self‑attention to model long‑range dependencies and are widely used in NLP and vision.

Transformer (original) : machine translation.

BERT : bidirectional encoder for diverse NLP tasks.

GPT series : generative pre‑trained transformers for text generation and dialogue.

Vision Transformer (ViT) : applies Transformer to image classification.

DETR : Transformer‑based object detection.

Generative Adversarial Networks (GANs)

GANs pit a generator against a discriminator to produce high‑quality synthetic data.

Original GAN : basic adversarial framework.

DCGAN : integrates convolutional layers for better image generation.

StyleGAN : high‑fidelity face synthesis with style control.

CycleGAN : unpaired image‑to‑image translation.

BigGAN : large‑scale GAN for diverse, high‑quality images.

Graph Neural Networks (GNNs)

GNNs operate on graph‑structured data by aggregating information across node connections.

GCNs : graph convolution for feature extraction.

GATs : attention‑based weighting of neighbor contributions.

GraphSAGE : scalable neighbor sampling for large graphs.

Choosing the Right Model

Different architectures excel in different domains:

Computer Vision : CNN, ResNet, ViT.

Natural Language Processing : RNN, LSTM, GRU, Transformer (BERT, GPT).

Generative Tasks : GAN, VAE.

Graph Data : GNN, GCN.

Sequential/Time Series : RNN, LSTM, GRU, Transformer.

Using Pre‑trained Models

Pre‑trained models (e.g., BERT, GPT, ResNet) are trained on large generic datasets and provide transferable knowledge. They can be obtained from platforms such as Hugging Face Hub , PyTorch Hub , and TensorFlow Hub . Fine‑tuning adapts these models to specific tasks with minimal additional training.

Fine‑tuning Techniques

1. Standard Fine‑tuning

Train all parameters on task‑specific data with a low learning rate.

2. Supervised Fine‑tuning (SFT)

Leverages labeled data to further improve performance on supervised tasks.

3. Low‑Rank Adaptation (LoRA)

Injects low‑rank matrices into weight matrices, freezing the original parameters to reduce added parameters.

4. Knowledge Distillation

Transfers knowledge from a large teacher model to a smaller student model, often used for model compression.

In matrix theory, rank denotes the maximum number of linearly independent rows or columns. Low‑rank matrices have far fewer independent components than their dimensions.

Practical Example: Sentiment Analysis with BERT

The following code demonstrates loading the IMDB dataset, defining a custom Dataset class, creating data loaders, initializing a pre‑trained bert-base-uncased model, and training it with AdamW and a linear learning‑rate scheduler.

<span>model = keras.Sequential([</span>
<span>    keras.Input(shape=(28, 28, 1)),  # input layer</span>
<span>    keras.layers.Conv2D(32, (3, 3), activation='relu'),  # conv layer</span>
<span>    keras.layers.MaxPooling2D((2, 2)),  # pooling layer</span>
<span>    keras.layers.Flatten(),  # flatten</span>
<span>    keras.layers.Dense(128, activation='relu'),  # fully connected</span>
<span>    keras.layers.Dense(10, activation='softmax')  # output</span>
<span>])</span>

<span># Compile and train the model</span>
<span>model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])</span>
<span>model.fit(train_images.reshape(-1, 28, 28, 1), train_labels, epochs=5, batch_size=32, validation_data=(val_images.reshape(-1, 28, 28, 1), val_labels))</span>

<span># Custom IMDB dataset class</span>
<span>class IMDBDataset(Dataset):</span>
<span>    def __init__(self, dataset, tokenizer, max_length):</span>
<span>        self.dataset = dataset</span>
<span>        self.tokenizer = tokenizer</span>
<span>        self.max_length = max_length</span>
<span>    def __len__(self):</span>
<span>        return len(self.dataset)</span>
<span>    def __getitem__(self, idx):</span>
<span>        text = self.dataset[idx]['text']</span>
<span>        label = self.dataset[idx]['label']</span>
<span>        encoding = self.tokenizer.encode_plus(text, add_special_tokens=True, max_length=self.max_length, padding='max_length', truncation=True, return_tensors='pt')</span>
<span>        return {'input_ids': encoding['input_ids'].flatten(), 'attention_mask': encoding['attention_mask'].flatten(), 'labels': torch.tensor(label, dtype=torch.long)}</span>

<span># Training loop (simplified)</span>
<span>for epoch in range(num_epochs):</span>
<span>    model.train()</span>
<span>    for batch in train_dataloader:</span>
<span>        input_ids = batch['input_ids'].to(device)</span>
<span>        attention_mask = batch['attention_mask'].to(device)</span>
<span>        labels = batch['labels'].to(device)</span>
<span>        optimizer.zero_grad()</span>
<span>        outputs = model(input_ids, attention_mask=attention_mask, labels=labels)</span>
<span>        loss = outputs.loss</span>
<span>        loss.backward()</span>
<span>        optimizer.step()</span>
<span>        scheduler.step()</span>

After training, the best model is saved as best_model.pth. Evaluation on the test set yields accuracy and F1 scores, and a helper function predict_sentiment demonstrates inference on custom sentences.

<span># Example inference</span>
<span>custom_text = "This film is a masterpiece! The cinematography and soundtrack are unparalleled."
<span>result = predict_sentiment(custom_text, tokenizer, model, device=device)</span>
<span>print(f"Sentiment: {result['sentiment']} (positive prob: {result['positive_prob']}, negative prob: {result['negative_prob']})")</span>

Running the script prints test accuracy (~0.89), F1 (~0.88), and the sentiment prediction for the custom text (positive with high confidence).

Conclusion

This article provides a roadmap for understanding AI model families, selecting appropriate architectures for specific tasks, and efficiently leveraging pre‑trained models through various fine‑tuning strategies, all illustrated with a complete end‑to‑end PyTorch implementation.