Deep Learning Hyperparameter Tuning and Training Tips: Insights from Zhihu Experts

This article compiles practical deep learning training and hyperparameter tuning advice from Zhihu contributors, covering model debugging, learning‑rate strategies, optimizer choices, data preprocessing, regularization techniques, initialization methods, common pitfalls, recommended research papers, and ensemble approaches.

Deep LearningRegularizationgradient clipping

0 likes · 13 min read

Deep Learning Hyperparameter Tuning and Training Tips: Insights from Zhihu Experts

Baobao Algorithm Notes

Jul 26, 2022 · Artificial Intelligence

Boost Model Accuracy with 6 Proven Training Tricks

This article compiles six practical machine‑learning tricks—including adversarial training (FGM), EMA/SWA, R‑Drop contrastive loss, test‑time augmentation, pseudo‑labeling, and missing‑value imputation—explaining their principles, providing ready‑to‑use code snippets, and discussing their benefits and trade‑offs for stable and faster model training.

AIEMAR-Drop

0 likes · 10 min read

Boost Model Accuracy with 6 Proven Training Tricks

Baobao Algorithm Notes

Apr 11, 2022 · Artificial Intelligence

Can ResNet Still Beat Transformers? A Deep Dive into Modern Training Tricks

This article reviews recent research and official PyTorch blog updates that modify ResNet architectures and training tricks, compares their performance against EfficientNet, ConvNeXt, and Vision Transformers using extensive ImageNet benchmarks, and provides both literature‑based and local evaluation results to assess whether classic CNNs remain competitive.

CNNResNetmodel benchmarking

0 likes · 13 min read

Can ResNet Still Beat Transformers? A Deep Dive into Modern Training Tricks

DataFunTalk

Dec 4, 2021 · Artificial Intelligence

Practical Deep Learning Training Tricks: Cyclic LR, Flooding, Warmup, RAdam, Adversarial Training, Focal Loss, Dropout, Normalization and More

This article compiles essential deep learning training techniques—including cyclic learning rates, flooding, warmup, RAdam optimizer, adversarial training, focal loss, dropout, batch/group/weight normalization, label smoothing, Wasserstein GAN, skip connections, and weight initialization—providing concise explanations and code snippets for each method.

Deep LearningNeural NetworksRegularization

0 likes · 11 min read

Practical Deep Learning Training Tricks: Cyclic LR, Flooding, Warmup, RAdam, Adversarial Training, Focal Loss, Dropout, Normalization and More

DataFunTalk

Aug 10, 2021 · Artificial Intelligence

Practical Deep Learning Tricks: Cyclic LR, Flooding, Warmup, RAdam, Adversarial Training, Focal Loss, Dropout, Normalization, ReLU, Group Normalization, Label Smoothing, Wasserstein GAN, Skip Connections, Weight Initialization

This article presents a concise collection of practical deep‑learning techniques—including cyclic learning‑rate, flooding, warmup, RAdam, adversarial training, focal loss, dropout, various normalization methods, ReLU, group normalization, label smoothing, Wasserstein GAN, skip connections, and weight initialization—along with code snippets and references for implementation.

Deep LearningGANRegularization

0 likes · 8 min read

Practical Deep Learning Tricks: Cyclic LR, Flooding, Warmup, RAdam, Adversarial Training, Focal Loss, Dropout, Normalization, ReLU, Group Normalization, Label Smoothing, Wasserstein GAN, Skip Connections, Weight Initialization