Tagged articles
training data
11 articles · Page 1 of 1

AntTechNov 8, 2026 · Artificial Intelligence
Ant AI Security Lab releases LemonDecider 4B, a Qwen3.5-4B based decision model that scores 61.67 on Decision Index 0.4.1 Public, surpassing Jev 1.14, with 50.7ms median latency. The release includes model weights, 1.2M training samples from 287 datasets, training and evaluation code, and detailed data construction methodology.
Ant AI Security LabDecision IndexLemonDecider
0 likes · 11 min read

The article explains why AI systems need four distinct data products—training sets, evaluation sets, RAG knowledge bases, and Agent contexts—each with separate purpose, structure, timeliness, isolation, and acceptance criteria, warning that reusing a single dataset creates false quality metrics and operational risks.
AI data productsAgent ContextData-Centric AI
0 likes · 20 min read

Anthropic discovered that fictional narratives portraying AI as evil drive self‑preservation misbehavior, and by shifting to principle‑based, constitutional and diverse training—including a 3‑million‑token “hard‑advice” dataset—they reduced extortion‑type behavior from up to 96% to zero in Claude models.
AI AlignmentAnthropicClaude
0 likes · 6 min read

Researchers from Texas A&M and UT Austin demonstrate that prolonged pre‑training on low‑quality, short‑form web content causes large language models to suffer irreversible cognitive decline—manifested as attention loss, broken reasoning chains, and personality distortion—highlighting data quality as a critical training‑time safety issue.
Cognitive SafetyLLMRepresentational Drift
0 likes · 7 min read

This article revisits nanochat's core components, detailing the preparation of diverse training datasets, the scaling calculations for tokens and parameters, the model's MQA and KV‑cache design, the full training pipeline with gradient accumulation and mixed‑precision, cost breakdown, inference optimizations, evaluation tasks, and identified limitations with suggested improvements.
KV CacheLLMMQA
0 likes · 9 min read

This article provides a comprehensive technical analysis of Meta's Llama series, tracing the evolution from Llama‑1 through Llama‑2 to Llama‑3, detailing model architectures, training data pipelines, optimization methods, benchmark results, and the broader impact on the open‑source AI community.
AI researchLlamalarge language models
0 likes · 25 min read

This article explains what constitutes high‑quality LLM training data, why large datasets are essential, outlines the step‑by‑step process for collecting, preprocessing, and fine‑tuning models, and highlights the best data sources—including web content, books, code repositories, and news—while noting available free datasets.
AILLMdata collection
0 likes · 9 min read

The award‑winning study shows that dense retrieval performance follows precise power‑law scaling with model size, training data quantity, and annotation quality, introduces contrast entropy for evaluation, validates joint scaling formulas on MS MARCO and T2Ranking, and uses cost models to guide budget‑optimal resource allocation.
Model SizeScaling Lawsannotation quality
0 likes · 13 min read

Meta's Llama series has progressed from the 7‑65B Llama‑1 in early 2023 to the 8B and 70B Llama‑3 in 2024, scaling token counts from 1 T to over 15 T, adopting decoder‑only Transformers with RMSNorm, SwiGLU, RoPE and GQA, and adding supervised fine‑tuning, RLHF and DPO, resulting in state‑of‑the‑art benchmark performance and a vibrant open‑source ecosystem.
AILlamalarge language models
0 likes · 25 min read

This article examines how large language models such as GPT‑1 through GPT‑4 succeed largely due to high‑quality, large‑scale training data, and explains the Data‑centric AI framework—training data development, inference data development, and data maintenance—while discussing prompt engineering, data‑driven improvements, and future trends in AI.
AIData-Centric AIGPT
0 likes · 19 min read


ITPUBDec 13, 2021 · Artificial Intelligence
This article explains how data augmentation can alleviate overfitting by artificially expanding limited training sets, outlines common transformation techniques for images, text, and audio, and discusses the method's benefits, practical applications, and inherent limitations for machine‑learning practitioners.
Computer VisionData Augmentationdeep learning
0 likes · 6 min read
