Skip to content
Tagged articles

Reproducibility

18 articles · Page 1 of 1
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Oct 17, 2026 · Artificial Intelligence

12K GPU-Hour Reality Check: GRPO Outperforms 10 Self-Distillation Methods in Agentic RL

A large-scale empirical study comparing GRPO with 10 self-distillation methods across 4 tasks and 3 model sizes finds no method consistently beats GRPO; some methods show high final scores but suffer training instability, and compute-equivalent comparisons often reverse apparent gains.

GRPOReproducibilityagentic RL
0 likes · 10 min read
12K GPU-Hour Reality Check: GRPO Outperforms 10 Self-Distillation Methods in Agentic RL
Geek Labs
Geek Labs
Oct 7, 2026 · Artificial Intelligence

Stop AI Hand-Drawn Style Drift: Two Open-Source Solutions — Style Supermarket vs. Precision Recipes

This article compares two open-source projects that solve AI hand-drawn style drift: handraw-style offers 326 numbered styles, 161 layouts, and 36 colors for rapid exploration, while hand-drawn-styles provides 19 verified recipes with anchor validation and a three-stage workflow for reproducible production.

AI image generationReproducibilityWorkflow
0 likes · 18 min read
Stop AI Hand-Drawn Style Drift: Two Open-Source Solutions — Style Supermarket vs. Precision Recipes
Architects Research Society
Architects Research Society
Sep 25, 2026 · R&D Management

ArchHarness: Making Architecture Reviews Reproducible with Deterministic Gates & Provenance

ArchHarness is an open-source MIT-licensed tool that codifies the checkable half of architecture reviews — PASS/WARN/BLOCK gates, SHA-256 artifact provenance, and versioned contracts — into a deterministic CLI integrated with AI coding assistants and CI pipelines, supporting six cloud platform standards.

AI coding assistantsArchHarnessArchitecture Review
0 likes · 10 min read
ArchHarness: Making Architecture Reviews Reproducible with Deterministic Gates & Provenance
Java Architect Essentials
Java Architect Essentials
Sep 5, 2026 · R&D Management

Codex Research Skills: When They Pay Off (and When They Don't)

The article argues that investing time in creating Codex research skills is worthwhile only for repetitive tasks like weekly paper tracking, experiment reproduction, and team onboarding, advising researchers to start small, validate with real runs, and prioritize traceability over automation speed.

AI-assisted researchCodexReproducibility
0 likes · 4 min read
Codex Research Skills: When They Pay Off (and When They Don't)
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Aug 18, 2026 · Artificial Intelligence

Survey of Autonomous Research Agents: AI Scientists and the Verification Gap

This survey audits 35 autonomous research agent systems, revealing that while code release is common, reproducible evidence, novelty validation, execution traces, and external verification loops remain scarce, and it proposes a reviewer‑focused reporting checklist to close the verification gap.

AI scientistReproducibilitySurvey
0 likes · 16 min read
Survey of Autonomous Research Agents: AI Scientists and the Verification Gap

Are Top Conference Papers Losing Credibility? AutoResearch Turns the Lens on Research Quality

An AI‑driven review of 168 ICML 2026 oral papers reveals that only 105 could be fully reproduced, with a median replication cost of $8,900, many hidden flaws, and 903 blind‑spot issues that human reviewers missed, questioning the trustworthiness of top‑conference publications.

AI agentsICMLReproducibility
0 likes · 8 min read
Are Top Conference Papers Losing Credibility? AutoResearch Turns the Lens on Research Quality
Data Party THU
Data Party THU
Jul 6, 2026 · Artificial Intelligence

Why Even a 10× Smarter AI Scientist Won’t Speed Up Science: The 300‑Year‑Old Paper Bottleneck

The article argues that despite rapid advances in AI scientists—automating literature review, hypothesis generation, experimentation, and writing—their impact on scientific speed is limited by a three‑century‑old research protocol, peer‑review bottlenecks, and incentive misalignments, which can only be overcome by redesigning the research artifact itself.

AI for ScienceAgent‑Native Research ArtifactReproducibility
0 likes · 12 min read
Why Even a 10× Smarter AI Scientist Won’t Speed Up Science: The 300‑Year‑Old Paper Bottleneck
Code Mala Tang
Code Mala Tang
Jul 1, 2026 · Artificial Intelligence

Anthropic Launches Claude Science: AI Workbench Tying 60+ Scientific Databases

Claude Science, Anthropic’s new AI workbench for scientific research, embeds the existing Claude model (including Opus 4.8) into a unified interface that links over 60 databases, supports local macOS/Linux execution, offers reproducible agent‑generated analyses, and positions itself against OpenAI’s GPT‑Rosalind by focusing on workflow integration rather than specialized model training.

AI workbenchAnthropicClaude Science
0 likes · 7 min read
Anthropic Launches Claude Science: AI Workbench Tying 60+ Scientific Databases
DeepHub IMBA
DeepHub IMBA
May 13, 2026 · Artificial Intelligence

5 Python Decorators to Stabilize Your Machine Learning Pipeline

The article presents five practical Python decorators—Concurrency Limiter, Structured Logger, Feature Injector, Deterministic Seed Setter, and Dev‑Mode Fallback—explaining their implementation, why they matter for AI workloads, and how they keep ML pipelines maintainable, reproducible, and resilient under load.

AI PipelineConcurrencyDecorator
0 likes · 9 min read
5 Python Decorators to Stabilize Your Machine Learning Pipeline
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Apr 28, 2026 · Artificial Intelligence

Why DeepSeek V4 Insists on Batch Invariance—and What It Costs

DeepSeek V4 achieves ultra‑long context, complex training pipelines, and custom high‑performance kernels by enforcing batch invariance, a design that guarantees bit‑wise identical outputs across varying batch shapes but incurs lower GPU utilization, reduced small‑batch speed, and added engineering complexity.

Batch InvarianceGPU utilizationLLM Engineering
0 likes · 8 min read
Why DeepSeek V4 Insists on Batch Invariance—and What It Costs
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Apr 7, 2026 · Artificial Intelligence

From Engine Tinkerer to Top AI Agent: How Zhang Xue Built a Groundbreaking Agent Without Reading a Single AI Paper

The article uses Zhang Xue’s 20‑year engine‑building journey to illustrate five concrete standards—novel contribution, reproducibility, ablation, impact, and paradigm shift—that separate truly transformative AI papers from incremental work, arguing that rigorous, reductionist engineering can change the world.

Paradigm ShiftReproducibilitynovel contribution
0 likes · 18 min read
From Engine Tinkerer to Top AI Agent: How Zhang Xue Built a Groundbreaking Agent Without Reading a Single AI Paper
AI Frontier Lectures
AI Frontier Lectures
Apr 17, 2025 · Artificial Intelligence

Why Reinforcement Learning Fails to Boost Small LLM Reasoning: A Deep Dive

This article analyzes a recent study on language‑model reasoning, revealing that reinforcement learning often brings little or no improvement, while evaluation variance caused by seeds, hardware, and decoding settings can dramatically affect benchmark results, and supervised fine‑tuning emerges as a more reliable path.

LLMReproducibilityreinforcement learning
0 likes · 12 min read
Why Reinforcement Learning Fails to Boost Small LLM Reasoning: A Deep Dive
Baobao Algorithm Notes
Baobao Algorithm Notes
Jun 27, 2024 · Industry Insights

How Open LLM Leaderboard v2 Redefines LLM Evaluation with New Benchmarks and Fair Scoring

Open LLM Leaderboard v2 introduces a revamped, reproducible evaluation framework for large language models, replacing saturated benchmarks with six carefully curated, unpolluted datasets, applying standardized scoring, updating the harness, adding voting and maintainer‑recommended models, and providing richer visualizations to guide the AI community.

AI metricsCommunity VotingLLM Evaluation
0 likes · 19 min read
How Open LLM Leaderboard v2 Redefines LLM Evaluation with New Benchmarks and Fair Scoring
Ops Development & AI Practice
Ops Development & AI Practice
Jun 26, 2024 · Fundamentals

Why Jupyter Notebooks Revolutionized Data Science and Machine Learning

This article explores the origins, key innovations, and lasting impact of Jupyter notebooks, highlighting how their multi‑language support, interactive computing, reproducibility, and extensibility have transformed data exploration, collaboration, education, and research in modern data science and machine learning.

Interactive ComputingJupyterReproducibility
0 likes · 5 min read
Why Jupyter Notebooks Revolutionized Data Science and Machine Learning
Architects Research Society
Architects Research Society
Jan 6, 2021 · Artificial Intelligence

DVC: Data Version Control for Machine Learning Projects

DVC is an open‑source data version control system that extends Git to manage large machine‑learning models, datasets, and pipelines, enabling reproducible experiments, low‑friction branching, metric tracking, and seamless collaboration across various storage backends.

DVCML PipelinesReproducibility
0 likes · 9 min read
DVC: Data Version Control for Machine Learning Projects