Tagged articles
Reproducibility
18 articles · Page 1 of 1
A large-scale empirical study comparing GRPO with 10 self-distillation methods across 4 tasks and 3 model sizes finds no method consistently beats GRPO; some methods show high final scores but suffer training instability, and compute-equivalent comparisons often reverse apparent gains.
GRPOReproducibilityagentic RL
0 likes · 10 min read


Geek LabsOct 7, 2026 · Artificial Intelligence
This article compares two open-source projects that solve AI hand-drawn style drift: handraw-style offers 326 numbered styles, 161 layouts, and 36 colors for rapid exploration, while hand-drawn-styles provides 19 verified recipes with anchor validation and a three-stage workflow for reproducible production.
AI image generationReproducibilityWorkflow
0 likes · 18 min read

ArchHarness is an open-source MIT-licensed tool that codifies the checkable half of architecture reviews — PASS/WARN/BLOCK gates, SHA-256 artifact provenance, and versioned contracts — into a deterministic CLI integrated with AI coding assistants and CI pipelines, supporting six cloud platform standards.
AI coding assistantsArchHarnessArchitecture Review
0 likes · 10 min read

The article argues that investing time in creating Codex research skills is worthwhile only for repetitive tasks like weekly paper tracking, experiment reproduction, and team onboarding, advising researchers to start small, validate with real runs, and prioritize traceability over automation speed.
AI-assisted researchCodexReproducibility
0 likes · 4 min read

This survey audits 35 autonomous research agent systems, revealing that while code release is common, reproducible evidence, novelty validation, execution traces, and external verification loops remain scarce, and it proposes a reviewer‑focused reporting checklist to close the verification gap.
AI scientistReproducibilitySurvey
0 likes · 16 min read

An AI‑driven review of 168 ICML 2026 oral papers reveals that only 105 could be fully reproduced, with a median replication cost of $8,900, many hidden flaws, and 903 blind‑spot issues that human reviewers missed, questioning the trustworthiness of top‑conference publications.
AI agentsICMLReproducibility
0 likes · 8 min read

This guide consolidates the AAAI‑27 timeline, paper length limits, anonymity requirements, format red‑lines, and a detailed checklist so authors can verify every compliance detail before the final upload and avoid desk rejection.
AAAI-27Reproducibilityanonymous review
0 likes · 13 min read

The article argues that despite rapid advances in AI scientists—automating literature review, hypothesis generation, experimentation, and writing—their impact on scientific speed is limited by a three‑century‑old research protocol, peer‑review bottlenecks, and incentive misalignments, which can only be overcome by redesigning the research artifact itself.
AI for ScienceAgent‑Native Research ArtifactReproducibility
0 likes · 12 min read

Claude Science, Anthropic’s new AI workbench for scientific research, embeds the existing Claude model (including Opus 4.8) into a unified interface that links over 60 databases, supports local macOS/Linux execution, offers reproducible agent‑generated analyses, and positions itself against OpenAI’s GPT‑Rosalind by focusing on workflow integration rather than specialized model training.
AI workbenchAnthropicClaude Science
0 likes · 7 min read

The article presents five practical Python decorators—Concurrency Limiter, Structured Logger, Feature Injector, Deterministic Seed Setter, and Dev‑Mode Fallback—explaining their implementation, why they matter for AI workloads, and how they keep ML pipelines maintainable, reproducible, and resilient under load.
AI PipelineConcurrencyDecorator
0 likes · 9 min read

DeepSeek V4 achieves ultra‑long context, complex training pipelines, and custom high‑performance kernels by enforcing batch invariance, a design that guarantees bit‑wise identical outputs across varying batch shapes but incurs lower GPU utilization, reduced small‑batch speed, and added engineering complexity.
Batch InvarianceGPU utilizationLLM Engineering
0 likes · 8 min read

The article uses Zhang Xue’s 20‑year engine‑building journey to illustrate five concrete standards—novel contribution, reproducibility, ablation, impact, and paradigm shift—that separate truly transformative AI papers from incremental work, arguing that rigorous, reductionist engineering can change the world.
Paradigm ShiftReproducibilitynovel contribution
0 likes · 18 min read

This article analyzes a recent study on language‑model reasoning, revealing that reinforcement learning often brings little or no improvement, while evaluation variance caused by seeds, hardware, and decoding settings can dramatically affect benchmark results, and supervised fine‑tuning emerges as a more reliable path.
LLMReproducibilityreinforcement learning
0 likes · 12 min read

The article analyses two months of community attempts to reproduce DeepSeek R1, highlighting that model scaling, high‑quality data, robust training infrastructure, and careful hyper‑parameter tuning outweigh pure reward‑based tricks, and it outlines common pitfalls and future research directions.
DeepSeekInfrastructureLLM
0 likes · 13 min read

Open LLM Leaderboard v2 introduces a revamped, reproducible evaluation framework for large language models, replacing saturated benchmarks with six carefully curated, unpolluted datasets, applying standardized scoring, updating the harness, adding voting and maintainer‑recommended models, and providing richer visualizations to guide the AI community.
AI metricsCommunity VotingLLM Evaluation
0 likes · 19 min read

This article explores the origins, key innovations, and lasting impact of Jupyter notebooks, highlighting how their multi‑language support, interactive computing, reproducibility, and extensibility have transformed data exploration, collaboration, education, and research in modern data science and machine learning.
Interactive ComputingJupyterReproducibility
0 likes · 5 min read

DVC is an open‑source data version control system that enables reproducible, collaborative machine‑learning workflows by tracking models, datasets, metrics, and pipelines across various storage back‑ends while integrating seamlessly with Git and supporting language‑agnostic pipelines.
DVCData ManagementGit-integration
0 likes · 9 min read

DVC is an open‑source data version control system that extends Git to manage large machine‑learning models, datasets, and pipelines, enabling reproducible experiments, low‑friction branching, metric tracking, and seamless collaboration across various storage backends.
DVCML PipelinesReproducibility
0 likes · 9 min read
