Swap One URL and Run Any arXiv Paper on a Single GPU with alphaXiv’s AutoReproduce

AI‑assisted paper reading is entering a new phase, and alphaXiv has launched an autoresearch feature targeting arXiv papers.

Users simply change the word "arxiv" to "autoarxiv" in a paper’s URL; the system then automatically fixes code‑base configuration issues, runs a minimized reproducibility test, and estimates the resources required for a full reproduction.

The accompanying demo video shows the workflow: the system extracts the arXiv ID, searches GitHub for the corresponding open‑source repository, imports it into an AI‑agent development environment, and the user asks the agent to produce a complete, end‑to‑end minimal reproduction.

Upon task start, the agent clones the repository, reads the README, inspects the project structure, and analyses the original experiment’s requirements. It discovers that the full experiment needs four H100 GPUs, runs for about 15 minutes, performs 100 steps, and depends on a specific local data‑set path.

This reflects a common reproducibility obstacle: open‑source code often assumes the authors’ hardware and environment, making it hard for other researchers to replicate results.

To shrink the experiment to a single GPU, the agent devises a "minimal reproduction" plan that includes:

Model replacement: swapping the base model for a lighter version.

Parameter compression: limiting training to 40 steps and checkpointing every 20 steps.

Resource optimization: setting num_processes=1, disabling DeepSpeed multi‑GPU acceleration, and enabling LoRA to save memory.

Script generation: automatically creating or modifying run.sh and summarize_eval.py for execution and log analysis.

The video then displays a side‑by‑side diff of the newly written run.sh and summarize_eval.py, highlighting the concrete code changes made by the agent.

Thus, autoarxiv’s AI agent is more than a paper‑reading tool; it acts as a reproducibility assistant that adapts complex open‑source AI training code to the user’s available compute resources.

This capability benefits both individual researchers and engineering teams: before committing large resources, they can quickly verify that the code runs and estimate the required compute budget.

In a hands‑on test with the 2017 classic "Attention Is All You Need," replacing the URL redirected to autoarxiv’s task page, which promptly identified the corresponding GitHub repository and asked the user to confirm the import.

The system then began importing tensorflow/tensorflow and configuring the environment—a process that, while functional, was noted to be slow.

alphaXiv clarifies that users may employ their own compute resources and agents, allowing integration with private infrastructure while still benefiting from the platform’s automation.

Overall, autoarxiv lowers the entry barrier for reproducing AI papers and enables researchers to run and iterate on papers without needing to provision their own GPUs.