The article analyses the open‑weights joint letter launched by Jensen Huang, explains what signing it entails, outlines the technical reasons companies support weight openness, and examines Anthropic’s security concerns that led it to stay out of the pledge.
Researchers from Zhejiang University and Alibaba Security reveal that large language reasoning models can be forced into excessive, self‑correcting reasoning—'overthinking'—by crafted inputs, dramatically inflating token output and computation cost, enabling a novel black‑box DoS attack demonstrated via a Hierarchical Genetic Algorithm.
The paper introduces a black‑box hierarchical genetic algorithm that automatically perturbs the logical structure of reasoning questions to induce excessive chain‑of‑thought in large language models, dramatically inflating output tokens (up to 26.1× on MATH) and creating a DoS‑style resource‑exhaustion attack, with extensive experiments across multiple models demonstrating the vulnerability and its transferability.
The article dissects AI model proxy services, exposing how ultra‑low prices stem from illicit cost structures, how proxies can swap or dilute models, the severe data‑leak and injection risks, and offers concrete red‑flag checks and safer alternatives for developers.
The 2024 Generative Large Model Security Assessment White Paper, jointly authored by the Chinese Academy of Sciences, the Ministry of Public Security's Third Research Institute, and Ant Group's Ant Security Lab, was unveiled at the inaugural CCF China Data Conference, offering a comprehensive review of model risks, ethical concerns, and evaluation methods to guide research, industry practice, and policy making.
