Why Anthropic Refused to Sign Jensen Huang’s Open‑Weights Letter

Jensen Huang opened a personal X account and posted a joint letter that advocates releasing model weights for developers to run locally, rather than announcing a new GPU. The letter asks whether organizations that have trained large models should make the model files publicly downloadable for further task‑specific adjustment.

Companies such as NVIDIA, Google, OpenAI, Meta, Microsoft, Mistral and Hugging Face signed the letter; Anthropic was the only notable absence, prompting online criticism.

Anthropic researcher Julian Schrittwieser replied, stating he does not oppose open weights but questioned the broader implications, mentioning CUDA, Windows and the MCP protocol. Engineer Atharva Ingle responded by listing open projects already offered by the two companies Julian cited: NVIDIA’s NCCL, CUTLASS and TensorRT‑LLM, and Microsoft’s .NET, TypeScript, ONNX Runtime and DeepSpeed.

The letter does not demand that every large model be open; it supports keeping weight release as a normal publishing option for models whose capabilities and risks are well understood. Open weights let developers quantize, fine‑tune or distill models, and enable researchers to fix a version for reproducible testing of capability and safety changes.

Signatories recognize that not all models should stay on the provider’s servers. Their decision to sign rests on three technical judgments:

Assessing whether a model’s capabilities and associated risks make it suitable for open release, rather than judging solely on the binary of “weight open or not”.

Understanding that many safety and behavior questions can only be answered after obtaining the weights, allowing fixed‑version testing of quantization, fine‑tuning or model merging effects.

Separating the model’s weights from the safety system, so that external deployments can still add input checks, output filters, permission controls and anomaly monitoring.

Anthropic’s hesitation stems from concerns that once weights are public, the original safety mechanisms—classifiers, system prompts and runtime guards—can be removed or altered, and fine‑tuning may weaken the model’s refusal behavior. Because security training does not erase knowledge from the parameters, external modifications can re‑enable undesired responses.

While API‑based services allow the provider to push updates and enforce new safety rules, open‑weight models are difficult to recall; older, quantized or fine‑tuned versions may persist indefinitely, making it hard for the original developer to verify continued compliance.

The core technical question emerging from the debate is whether pre‑release evaluation is sufficient to guarantee that a model’s weights can be safely handed over for long‑term external use. The discussion has thus shifted from “should we open weights?” to “which parts of the stack should be open and how do they affect safety and control.”