DeepSeek-V4.1-Flash-1017 Open-Sourced: Self-Feedback RL Boosts Agent Performance

DeepSeek-V4.1-Flash-1017 Release and RLSF Technique

DeepSeek has released and open-sourced DeepSeek-V4.1-Flash-1017 , a minor update that introduces a new post-training mechanism called RLSF (Reinforcement Learning with Self-Feedback) . The model weights are available under the MIT license on Hugging Face.

Benchmark Results: Across-the-Board Improvements

Official benchmarks show improvements across all 11 evaluated tasks, achieving a Pareto improvement: higher scores, faster completion, and shorter outputs. The article notes this means "Agent runs the same task, the bill gets thinner directly." Reddit community reactions include "wtf wow" and claims it "obliterates GLM5.3, Kimi K3 and GPT5.6 Sol."

Official benchmark comparison table

RLSF: Reinforcement Learning from Self-Feedback

The core innovation is RLSF, a technique not mentioned in the previous DeepSeek-V4.1-Flash technical report. The official announcement states:

DeepSeek-V4.1-Flash-1017 uses reinforcement learning with self-feedback to improve end-to-end project development — from game creation to 3D modeling.

RLSF originates from a paper by researchers at Heinrich Heine University Düsseldorf: Post-Training Large Language Models via Reinforcement Learning from Self-Feedback (arXiv:2507.21931). The method works as follows:

Reward from the model itself: A frozen model samples multiple chain-of-thought reasoning paths; preference pairs are constructed by ranking each answer's token confidence, then the model is fine-tuned with PPO or DPO — no human labels, ground-truth answers, or external reward models required.

Dual benefit: It fixes calibration (wrong-answer confidence drops from 0.993 to 0.712, eliminating "mystical overconfidence") and improves reasoning (on Qwen 2.5 7B with poor calibration, GSM8K accuracy rises from 53.7% to 58.6%).

Applicability boundary: Gains are largest for poorly calibrated models; the paper only validates single-turn reasoning tasks, leaving multi-turn, long-horizon agent scenarios as open problems.

RLSF pipeline: multiple reasoning paths ranked by answer confidence

RLSF vs baselines on MultiArith/GSM8K: accuracy and calibration error

Same wrong question before/after RLSF: better explanation, lower confidence

The DeepSeek announcement only mentions "self-feedback" briefly; whether it adopts the exact confidence-based reward and how self-feedback is constructed for end-to-end project development remains undisclosed pending a future technical report.

RL Scaling Evidence

The paper's Figure 7 shows that as RL training steps accumulate, performance on DeepSWE, SWE-Bench Pro, and Terminal-Bench consistently rises, and extending context to 1M tokens further improves long-horizon tasks.

RL scaling curves: continued improvement with more steps and 1M context

End-to-End Project Demonstrations

DeepSeek showcases the model building complete projects from scratch:

Forbidden City 3D modeling

Web-based racing game

Godot engine replica of Portal 2

Valley of Ages: 3D scene evolving from prehistoric to modern

Forbidden City 3D modeling

Web racing game

Godot Portal 2 replica

Valley of Ages 3D scene

Infrastructure and Access

DeepSeek Harness must be upgraded to v0.2.1-rc.1 or later.

Search & Fetch API is now fully open at $0.75 per 1,000 calls; the engine already handles hundreds of millions of requests daily internally.

Search & Fetch can be called via Harness, Claude Code, or Codex; free until October 31.

Harness upgrade and Search & Fetch API details

Community Reception and References

Reddit users summarize: "It's the deepseek way." Additional benchmark image suggests the model "crushes Grok 4.7, approaches Opus 5.5 / Gemini 4 Argon level."

Benchmark comparison vs Grok, Opus, Gemini

Post-Training Large Language Models via Reinforcement Learning from Self-Feedback
https://arxiv.org/pdf/2507.21931
https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/tree/main