How Does DeepSeek‑V3.1 Perform on Professional SQL Tasks? A Detailed Benchmark

1. Overview and Key Highlights

In August 2025, the SCALE benchmark incorporated the GPT‑5 family and shortly after added DeepSeek's latest model, DeepSeek‑V3.1 . The report uses a standardized test set to objectively assess the model’s comprehensive ability on professional database SQL tasks and reveal its performance in real‑world enterprise scenarios.

Results show DeepSeek‑V3.1 delivers a relatively balanced performance across three dimensions— Understanding, Optimization, and Conversion . Its strongest area is SQL Optimization , scoring 67.3 points, which supports deep code analysis and system maintenance tasks.

2. Benchmark Dimensions

The evaluation focuses on three core dimensions to ensure vertical comparability and result stability:

SQL Understanding

SQL Optimization

SQL Dialect Conversion

3. In‑Depth Model Analysis

SQL Understanding (Overall Score: 70.2)

Key sub‑metrics:

Syntax Error Detection: 81.4

Execution Accuracy: 70.0

Execution‑Plan Detection: 57.1

Strength: Robust syntax error detection, indicating reliable code‑review capability.

Weakness: Lowest score in execution‑plan detection, revealing limited understanding of deep performance and execution logic.

Horizontal Comparison: Ranked 12th in this dimension, trailing the top model Gemini 2.5 Flash (82.3 points) by 12.1 points, mainly due to lower execution accuracy (70 vs. 90).

SQL Optimization (Overall Score: 67.3)

Syntax Error Detection: 94.7

Logical Equivalence: 78.9

Optimization Depth: 57.8

Strength: High reliability with excellent syntax compliance and strong logical consistency.

Weakness: Conservative optimization depth, lacking advanced, complex strategies.

Horizontal Comparison: Ranked 9th with 67.3 points, behind specialized tool SQLFlash (88.5) and peer model DeepSeek‑R1 (71.6).

SQL Dialect Conversion (Overall Score: 63.2)

Domestic Database Conversion: 100

Logical Equivalence: 71

Syntax Error Detection: 57.1

Large‑SQL Conversion: 25.8

Strength: Perfect score in domestic database conversion, demonstrating strong domain‑specific knowledge and scenario adaptation.

Weakness: Poor performance on large‑SQL conversion and long‑context handling, indicating a critical bottleneck in processing extensive, complex inputs.

Horizontal Comparison: Ranked 13th with 63.2 points, far behind GPT‑5 mini (79.6) and o4‑mini (77.4).

4. Summary and Outlook

DeepSeek‑V3.1 provides a valuable data slice, confirming that current general‑purpose LLMs exhibit both strengths and weaknesses on SQL tasks. It excels in specific scenarios (e.g., domestic database conversion) but falls short on long‑text handling and deep optimization.

Key Insight: Ranking models without considering concrete use‑cases yields an incomplete picture.

5. Future Plans

We will continue tracking cutting‑edge models and publish a detailed evaluation of the highly anticipated professional‑grade application SQLShift . Community feedback is welcomed to help establish an open, transparent LLM‑SQL capability assessment standard.

References

[1] SCALE: https://github.com/actiontech/sql-llm-benchmark

[2] DeepSeek: https://www.deepseek.com/

[3] SQLFlash: https://sqlflash.ai/

[4] SQLShift: https://sqlshift.cn/